With the practice of search engine optimisation predating even Google, are there still opportunities out there in one of the biggest markets online?
That’s the question I looked to answer in this breakdown of some of the most successful companies in the industry.
There are some names on the following list you’ll likely recognise.
There are also some notable exceptions, and that’s not a mistake.
You can get a good idea of how well popular brands like DeepCrawl, Screaming Frog and Yoast are doing by their relevant countries Companies House information.
Similarly, when it comes to brands like Backlinko who have a private Facebook Group attached to their $1,000+ training programs, you can count how many members are in the group and work out how many millions said training might have resulted in.
With this report I didn’t want to make many guesses, and I didn’t want to share the financials of businesses who had made no effort to reveal them.
I haven’t tried to include every SEO brand out there – agencies alone would make this list enormous – but rather focused on showing a variety of the ways people are making money.
And because this is Gaps, the second half of this post is about opportunities for capitalising on my favourite industry of all.
Let’s dive in…
Credo (Agency Matchmaking) – $32,000 Per Month
It’s been impressive to watch Credo founder John Doherty go from running the marketing team at HotPads (an arm of Zillow) to starting what is essentially a digital marketing matchmaking service for those looking for quality, vetted service providers.
While Credo isn’t just about connecting people with SEOs, it’s certainly the industry that led to me knowing him and is first in line on the site before those looking for PPC or Facebook ad specialists.
Update: John revealed to me in private he’s generating around $32,000 per month with the site now.
Nightwatch (SEO Software Platform) – $40,000 Per Month
I don’t care how good Nightwatch’s offering is: They have the coolest homepage design of any company in the industry.
In 2018 founder Aljaž reported they were generating $35,000 per month, which has now been updated to $40,000 per month.
I haven’t yet had the chance to give Nightwatch a spin but they have trials available so based on their design alone I’m going to have to pay them another visit.
NinjaOutreach (Email Outreach) – $50,000 Per Month
NinjaOutreach is an interesting one because I actually know their to the minute revenue numbers. However, because they don’t share that publicly – at least not to my knowledge – I’m not going to report on it here.
I would joke that they’re responsible for 99% of the automated emails I get for guest post requests but that probably isn’t even a joke. Still, they’ve done an incredible job of finding what the SEO world wants and bringing an outreach solution that ticks the boxes.
Co-founder Dave Schneider left the company this year under circumstances I’m not familiar with, but not before revealing on IndieHackers that they had passed $50,000 per month in revenue.
Serpstat (SEO Software Platform) – $840,000 Annual Revenue
Ukrainian-based Serpstat describe themselves as “The growth hacking tool for SEO and content marketing” and lay claim to being the most upvoted SEO tool on ProductHunt.
Their platform generated $70,000 in revenue in December of 2017 and $88,000 just a month later. They’re growing, and fast.
In a (now removed) revealing interview with Nathan Latka, founder Artem shared some of the tactics that are helping his company thrive in a competitive marketplace.
GoPBN (PBN Links & Hosting) – $70,000 Per Month
GoPBN was founded by Udit Goenka who, at just 19 years old, had already created a security plugin for WordPress that generated six-figures in revenue.
Udit has built a number of successful ventures since then with GoPBN starting out in 2016.
I was surprised to read the average GoPBN customer is only paying around $24 per month and has an expected lifetime value of $400-$500.
Though this interview went live just last month, I have a hunch the numbers are from 2016. Either way, PBNs are still such a huge part of the SEO landscape that it’s nice to have a public example I can share.
Traffic Think Tank (SEO Community) – $73,000 Per Month
Founded less than a year ago, Traffic Think Tank has taken three big names in the digital marketing world and combined their reach and expertise to put together a Slack group and educational resource for people looking to learn SEO.
The trio consists of Nick Eubanks, Ian Howells, and Matthew Barby. All three have great credentials, but you’re likely to be familiar with Matthew if you know anything about HubSpot, where he’s their Director of Acquisition.
RankScience (SEO Split-Testing) – $80,000 Per Month
Rank Science offers automated SEO split testing, designed to help you tweak a % of pages on your site to find what Google and other search engines are looking for so you can implement winning variations across the board.
Since initially writing this piece Rank Science have received an additional $1.8 million in funding from investors, taking their total to $1.9 million. This not only shows the potential for the brand but also the belief in both their model and execution.
They’re in an interesting space with few competitors (for now) and have some impressive brands on their client roster.
Growth Machine (SEO & Content Marketing Services) – $85,000/m.
A year ago I didn’t know who Nat Eliason was. A year ago his company, Growth Machine, didn’t even exist.
He’s already on track to pull in 7-figures over the next 12 months thanks to an enticing proposition: We’ll double your search traffic in the next three months.
I had a potential client call with one of Nat’s current clients recently – not for a competing service (they’re very happy with him) – so I know there are some very exciting brands helping make up those numbers.
Rankings.io (SEO for Lawyers) – $2.1M Annual Revenue
Chris Dreyer’s Rankings.io is a great example of what I teach when I recommend people go niche with their marketing agency.
The business was founded in 2013, reporting to the Inc 5,000 that they had 20 employees at the end of 2017, with revenues of $2.1M.
The Inc 5,000 are more recently showing their three-year growth figures are a hugely impressive 262% in 2020.
They enlisted Case Study Buddy to help them put together a case study to pull in more clients and had this to say in their feedback,
We’ve closed $179,444 worth of deals in the past month and case studies helped close them all.
Chris is a well-known name in the SEO world so it’s great to see he’s doing so well.
It’s interesting to note that the branding of the business has been tweaked slightly from when we first published this article.
Initially, the header of Rankings.io displayed:
Grow Your Law Firm with First Page Rnakings
However, this now displays:
We help elite personal injury law firms dominate first page rankings
Which feels hyper-niche in comparison to their previous focus.
So whilst this focus from law firms to personal injury lawyers is subtle, it does once again highlight the potential value of really niching down.
gShift (SEO SOftware Platform) – $2-5M Annual Revenue
If it weren’t for Moz founder Rand Fishkin mentioning gShift on Twitter a few years ago I would never have known they existed. Fortunately their revenue numbers aren’t so shy.
gShift describe themselves as an “SEO and content optimization platform” and placed #305 on Canadian Businesses G500 Awards, reporting annual revenues of $2-5M.
They were happy to report on that ranking with their own press release, which is confirmation enough for me that the numbers – while broad – are accurate.
The business has also received a total of $3.5 million from investors across four funding rounds. Showing the belief there is in gShift and their business model.
RankWatch (SEO Software Platform) – $2.3M Annual Revenue
Their average customer spends $125 per month to get actionable insights on rankings, backlinks and their competitors in the search results.
Their homepage states they have 25,000 active customers in 25 countries, though just last month they stated they have 1,500 paying customers. I’m not totally sure where the 25,000 number comes from, since they don’t appear to have any kind of free offering.
Either way, it’s surprising to see huge agencies like iProspect on their clients page, so they’re clearly doing something right.
Linkdex (SEO Software Platform) – $2.86M Annual Revenue
In the middle of 2016 the self-dubbed SEO platform for professional marketers, Linkdex, was acquired by ScribbleLive. Though the numbers weren’t revealed at the time, we can get some insights from this press release. At least part of the payments for Linkdex amounted to £1.2M (~$1.56M).
Craft notes their revenue for the year was $2.86 million. They don’t source their numbers, but further research makes me confident they’re pretty accurate.
I was unfortunately unable to find any numbers on Authoritas but on a personal level, I love finding big companies in this space that I had no idea existed. After winning in their category at the 2018 European Search Awards, they’re certainly one to watch.
Authority Labs (Rank Tracking) – $3M Annual Revenue
Their software, which primarily focuses on rank tracking, consisted of just 4 employees helping support more than 2,000 paying customers.
I don’t know about you, but I think it’s great to see a company so focused on one thing still doing so well.
So well in fact, that in 2017 the business joined Traject suite along with SERPS.com and ZoomRank. They now have a team of more than 60 employees.
PlumberSEO (SEO & Website Design) – $3.3M Annual Revenue
If you read my Rankings.io description you’ll know I’m a huge advocate for going niche if you’re to offer SEO services so I had to include one of my favourite examples of that: PlumberSEO.net.
See, you can still have success even when the dot com is taken.
Reporting to Inc.com they revealed their 2017 revenues were $3.3M, with three-year growth of 78%.
One of the main reasons going niche can work so well is potential clients instantly think, “Yep, that’s the company for me.”
They’re limiting the number of companies they can work with, but their close rate is likely far higher because of it.
BuzzSumo (Content Analysis Platform) – $450,000 Per Month
If I’m being truly honest, I wouldn’t truly class BuzzSumo as an SEO brand but they are such a popular research tool for coming up with linkworthy content ideas that they get a lot of press in the industry.
It’s not hard to find their revenue numbers being thrown around but for this post, I wanted information from the source.
Fortunately, Brandwatch founder Giles Palmer, who purchased the BuzzSumo brand, revealed in October 2017 it was generating $450,000 per month.
The story behind Brandwatch is even more interesting. They’re definitely too focused on social media to get their own section in this article but are generating more than $50M per year.
Link Research Tools (SEO Research Platform) – $5M Annual Revenue
Christoph Cemper is one of the more interesting characters in the SEO world and to be honest with you, I don’t think he gets as much recognition as he deserves for what he’s created at Link Research Tools.
In a July 2017 interview with Grant Cardone of all people, Christoph revealed that his link analysis platform was generating around $5M per year.
Perhaps more interestingly is that he stated this revenue comes from just 195 customers, meaning the average customer value is around $25,000/year.
StoneTemple (SEO & Content Marketing Services) – $9.2M Annual Revenue
You may not be familiar with the name Stone Temple, but if you’re in the SEO space you’ve likely heard of its founder, Eric Enge.
In the 2017 Inc. 5000 list, Stone Temple’s revenues were recorded as being $9.2M for the year.
We’ll likely never know their figures again, since they were acquired by Perficient in July of 2018. An announcement that took many in the industry by surprise.
ThirdDoorMedia (SEO News & Conferences) – $9.4M Annual Revenue
Though the numbers I’m about to reveal are slightly older than I would like, I think it’s important to showcase a wide variety of success stories in the SEO world and ThirdDoorMedia certainly help with that.
You may know them by their popular media brands like Search Engine Land and Marketing Land, or their marketing conferences (primarily for SEOs) such as SMX.
Founded by Danny Sullivan – who now works at Google interestingly enough – Third Door Media revenues were $9.4M for the year 2015.
These numbers are likely higher now, especially since revenue had grown by 53% over the previous three years.
The Hoth (Link Building Services) – $10.4M Annual Revenue
They have raving fans, over 300,000 customers, and vocal critics, but whatever you feel about link building services from the Hoth, one thing is clear: They know how to sell them.
Hoth, which stands for “hit ’em over the head” was founded in 2010 and later acquired by Next Net Media who also own Backlinks.com.
A January 2018 blog post highlighted the journey that helped them sell over $1M in SEO services in a single month.
In 2017 they sat at number #561 on the Inc 5,000, reporting earnings of $10.4M. A more recent report in 2019 showed they had achieved a three-year growth of 461%.
Whilst the most recent 2021 Inc. 5000 report highlights this continued success with rise of 116% growth over the previous two years.
ChatMeter (Local SEO Platform) – $10-12M Annual Revenue
On LinkedIn Chatmeter describes itself as a local SEO platform that “helps enterprise retail brands and agencies managing multiple locations increase their revenue”.
In an interview with Nathan Latka, Chatmeter founder Collin Holmes revealed the company spends $80,000 per month on paid advertising which has helped them grow to $10M in annual recurring revenue.
They’re clearly proud of their status as an ‘Official Yelp Knowledge Partner’ and are primarily focused on helping companies manage their local listings.
StatusLabs (SEO Reputation Management) – $10.4M Annual Revenue
Look your best in search results reads the attention-grabbing headline on StatusLabs’ homepage.
Self-reporting their 2017 revenue at $10.4M with three-year growth of 245%, and a further 112% three-year growth in 2019, they’re clearly in a lucrative space.
In a time where Google search results can mean everything to people it makes sense their client roster is made up of politicians, CEOs and athletes.
Go Local Interactive (Local SEO Services) – $12.1M Annual Revenue
Go Local Interactive dub themselves as a full-service digital marketing agency but when the first three call to actions on their homepage are focused on SEO it’s fair to say it’s a priority for them.
Based out of Kansas City, Kansas, the seven-year-old company reported to Inc that their 2017 revenues were $12.1 million.
This is their second year in a row on the Inc. 5000 and with having grown to over 70 employees, it’s clear they’ve found their groove when it comes to client acquisition and retention.
Ayima (SEO & Digital Marketing Services) – $18M Annual Revenue
Thanks to Ayima the Nasdaq graphic at the top of this article actually makes sense: The digital marketing and SEO agency were accepted on Nasdaq First North on September 4th, 2018. In fact, the graphic is actually from Ayima.
Due to being a public company, each quarter they must share their financials for investors to look over. These numbers are reported in SEK (Swedish Kroner) due to where the company was founded.
With their Q1 2018 revenue coming in at around $4.5M, I simply multiplied that to estimate their 2018 numbers, though you can also easily see exactly how much they made in 2017 as well.
They had been on quite an acquisition spree, so it’s no surprise to see their revenue for 2020 has risen to over $16M, with over $10m profit.
Ahrefs (SEO Software Platform) – Est. $40-50M Annual Revenue (Verified: $40M ARR)
You’ll notice in the rest of this post, I don’t like to make estimations.
In fact, only the numbers for Ahrefs are estimated in this entire post. Because they’re both so popular in the world of SEO and often mentioned in the same breath as Moz I didn’t want to leave them out.
I know this number is lower than some people will estimate. After all, the founder of the Moz predicted they were generating $40-$80M two years ago.
I’ve had the honour of working with Ahrefs once-off via Detailed and I consider myself to be fairly close with the company, but I have zero insider insights on their revenues (I’ve never asked, and I doubt they would tell me if I did).
On their homepage they reveal how many trial signups they get each week, and based on those numbers we can make some estimations but take my figures with a grain of salt.
The closest we’ve got to any insights from them is when they publicly admitted they’re an 8-figure company but that wasn’t much of a surprise to anyone in the industry.
Update: I received some criticism for this prediction – at least on Twitter – but three weeks after this article went live, Ahrefs CEO Dmitry confirmed that their ARR is $40M.
SoloSEO (SEO Tools & Software) – $48M Annual Revenue
If I know this industry as well as I think I do, SoloSEO is going to be one of the links in this article that gets clicked on the most. Many SEO veterans won’t even have heard of this SEO provider that’s doing such big numbers.
How they’re doing such numbers is much more interesting to me.
Ever sign up at a shared hosting company and been offered SEO as an add-on package for a monthly fee? That package might just be getting handled by SoloSEO.
SoloSEO pays their hosting partners anywhere from 50-60% of the Per Months to include their product as an upsell during registration flows.
That helps explain why their average revenue per user is just $20 (they previously disclosed in a now removed interview), and why they’ve managed to mostly fly under the radar of the SEO world’s biggest names.
I have to imagine a few SEO providers will be reaching out to their contacts at hosting companies after learning about this one.
BrightEdge (SEO & Content Marketing Platform) – $50M+ Annual Revenue
It’s hard for me to like BrightEdge. They started (and lost) a four year legal battle with Searchmetrics who they claimed were infringing on their SEO-related patents. The five BridghtEdge patents up for discussion were later invalidated.
My personal feelings aside, BrightEdge have built something that a lot of companies are happy to pay for. Their platform, which they describe as “driving the convergence of SEO and content marketing” is relied on by some of the biggest brands in the world.
Growing to over 400 employees in 2018, their annual revenue is now estimated to be around $80M.
Moz (SEO Software Platform) – $60M Annual Revenue
Moz are undoubtedly one of the most loved brands in the SEO world. Not just because of the content they put out but for the transparency they provide regarding their business operations as well.
This transparency extends to their finances, where they’re happy to report on revenues of $41.1M in 2017. An 11% growth over the previous year, having raised $29.1M via multiple funding rounds to help make those numbers happen.
This revenue growth continued into 2018 as it grew to $60M, with an expected $70M ARR in 2019.
Their founder Rand Fishkin recently left the company to build another startup and is hoping to replicate his success, recently raising $1.3M in an unusual round of funding.
Update: Moz was purchased in June 2021 by iContact, an email marketing company and subsidiary of J2 Global.
It looks like there are big plans for Moz going forward, and you can read more about the acquisition here.
As of yet no financial data regarding the acquisition has been released, but we will update this post when that information is made public.
SEMRush (SEO Software Platform) – $188M Annual Recurring Revenue
Offering a wide range of tools to help you reverse engineer the success of your competitors, SEMRush appear to be everywhere these days.
On the back of raising $40M this year, they’re putting on in-person events from Australia to Finland and everywhere in-between.
Like Ahrefs, they were always fairly secretive about their actual revenue numbers, with their fundraising press release stating that they’re “trusted by more than 2,000,000 marketing professionals worldwide” but didn’t say much more than that.
However, a release of their figures for 2020 shows they made just short of $125M in annual revenue (up from $92M in 2019), with gross profits of almost $95M.
Interestingly, we can also see that they spent a huge $54.5M on marketing to achieve these kind of mighty sales figures.
There was a 57% year on year growth for 2021, which saw annual recurring revenue grow to $188M.
It’s also worth noting that as of September 2021, SEMRush was serving an incredible 79,000 customers. This is an increase of 12,000 on the 67,000 customers they had in March 2021.
The fact this figure was just 54,000 in December 2019, it really does show how the business has gone from strength to strength.
Yext – $298.8M Annual Revenue
While many may no longer view Yext as just an SEO company, seven of the twelve solutions they present to clients are still SEO-focused.
With a primary focus on local SEO, they’re the biggest company I’m aware of in the entire industry, generating $170.2M for fiscal year 2018. Their gross profit was a staggering $126.1M.
Their updated report for 2020 shows their fiscal year revenue climbed to $298.8M, which was a huge 31% increase on their 2019 figures.
This revenue increase resulted in an annual 30% rise in profits for the business, with their 2020 gross profit figures achieving an incredible $221.8M — which leads us to believe their impressive growth shows no signs of slowing.
Besides the companies mentioned in the introduction who I purposefully didn’t share the revenue numbers of, there were a lot more companies I looked into that unfortunately haven’t made any public announcements regarding their revenue numbers. At least not to my knowledge.
Some companies I would have loved more info on include:
- Ryte (claim 600,000+ users)
- Raven Tools (acquired by Tap Clicks so likely impossible)
- WebCEO (ZoomInfo claims $60.4M revenue but don’t show sources)
- Conductor (recently acquired by WeWork)
- Semper Plugins (creators of All in One SEO Pack)
If you work at one of these companies – or any other big SEO brand – and want to share your numbers, I would happily add them to this post.
this is what we’re known for
Gaps in the SEO Market
— Noah 🤙🏼 (@noahpbrinker) April 15, 2020
Below this box is the text we’re famous for, but out of respect for you, we do have a disclaimer in place.
We spend dozens of hours preparing these reports and coming up with opportunities you can capitalise on, but we also don’t want to put your life-savings into an idea just because we wrote about it.
For that reason, we have an $8.88 request: Please don’t spend more than that testing out an idea (it’s the cost of a .com on Namecheap) to see if it has legs and makes sense for your business.
We’ve made many successful predictions and even ran our own case study, but we’ve also invested time and money into ideas that didn’t pay off. We’re not directly making money from this report, but still want to be respectful of your own finances.
Finding Gaps in an $80BN/Year Industry: My Take
If Search Engine Land trust a source then that’s good enough for me. Said source predicted that by 2020 the SEO industry would be worth $80 billion dollars.
That’s just in the US alone. And it’s only focused on SEO as a service.
It’s fair to assume the industry is worth hundreds of billions if we include software providers (and the rest of the world), but Borrell Associates appear to be the only ones who have done the research so $80 billion is the best number we have.
Though the above list is just a small sample of companies that are succeeding in the SEO world, it’s pretty clear that the companies which make the most revenue are typically reliant on implementing services or providing software.
Many transitioned from one to the other, such as Moz which started as an SEO consultancy before making the switch.
Few do both.
As Gaps isn’t just about reporting on success stories but also about finding gaps that you could potentially capitalise on, I’ve done a ton of research before sharing my findings.
I’m splitting up the following sections just as I have so much to say.
The 10 Niches I Would Offer Services In
If this isn’t your first visit to Gaps, you probably know that one of the first websites I ever put on the internet was ViperChill.com. I was 16 years old and quickly fell in love with the idea of being able to “control” Google rankings and genuinely dreamed of being paid to do that for other people.
Here’s how the site looked thirteen years ago:
My dream took a long time to become a reality though, and one of the main things I put that down to is trying to help anyone and everyone. I wasn’t focused on who I wanted to help, or how I wanted to help them.
I ended up giving up on my dream of selling SEO services until I realised that knowing how to sell was just as important (if not more so) than trying to become the world’s best SEO. A weird goal I had at the time. I say weird because well…there’s no way to determine who is best (and it doesn’t really matter).
If I’m being totally honest, I think it’s clear that ‘general’ agencies which don’t focus on a niche can make a lot more money than specialised ones. Stone Temple, featured above, are a good example of that.
The downside I’ve found though is that they tend to have offices, presence around the world and have a sales team who know how to get connections and broker big deals.
That’s not the kind of thing I ever wanted to be involved in.
Going niche gives you the freedom to be anywhere, and makes it less likely for potential clients to just focus on companies on their doorstep.
Update: The training is no longer online. Thanks to all who emailed!
SEO Software: All-But-One Market I Still See Potential In
The business of providing SEO software is extremely competitive and in many cases, extremely expensive to even get started in.
Thanks to Moz being so open about their revenues we can see they spent $8.7 million on hosting and data providers alone for 2017. That’s not something a newcomer to the space can compete with very easily.
Not forgetting they’ve raised $29M to help them get to this point.
Even SEMRush, who are likely much bigger than Moz, raised $40M this year to help grow their operations. While rumours suggest they probably didn’t need the money, it will still be spent with the aim of pulling away from the competition.
That said, Munich-based Ryte give us a bit of hope. If the numbers on their site are to be believed, they’re growing at an incredible pace.
Last year they raised €3.1M ($3.59M) which, while no small amount, is tiny compared to their competitors with similar offerings.
Companies like ScreamingFrog and Sitebulb have been able to carve out dominance with more focused offerings – they’re primarily used as website auditing tools – and remain self-funded whilst they do so.
As with many industries, it seems the more you want to offer and the bigger you want to be, the more you’re going to need to invest.
But, you can focus on a smaller segment of the market – like BuzzSumo did with content – and dominate the space with cheaper offerings than the all-in-one providers.
So what do I think the best gaps are when it comes to SEO software?
I don’t mean to be boring, but I don’t think you need a gap at all.
In fact, I think your best chance of success in such a huge market is just to model what people are clearly happy to pay for already.
There are hundreds of rank trackers.
Dozens of tools to find domains for your private blog network.
More site audit tools than anyone could wish to use.
If they can exist and thrive now, I’m sure there’s room for one more.
I don’t want to be too anti-climatic, so let’s dive into a real world example of a sister-industry proving my point.
There’s More than One Buffer: There’s Room for One More SEO [Blank] Tool
One of my favourite examples of this modelling in action comes thanks to the massive success Buffer found with their social media scheduling tool.
With their live revenue dashboard showing $1.5 million in monthly recurring payments, their public success has likely inspired a few competitors.
Laura Roeder, who I was following for years before she came out with her own tool, MeetEdgar, has found success with a similar offering.
In just thirteen months her software was already generating over $150,000/m. A big part of that is undoubtedly down to her marketing skills, but Buffer helped educate an entire market on how powerful and time-saving social media scheduling can be.
After all, Buffer were rejected from YCombinator with their idea and were pulling in just $280 a month when they started out.
Plann, a scheduler focused on Instagram, generated 7-figures last year. It was only launched the year prior.
It’s natural to assume that Tailwind are easily making 7-figures annually after proudly claiming they work with over 700,000 brands (was 200,000 when this article was first published) with their Pinterest and Instagram scheduler.
I could go on, but you hopefully get the idea.
To have success in the SEO software space, you don’t have to be the first:
- SEO split-testing solution
- SEO plugin for WordPress
- Keyword research tool
- Site audit tool
- Local SEO management platform
- Rank tracker
The market is so huge – potentially anyone who wants more website traffic – that I honestly think you can just build the next one.
New ideas still exist, such as Little Warden who fairly recently came out with a smart client tracking system for agencies, but they’re few and far between.
Speaking of tracking things, I couldn’t end this post without one more prediction.
Unpopular Prediction: Google Will Limit Prominent SEO Software Providers from Crawling Their Search Results
I have a hunch that Google will start to crack down on the most popular companies which profit from tracking their search results. I would be surprised if it happens in the next year or two, but I do think it will happen.
It’s not the most likely event, I admit. After all, Google have essentially been allowing companies to profit from scraping their data for a very long time.
It would also be incredibly ironic. They’re the biggest scraper on the web. And they ask you to opt-out before they start crawling your site, rather than opt-in.
That said, it’s just a hunch of mine. I have no problems with being wrong.
If it happens, it certainly won’t mean the end of SEO software providers: The majority of their valuable data (at least to me) is from crawling websites themselves, rather than Google.
I also can’t help but think their rank tracking features will only become less valuable anyways due to changing SERPs, but they’ll also be asked to stop.
It wouldn’t be the first time a big company has taken such an action.
Instagress used to be one of the most popular Instagram bots out there and they were specifically targeted by the company.
Here’s how their homepage looks today:
Google’s John Mueller has held no punches when talking about those who track Google’s data, referring to one of the companies featured in this article as “blackhat scrapers”.
Or maybe it's just blackhat scrapers having a hard time?
— 🍌 John 🍌 (@JohnMu) August 17, 2017
Outside of rank tracking, even the idea of so many different companies crawling websites is starting to get a negative reputation; perhaps deservedly so.
Do we really need so many link indexes focused on crawling the entire web?
Joost de Valk had an eye-opening thread on this over on Twitter.
We store all our logs in Elastic, through Logstash, and analyze them in Kibana.
We have to talk about this. Some of these results are shocking.
— Joost de Valk (@jdevalk) May 30, 2018
If you’re looking for a gap in the market, that’s the thread for you. The replies to his final tweet might spur some great ideas for the developers reading this.
Joost’s comments came after a controversial article by Moz on the “link index olympics” which covered, in part, which SEO software tools were blocked from crawling by the most websites.
If the idea of blocking crawlers to help save hosting costs and energy ever becomes a trend, it wouldn’t kill link research tools, but it could certainly make them less valuable.
Back to my original point, the amount of data extraction people do on Google’s search results clearly isn’t unknown to them. The last comment in the picture is from Google’s John Mueller.
Even more recently, John replied to an article on the topic from Search Engine Journal.
Is it kinda awkward that you're publishing a guide on how to break our terms of service & webmaster guidelines? 🤔
— 🍌 John 🍌 (@JohnMu) September 5, 2018
Of course, you can’t expect just one Google employees opinion would lead to a company-wide decision, but John’s thoughts may be voicing what is being discussed internally at the company.
Just like with Instagram, of course, some companies will fly under the radar and try to cash in if big names ever do lose some of their biggest features.
As far as Gaps are concerned, this will either be an incredible opportunity for newcomers if that does happen, or a sign to stop developing competing tools altogether.
Of course, as I mentioned above, the irony would not be lost on me if Google ever make such a move.
If you want to succeed in offering SEO services, there are some niches which are growing fast.
If you want to succeed with software, I genuinely think your best bet is to replicate what is already out there and doing well.
And if you want to carve out a new space, opportunities do exist, but you’ll likely spend a lot of time educating the market to your way of thinking.
Gaps is going to undergo some massive changes over the next few weeks which I’m pretty excited by – and can’t wait to share with you – but that’s all for now.
Thank you for reading!
Like this post?
Clicking the heart button below lets me know what kind of content you would like to see more of. You can also join our SEO Facebook Group to discuss it.