Raven’s CEO on Ahrefs and choosing to end use of scraped data


SEO tool providers used to operate in a gray world with both authorized and scraped data. That opportunity no longer exists.

Everybody using and building SEO software will have to choose what’s most important – because no one provider can have it all.

Following our announcement on Friday, Ahrefs announced that it will also end the use of scraped rankings in January. I commend them for making what I know is a difficult decision.

Raven Tools and Ahrefs will not be the last software providers to make this decision. Other well-known SEO software companies have reached out to me for counsel and to commend Raven for making what they deem a brave choice.

I see three camps forming:

  • Those with authorized data, like Raven and Ahrefs.
  • Those who provide scraped data.
  • And those trying to straddle the line and say they have both authorized data and scraped data.

I can understand why companies choose to offer scraped data. But the undeniable current demand for rankings carries some undeniable future risk for those companies. As Raven’s CEO, I’ve lived through more rankings nightmares than I care to explain.

The companies still trying to operate in the middle are going to get hit on both sides. I don’t understand that business model.

This means customers will have to vet their software providers more carefully; evaluate the accuracy and reliability of data closely; and possibly rethink their investments. Eventually, no provider will have both authorized and scraped data.

If Google would sell this rankings data, I would bring back the SERP Tracker tomorrow. I don’t expect that to happen. So, Raven and our talented team moves on to what’s next.

We’ve been listening as the industry reacts to Raven’s decision, and we’ve posted much of the feedback on our website. Here are a few pieces that I recommend for their overall focus on where the industry is headed.

  • Majestic SEO: “The Importance of Trusted Data
    Majestic SEO is a valuable data partner in Raven. The perspective here – on who to trust with the data you use to run your business – is an important one.
  • Webpresence: “Google Clips Wings of Raven Tools
    Despite its title, this post is thoughtful about Google’s potential rationale and impact on SEO.
  • Scott Offord: “What is the Future of SERP Tracking?
    I appreciate that Scott pulled in perspective from both sides of the debate.

You can see more headlines at raventools.com/scraped-data.

I appreciate all the feedback we’ve gotten. Knowing what I know today, I would still make the same decision. I believe Raven will be a better company in the long-term – and a better value for all of our customers.

Still, I wanted to further address two big questions we’ve heard more often than the rest.

Why is Raven abandoning SEO?

We’re not.

We’re exiting the scraped rankings business. It is a calculated risk that we have considered from almost every angle.

Raven will not be of value to anyone whose business depends exclusively on scraped rankings. It’s understandable if those customers need to seek an alternative. We recommend Authority Labs and Advanced Web Rankings.

We also accept the challenge to create new metrics for SEO campaigns. It’s not enough to authorize Google Analytics in Raven. When we exit the scraped rankings business on Jan. 2, we will face a measurement void in our product.

Two immediate projects are already in the works:

  • Accessing more Webmaster Tools data through Raven
  • A new SEO Report module focused on organic traffic

The rest of Raven’s SEO tools remain and will continue to be an integral part of our value proposition, including:

  • The powerful and soon-to-be-improved Research Central
  • CRM, for managing link outreach and relationships
  • Keyword Manager
  • Keyword competition and volume data from AdWords
  • Link Manager
  • Link Monitor
  • Backlink Explorer

We’re also putting final touches on a Website Auditor for on-site SEO. It will be released in January.

Why did Raven choose AdWords data over SERPs?

It was not a choice of AdWords versus SERPs.

Raven had to decide between keeping scraped data or dropping it from our platform. Our decision factored in the short-term risks, the long-term risk and commitment to our vision.

Short-term risks

The short-term risks of keeping scraped data were minimal. We would have to add significant costs to get scraped AdWords keyword data for research, but the SERP Tracker would still function with scraped rankings. And many of our customers would remain happy.

Long-term risks

The long-term risks of keeping scraped data could be devastating. For reasons unknown, Google is strictly enforcing the AdWords API ToS. And we don’t know what might be next.

What if Google sends cease and desist letters to every company selling or reselling scraped rankings? Where would we — or any SEO, for that matter — get the data then?

What if Google goes after companies who are scraping AdWords data and selling it as authentic? Google charges customers like Raven for access to its AdWords API. Third-parties scraping AdWords data and reselling it, in effect, cost Google money by taking away potential business. Will they go nuclear when they find out?

What if Google added “no scraped data” to the Google Analytics API ToS? Given the recent AdWords enforcement, this scenario is plausible. And scraping for analytics data isn’t feasible.

These scenarios were speculative.

But if we chose scraped data in the short-term, we might have to give it all up — Google rankings, Google AdWords and Google Analytics — in the long-term, anyway.

Long-term vision

The final consideration was our long-term vision.

From 2008 to 2010, Raven was almost exclusively an SEO tool. So much so that our domain was raven-seo-tools.com.

In 2010, after we changed our name to Raven Internet Marketing Tools and domain to raventools.com, we began to diversify our offerings because we predicted a change in the industry – a consolidation of Internet marketing practices coming together under one roof.

This meant adding social media tools, PPC tools, better data partners and more. We continued to build SEO tools, like Research Central, a better Link Manager and the new CRM. We switched our social media data provider to uberVU and added content analysis from Scribe by Copyblogger. Here’s a timeline of Raven’s product evolution. And we built a reporting engine on top of all of it.

Continuing with scraped data was not possible and would put much of that in jeopardy.

In the end, I made the difficult decision with the full support of co-founders Jon Henshaw and Scott Holdren.

Patrick E. Keeble
Co-founder, President and CEO

  • http://searchmarketingwisdom.com alanbleiweiss

    Patrick, I applaud you for the difficult business decision the team must have gone through. Not unlike the (not provided) data, tracking rankings on a mass scale may have been beneficial, yet all too often I’ve seen it become shiny object chasing. And while I think Google would be wise to make a revenue model for non-AdWords clients getting all that data again (not-provided and rankings on scale), the fact that such an offering doesn’t exist at this point actually helps people like me, who specialize in deep audits. We get to spend less time explaining why clients need to lose the shiny object syndrom, and more time guiding them on more productive use of their energy.

    • RavenCourtney

      Glad to hear from you, Alan. Thanks for your smart thoughts.

  • art

    Understood, but still want the SERP tracker.

    • RavenCourtney

      We’ve tried to make it clear why that isn’t an option for us. If ranking are important to you and your clients, we recommend Authority Labs or Advanced Web Rankings.

  • http://www.thedsmgroup.com/ Jason Diller


    Well done… I think you guys have made the right call and I applaud you and everyone on your staff for being so professional during this period.

    Yeah, I have clients…and yeah…they won’t like it…

    I think you guys should write a blog post or host a webinar on how to explain this to our clients / bosses / etc.

    Now THAT would be good content.

    Cheers my friend.

    • RavenCourtney

      Jason, those resources are on the way!

  • http://www.facebook.com/jemenglert Jeremy Englert

    Really excited to hear more about the new organic search tools! We’ve always used organic rankings as a way to gauge success and only recently started using the SERP tool – so the loss of that tool was not that big of a deal when compared to the loss of the organic data pulled from SEMRush, such as organic rankings and organic costs.

    However, the news of a new Organic Search module is awesome! Looking forward to it.

    • RavenCourtney

      Glad you’re excited, Jeremy! We are, too.

  • uberVU

    Patrick, we’re glad to be your social media data partners and totally agree with your stance on no data-scraping! Thanks for the shout-out.

  • Brian Fitzgerald

    Can’t wait for the new SEO metrics you’re working on. Organic traffic measures are so much better than rankings

  • Paul

    If we have a separate account with Authority Labs for ranking data, is there any way of automatically pulling in that data to a Raven Tools report?

  • http://www.yardstickservices.com/blog Kevin McLeod

    Hi Patrick, this is a much better communication and I personally appreciate the different perspectives and alternatives offered. Thank you.

  • Michael Drisko

    you had to have this happen. Have you contacted a lawyer on this matter? The
    Google Adwords API is under FTC investigation and this is the exact kind of
    matters that would interest them. While you are doing what you have to do,
    Google is forcing your hand to make your product less attractive to marketers –
    and I am sure this is going to affect your revenue in the short term.

    Google Adwords has a lot of reasons to protect their business by limiting data
    that exposes adjacencies where marketers can spend their dollars – like SEO.
    If you put rankings next to paid data, you can start to see that there are
    places you may not need to spend as much with Adwords. They want to stop this,
    so they say it messes up their server logs. The truth is, it’s very easy for
    them to not let crawlers mess up their logs.

    It’s great your product is now Google compliant, but it’s at the expense of
    freeing data for marketers and your revenue. I strongly urge you to hire a lawyer and to alert the FTC. Also contact your senator. This is the exact kind of stuff they are looking for
    – the big guy crushing the little guy with unfair business practices.

  • Webarts

    I must say that considering the cost raven tools are already providing to us great value. We use an agency account even if we dont fully need it. I believe raventools make a correct decision even if im very sad having the serps gone. I would have to explain my clients next month why they didnt receive their ranking report. i am looking forward to see whats coming next from raven tools. Personally i would expect more on page analysis tools and more marketing intellegent reports. I realy hope this decision not cost you but instead help you growth

    • RavenCourtney

      Thanks for the kind words. We hope you’ll be excited when you hear more about the direction we’re moving.

  • jonathan

    A far more circumspect post than the initial announcement blog post. Rhetoric improving.

    But even better Machiavellian giggles can be heard direct from Google’s horse’s mouth these days: http://www.smh.com.au/business/world-business/google-on-tax-schemes–its-called-capitalism-20121213-2batw.html

    Death n Taxes … certainly Sir InterWeb!

  • http://www.bacarat.si/ Porocni Prstani

    Thanks for great and in deepth explanation. Anyone knows how something about link-assistant-com ? Do they really have own spider cappable of finding relevant links as they say ?

  • Spook SEO

    as you said “no one can have it all” but so far I think you guys have done a great job providing reliable tools for us

  • RavenCourtney

    Brian, we hope to report back soon with more details on what we’re working on.

  • RavenCourtney

    Thanks for the great social media intelligence and the support, uberVU!

  • RavenCourtney

    No, there is not. Providing a way to import rankings data is also against Google’s terms of service.