Hey White Hats, Automation Is A Good Thing…Stop Whining

NOTE: I’m absolutely not advocating automated link building or blogger outreach or anything related to such practices in this post. While I’ve done these things in the past (and quite honestly, they work really well…if done correctly), that’s not the point of this post. I’m focusing on automated data collection for SEO purposes only.

Let’s get started.

Just because spamming is automated doesn’t mean that automation is spamming.

Why am I yelling? Because I am annoyed.

I learned SEO from affiliate marketers, so my viewpoint on SEO processes is biased towards automation compared to most of the mainstream guys. In internet marketing, most SEOs don’t bat an eyelash at automation; it speeds up your processes, giving you more free time to do more work that will ideally make you more money.

Now, automation can take many forms, ranging from automated Twitter feed and Google Alerts monitoring to aggressive black hat link spam. Somewhere in between falls ideas like automated data collection (such as what we do at serpIQ or what every single rank tracker in the world does) and crowdsourcing (such as what Kevin suggests in his blog post about collecting blogger contact information using Mechanical Turk).

SEO is a fairly contentious field when it comes to deciding what is OK to do and what is just spammy spam spam, so there is a good amount of FUD floating out there unfortunately. The FUD is created and spread by people who don’t actually understand the techniques they’re criticizing, and that’s not fun for anyone.

The reason I decided to write this post was specifically because of a response I saw on Inbound.org to my friend Kevin’s blog post. I’ll keep the response anonymous in this post.

First, let’s look at Kevin’s strategy

  1. Gather a list of blogs you’re interested in reaching out to. You can use sites like Alexa or Technorati to find high quality, popular blogs for this step
  2. Create a job at Mechanical Turk and ask the Turkers to collect all of the contact information they can find for each of the blogs in your list
  3. Go do real work instead of mindlessly copying and pasting data into spreadsheets

Simple, efficient, extremely smart.

Now for the response:

“”High quality blogger outreach”? No way. You get what you pay for. It’s more like the good old “submit to 1000 search engines” kind of SEO automation.”

It took me a few minutes to recover from my head exploding, but once I gathered myself I decided to address this very naive White Hat FUD here. This trend bothers me so much that I’ve decided to yell my response with very large, bold text:

There Is No Value In Manually Collecting Any Data That Can Be Automated

I see so many SEO guides floating around that advocate manually collecting all kinds of information, from contact information to broken links, and they almost always consist of digging around the internet for the information and copying and pasting…and copying and pasting…and copying and pasting. This is creating an entire generation of SEOs who have no idea what good automation is and causes the FUD train to keep chugging along.

Here’s the problem with these manual data collection methods: Once you know what the data means and where the data comes from, you gain no benefit from manually collecting it in the future.

Let’s write that one more time for good measure:

Once you know what the data means and where the data comes from, you gain no benefit from manually collecting it in the future.

There’s simply no point. You might feel like you’re doing a lot of work because you’re toiling over your laptop, filling in cell after cell with blogger Twitter handles, but the reality is:

  1. You’re not getting better at your job. No matter how many times you paste a Twitter handle into a spreadsheet, you won’t ever become a better SEO. You might improve your ability to press CMD + V, but who gives a damn.
  2. You’re wasting precious time. Whether you work on your own campaigns or do client work, you’re simply wasting your time doing manual labor that can be automated. If I ever hired an SEO and they told me they spent 6 hours manually compiling a list of 100 bloggers to contact, I would fire them on the spot. The data is out there, don’t be a dummy and collect it efficiently.
  3. Google is an automated system. They crawl the internet in a fully automated fashion, normalizing, indexing, and ranking information. You should be doing the same, otherwise you’re trying to breach a castle wall with a hammer.

If you’re ever faced with a task that is both normalized and repeatable, you should be looking for ways to automate it. Now, a lot of automation tasks require a decent amount of coding skill, I realize this. However, with the emergence of such tools as Zapier and IFTTT, in addition to software like uBot, you can develop quite an impressive set of automated tools that can be used for monitoring and data collection, with very little technical skills necessary.

Automation saves you time.

Automation makes you money.

I see so many White Hat SEOs dismissing anything even remotely resembling automation, and quite frankly, it’s an irresponsible thing for them to be doing. Automation is why we have Google in the first place, otherwise we’d still be clicking manually collected links in the Yahoo Directory. By spouting off about how some technique is spam when it isn’t, you’re hurting the SEO community at large.

And because of this, I am going to personally call out anyone who makes such claims. That’s right, I’m coming for you if you decide to unfairly dismiss a useful technique as spam. Disinformation helps no one, and we need to get it under control in the SEO world or else the opinion of SEO will just keep getting more negative.

There are a lot of completely appropriate ways to automate your job. Look for them and use them as much as possible, otherwise you’re just being inefficient.



  1. My guess is that “white hats” talk down automation to hide the fact that automation can be used very effectively in scaling efforts.*

    *Shhh. It’s a secret. Don’t let anyone know. They’ll catch up if they do.

    1. I agree. I honestly think some white hats dismiss automation because they know that if they automated the work they do everyday, they’d really only be doing like 15 minutes of real work as opposed to hours of copying and pasting. The lengths I’ve seen people go to to get a single backlink are just disgusting sometimes.

    2. I use this kind of automation and consider myself on the whiter side of grey. But as Tyson says, shhh… else they’ll catch up. 😉

  2. Damn this post has really made me think of all the useless shit I do on a daily basis that I could be outsourcing or automating. Real light bulb moment for me, thanks.

  3. So true in many different aspects.
    I tend to try automating just about everything I can and it perplexes me how people act like they against the concept of automation. They remind me of the plight of the luddite.

    PS. How much wood would a wooddchuk, dchuk, if a wooddchuk could dchuck wood?

    Disclaimer: This comment was created by an automated link-builder

  4. The example you’re referring to is mostly outsourcing, and not really automation though. He still advocates manually collecting that list he sent to Mechanical Turk to find contact information. I agree though that you don’t learn anything from copying and pasting all day long, and outsourcing tedious tasks can save you time and energy that can be used in your other efforts. I just don’t subscribe to any service, program or tool that promises SEO through automation because it’s hugely risky for your brand.

    1. Hey Jeremy,

      The SEO automation tools I’m referring to here are actually the types of tools that your company (Raven Tools) and mine (serpIQ) handle, such as automated rank tracking, backlink collection and analysis, social monitoring, competition analysis, etc. I’m not advocating any sort of automated rank building systems or anything like that here (nothing that creates links automatically, or emails bloggers en masse, etc).

      Essentially, I’m encouraging the automation of any tasks that free up time for you to do more good SEO practices rather than wasting time manually performing solved problem tasks.

      Thanks for the comment!

  5. If something you do can be broken down into a repeatable system that someone else (a person on Mechanical Turk) or something else (software/app) can do, then it’s somewhat “automatic”.

    I’m not advocating automated link building, but even if you had someone else do manual link building, it’s still “automatic” in the sense that you’re not doing it yourself, but it gets done.

    Automated link building is mostly automation at scale. Whether it’s “good” (as in quality) link building and useful, that’s a different story (think Xrumer, etc).

    Anyway, you can use Google Docs to build really simple, but effective, scrapers that collect information.

    Why would I waste time and energy going to websites to manually copy-paste information that I can scrape? Doing so is not more “legit”, it’s stupidity.

  6. Great article Darrin – like the inclusion of Kevin’s article too!

    I have been leveraging this sort of automation for years, and frankly could not have built what I have had it not been for these platforms.

    Jeremy (I think) is eluding too bots and scripts. I have grown to depend on platforms like MechTurk and unfortunately have shelved any of the former. Do you have any advice for those looking to build bots and scripts? Best places to find talent for such tasks?

    Thanks Darrin!


Leave a Reply

Your email address will not be published. Required fields are marked *

Optimization WordPress Plugins & Solutions by W3 EDGE