6 SEO Experiments That Will Blow Your Mind

In SEO, there’s no shortage of theory and best practices. But experimentation is what really keeps this great industry moving forward.

All of today’s best practices came as a result of past experiments – both failures and successes. Everything we do in SEO is an opportunity to learn and improve.

That’s why I love doing SEO tests and experiments. While Google may reveal a few bits and pieces of information, they’ll never tell us everything we need to know to evolve our SEO strategies from the ordinary into the realm of unicorns.

Today I’d like to share six mind-blowing SEO experiments we did this year, what we learned from them, and what it all means.

1. Does Organic Search Click-Through Rate Matter?

We know that machine learning – including Google RankBrain – is changing SEO as we’ve known it. Already, Google uses RankBrain for every search, and it impacts “a lot” of queries.

Is RankBrain (or other machine learning-based elements) impacting rankings? If so, how? Well, that’s exactly what wanted we wanted to find out: what’s a good click-through rate for organic search.

For this experiment, we looked at a set of 1,000 keywords from Google’s Search Console for the WordStream site. What we found is very interesting. In April, the average CTR for the top position was 22 percent. That increased to 24 percent by July and 27 percent by September.

So our data clearly showed that are top ranked results had its highest CTRs by September. Meanwhile, on the other end, the data showed that lower positions (4-10) were being clicked on less than ever.

Clearly, the click curve is bending. I believe this is exactly what you would expect to see from a machine learning algorithm – it’s about providing the best answers (based on the highest user engagement), which means fewer people will need to scroll down and click lower results.

So does organic search CTR matter? YES! More than ever before.

2. Is Organic CTR an Organic Search Ranking Factor?

So we’ve established CTR is important. But what’s the relationship between organic CTR and organic search rankings?

Numerous people who work at Google have said they don’t use click-through rate for the purposes of ranking. But when they say this, they mean they don’t use it as a “direct” signal. Could it be that clicks have an indirect impact on Google’s search results?

We decided to find out the answer to this question with an experiment designed to figure out whether CTR impacts SEO rankings.

The goal of this experiment was to see whether there was any clear relationship between organic search CTR and organic search position. The biggest challenge was that separating CTR and ranking is like separating Kanye West from his ego.

So we attempted to isolate the natural relationship between CTR and ranking by taking the difference between an observed organic search CTR minus the expected CTR:

After looking at our data, we found that, on average, pages that beat the expected average organic CTR for a given position were far more likely to rank in the top four positions. These are unicorns! For example, a page that beats the expected CTR for a given position by 20 percent will likely appear in position 1.

Also, pages that failed to beat the expected organic search CTR were more likely to appear in positions 6–10. These are donkeys. For example, a page that falls below the expected CTR for a given position by 6 percent will likely appear in position 10.

So, based on the data, does CTR impact organic search rankings? It certainly seems that way!

3. Can Rewriting Your Titles Boost Your CTR?

So if you’ll be rewarded for having a higher click-through rate, what’s the best way to raise your CTR?

If people see nothing else, they will see your headline in the SERPs. Your content may be totally awesome – but they won’t click on it if the headline is boring.

SEO has evolved. It’s silly to write title tags like it’s still 2008!

WordStream has been trying to move away from overly “optimized” “SEO titles” like this one: “Guerilla Marketing: 20+ Examples and Strategies to Stand Out.”

That old headline followed “SEO best practices.” The most important keyword was at the front and everything fell within 60 characters. But it’s kind of a snore, right?

So we ran a little CTR optimization experiment. Our content and SEO manager Elisa Gabbert changed only the title of the post – to “20+ Jaw-Dropping Guerrilla Marketing Examples.” The new headline is closer to this super-successful headline template that foregrounds the list format, the emotional impact and the content type (examples):

The article text, images, links, or anything else you can think of were left untouched.

After updating the headline, the article CTR increased to 4.19 percent (up from 1 percent) and it ranked in position 5 (up from position 8).

So can you increase your CTR just by changing your title? Yes!

Don’t be boring! Write brilliant headlines that people will click on like crazy. (Just make sure the content behind them backs them up.)

4. Do Website Engagement Rates Impact Organic Search Rankings?

It’s super important to create clickable headlines, but the goal isn’t just to create clickbait. You also must have great engagement metrics. If people feel cheated and go right back to the SERP, Google can detect that.

Dwell time is really the thing that matters. And time on site is a much better proxy for dwell time than bounce rate.

My theory is that Google uses dwell time (which we can’t measure, but is proportional to time on site) to validate click-through rates. These metrics help Google figure out whether users ultimately got what they were looking for (i.e., a successful search).

So do engagement metrics (bounce rates, time on site, conversion rates) impact organic search rankings?

To put this theory to the test, we gathered some engagement rate data. First, we looked at whether the bounce rate of the pages/keywords we rank for had any relationship to their ranking:

See that “kink” in the graph? Kind of hard to miss, right?

Landing pages that had a bounce rate below 76 percent were more likely to show up in one of the top four positions. But landing pages that had a bounce rate of 78 percent or higher were more likely to show up in position five or lower.

What about time on site?

This graph shows that if your keyword/content pairs have decent time on site, then you’re more likely to be in top organic positions 1–6. If engagement is weak on average, however, then you’re more likely to be in positions 7 or lower.

And how about conversion rates? This data shows that higher CTRs tend to lead to higher conversion rates:

Why is this? Because if you can get someone excited enough to click on your offer, that excitement typically carries through to a purchase or sign-up.

Higher CTRs, engagement rates, and conversion rates lead to more leads and sales. But I believe this data clearly shows proof that improving engagement metrics and conversion rates will also lead to better organic search rankings.

5. Do Engagement Metrics Impact the Selection of Featured Snippets?

Google’s Featured Snippets, which appear in so-called Position 0 above the organic search results, come in the form of text, lists, images, and charts, among others. But how does Google’s algorithm pick Featured Snippets?

First, I wanted to find out whether Google’s traditional organic search ranking factors impacted whether your site gets snipped. So I looked at data for 981 snippets that the WordStream site has earned.

Clearly not. Otherwise, the top ranked position would get the snippet every time. Google is featuring snippets from content that ranks on page 2 to as far back as the 71st position!

Having on-page copy that is clear and concise is also clearly important. But, again, word count isn’t the full picture.

So we dove deeper and investigated this page after seeing it as a snippet for searches relating to getting Bing Rewards points. We discovered two interesting things from our Google Analytics and Search Console data:

• An unusually high CTR (21.43 percent), even though it had an average position of 10
• Unusually high time on site (14:30), which was 3x above the site average.

So do engagement rates play a role in the selection of Featured Snippets? I absolutely believe so!

6. What’s the REAL Relationship Between Organic Rankings & Social Shares?

We’ve heard about the ridiculously high correlations between social shares and organic search rankings for about five years now (see the ranking factor studies done by SearchMetrics and Moz). Many people have assumed that social shares are a ranking signal, even though Google shot this down every single time.

My belief was that it’s not the visible social share counts that matter. What’s more important is having high social engagement.

So we tested it out to find the real relationship between organic rankings and social shares. Here’s what we found:

This data showed that Facebook posts with super high engagement rates (above 6 percent) also had an organic search CTR that beat expectations. In other words, if you have Facebook engagement that is 4x higher than average, you’ll have an organic search CTR that is 4x higher than average.

Why is this? Well, I believe that the same emotions that make people share content on social media also make people click on those same pieces of content when they see them in the SERPs. This is especially true for headlines with unusually high CTRs.

The correlations were much stronger with unicorn content (those “blockbuster” pieces of content that drive 10-100x more traffic to your site than most of your other donkey content put together). Unicorns with high social engagement rates almost always had high organic CTR, and vice versa.

The correlations were substantially weaker with donkey content. Donkeys sometimes had high engagement rates, sometimes low engagement rates. The same was true with CTR, some high, some low.

So yes, high social engagement rates correlate with high CTR, and vice versa. That’s the real relationship between search and social. It’s all about how engaging your content is!

What Does It All Mean?

As these five experiments highlight, SEO is continuing to evolve in a way that rewards your pages and site based on how people engage with your content.

That means it’s mission critical to optimize for engagement.

In other words: Optimize for PEOPLE! Write headlines that will make them click and then reward them for that click by publishing amazing and memorable content that will make them want to stay on your site and share your stuff.

Check out all six experiments summarized below:

Did any of these experiments surprise you? What SEO experiments would you like to see next?