The Future of Content Creation: Keyword Research on Steroids

Wired this week published a fascinating profile piece on a company called Demand Media: “The Answer Factory: Fast, Disposable, and Profitable as Hell.” The company’s approach to content generation almost sounds like science fiction or satire, but it’s real, and it works. It’s a purely algorithmic, data-driven method of prioritizing content designed to rank on the first page of the Google SERPs: basically keyword research in hyperdrive. And like it or not, this may be where we’re all headed.

Robots will take your content creation job

Keyword-research-driven content production is nothing new; we practice this ourselves (to an extent). Workflow is based on the keyword groups that are currently driving traffic and conversions. If tons of people are finding our site after searching on “keyword organization tools” and we don’t have dedicated content for that keyword, we write a page. It’s just good SEO. Naturally, we want to be writing content that speaks to what our prospective customers are actually looking for.

Demand Media, however, has taken this process to another level:

Pieces are not dreamed up by trained editors nor commissioned based on submitted questions. Instead they are assigned by an algorithm, which mines nearly a terabyte of search data, Internet traffic patterns, and keyword rates to determine what users want to know and how much advertisers will pay to appear next to the answers.

The algorithm, created by Byron Reese, works like so:

To determine what articles to assign, his formula analyzes three chunks of information. First, to find out what terms users are searching for, it parses bulk data purchased from search engines, ISPs, and Internet marketing firms (as well as Demand’s own traffic logs). Then the algorithm crunches keyword rates to calculate how much advertisers will pay to appear on pages that include those terms. (A portion of Demand’s revenue comes from Google, which allows businesses to bid on phrases that they would like to advertise against.) Third, the formula checks to see how many Web pages already include those terms. It doesn’t make sense to commission an article that will be buried on the fifth page of Google results.

Content is then assigned to freelancers who work fast and on the cheap. Content quality takes a backseat to content targeting: According to Reese, “the most valuable topic in Demand’s arsenal” is “Where can I donate a car in Dallas?” That just happens to be their most profitable keyword—no expert could guess that based on intuition or even figure it out with less sophisticated data analysis.

This is super-long-tail targeting—and the effectiveness of Demand’s approach should make us all stop and think. Originally, the company was driving content creation through a combination of algorithmic keyword research and editorial insight:

But once it was automated, every algorithm-generated piece of content produced 4.9 times the revenue of the human-created ideas. So Rosenblatt got rid of the editors. Suddenly, profit on each piece was 20 to 25 times what it had been. It turned out that gut instinct and experience were less effective at predicting what readers and viewers wanted — and worse for the company — than a formula.

Kind of creepy, huh? What’s that clanging sound? Just the sound of robots coming to take your job.

Speaking of creepy, the article also gives us a peek into the life of Richard Rosenblatt, the cofounder, chairman, and CEO of Demand Media. His email signature includes the phrase “Go big or go home.” He’s friends with Lance Armstrong (You might be a douchebag if…). Check out his bio page: He looks like the Rob Lowe character in Wayne’s World!

To be fair, the guy supposedly only seems sleazy:

Numerous executives told me that when they first met Rosenblatt, they were immediately repulsed: He was too slick and seemed to be missing the geek edge. “Then in five minutes you’re like, ‘Holy cow, this guy has it all to back it up,’” says Quincy Smith, CEO of CBS Interactive.

I love that: the geek edge. Give me geek edge over Rob Lowe hair any day.

Bringing it back to search … I recommend reading the piece, and giving some thought to your own content creation priorities and practices. If you think you know what your readers want without digging into the hard data, think again. I.e., watch your back (for robots).

It’s #FollowFriday Time

@Matt_Siltala Owner of Dream Systems Media, Blogger, Speaker, SEO, Internet Marketer including Social Media, Viral Marketing, Branding and Rep. Management
@dizzySEO 100% Natural and Organic
@AlhanKeser Head of marketing @ Blue Fountain Media, ex-triathlete turned Muay Thai beginner, intl traveler, 1/2 Turkish, wine lover, Francophone, & geek (duh).
@FuziJuzi Chief cook and bottle washer at an online agency. Copywriter by trade, tall dark and then some by nature…
@aleyda Web Enthusiast, SEO and SMM
@bosscreative Boss Creative specializes in custom web design, search engine optimization and photograp / video production.
@peterpletsch Full-time Web Analytics professional, SEO/SEM interested, Outdoor Fanatic
@stephendebruyn Integrated marketing strategist, international branding professional. Social media aficionado. World traveler.
@Domingae Affiliate marketer, Blogger, SEO advisor

Follow Us on Twitter

@WordStream Best Keyword Management Platform ever!
@WordStream_PPC WordStream is a dynamic PPC productivity tool that helps search marketers research, organize and act on PPC keyword data.
@LarryKim Avid search marketer, newlywed
@TomDemers New father, PPC wiz, has a great laugh
@SMRichardson Trivia, SEM, Red Sox, animals, travel and music
@Boston_SEO SEO genius, former rockstar, 2 adorable daughters
@EGabbert Writer, poet, editor, SEO, SEM, foodist, aesthete/rationalist
@John_A_Lee Consult on PPC, SEO, Social Media & Blogging. Paid Search Manager with WordStream. Teller of bad jokes, music junkie and aspiring chef.
@Quality_Score PPC pundit (identity unknown), mysterious dispenser of sage Quality Score advice