Evaluating whether theming content is an important part of Google’s algorithm.

I have been studying search engines for several years, and built quite a few tools for my private use. One of the things that has really stood out over the last couple of years is how well themed the pages are in the Google SERPs. Everyone talks about inbound links as the main ranking factor, but I disagree. Links are really important, don’t get me wrong, but the content on the page has seen a revival in terms of being a ranking factor, especially since the recent Panda update in early 2011. 

Inbound links can be manipulated – everyone knows it, and everyone does it, and Google knows this. They therefore had to find ways to ensure that the content they are showing their searchers is highly relevant to the search phrase typed in, and what better way than to evaluate the words, and combinations of words on the page to decide if something is a good, relevant, quality fit for a search term?

Over the years, Google has acquired the technology to read content in a more human way. Now think about this. When someone writes a great piece of content, they will use words that are “niche-specific” to the topic of that content. If they are writing about dogs grooming, they will write about shampoo, brushes, combs etc. That’s obvious isn’t it? So isn’t it obvious that Google would be looking for sets of words related to a searchers query to help determine relevancy of a page in their index?

I mean if someone searches for dog grooming at Google, is it not a given that Google will look for pages in their index that contain words like:

Dog, coat, hair, fur, groom, clippers, brush, nails etc, etc

I have used the software code that drives the theme reports in Web Content Studio to create a feature for my own personal use that goes through the SERPs and measures themeing on pages so as to return average values for a range of things, like Theme Score (this is a proprietary formula used in Web Content Studio) and % of theme words used on the page. My program will allow me to get averages for groups of pages on autopilot, meaning I can do a lot of testing fairly quickly.

There are certain types of pages that I remove from the equation – things like youtube videos which are known to rank quite well because of who owns youtube, plus PDF files. I have also removed any double listings from these results so that only the first ranked page on a site (in a Google double listing only) is the one that is used in the study. The second ranked page is omitted.

Here is how I am going to find the theme words for each case study:

I identified 41 theme words, and 13 Theme Phrases that are asscociated with the search term h1n1 vaccine.  Here they are:

Theme Words:

information, vaccine, flu, 2009, h1n1, influenza, dose, people, virus, 2010, against, health, nation, cdc, year, children, pregnant, swine, vaccination, vaccines, die, cause, test, vaccinated, care, nasal, report, protect, safety, women, related, disease, doctor, treat, control, ages, government, medical, administer, source, drug

Theme Phrases:

disease control and prevention, seasonal flu vaccine, swine flu vaccine, swine flu, h1n1 influenza, pregnant women, influenza virus, h1n1 vaccine, nasal spray, health care, h1n1 virus, injectable vaccine, nasal vaccine

Results for the top 70 pages in Google averaging out the pages in groups of 10:

NOTE: Numbers in brackets after the SERP positions indicates how many pages were included in the average.  If its less than 10, then that is because the page was not accessible when my software went to download it.

Google Retuned around 600 results in the main index.   Analyzing batches of 10 results from sections of those SERPs yielded the following results:

OK, now this is another completely different niche, so let’s see whether the pages ranking for this term are themed.  Google only gives 486 pages in their main index that they see as relevant and unique to this search phrase.

Firstly, Web Content Studio initially collected a bunch of relevant theme words and phrases that I narrowed down to 147 theme words and 50 theme phrases.  While my testing does not include the theme words, 147 theme words is too many so I got Web Content Studio to tell me which of them are the most important.  I ended up with 43 theme words:

Theme Words:

resume, job, employer, write, writing, format, view, letter, cover, sample, date, interview, best, resumes, first, rate, top, work, line, page, tips, professional, web, examples, fit, process, perfect, program, information, service, employers, executive, sign, written, contact, level, tech, how, make, out, builder, most, search

Theme Phrases:

making a career change, thank you letter tips, person doing the hiring, fill in the blanks, how to write, write a resume, thank you letter, state zip code, resume writing guide, resume writing service, cover letter writing, type of position, make a resume, effective resume writing, cover letter, resume sample, prospective employer, resume writing, stand out, resume examples, executive resume, resume builder, functional resume, chronological resume, phone number, most important, effective resume, resume format, contact us, career change, job hunt, job title, writing guide, perfect candidate, job search, letter tips, manager resume, zip code, great resume, cover letters, writing service, interview tips, letter writing, resume distribution, very important, young person, job market, body language, job interview, resume formats

So let’s look at the top 69 pages, and whether or not they use the theme phrases I’ve chosen.

and those a wider range of pages extending through Google’s chosen set:

For this term, Google only shows 630 unique pages in its index.  However, after 100 or so results, a number of the pages shown become quite off topic.  Does this mean the theming will drop off as we move through less relevant results?

Actually if my theories are correct that theming is one of the important factors in ranking pages, then no.  If Google returns 630 pages it believes are relevant to this search query, then all of the 630 pages should be themed with a core set of the same words and phrases. 

Google only has 1300+ exact match pages for that term, which they narrow down to around 60 pages that they feel deserve to be in the search results for the exact term “Why do earthworms come out in the rain”.

Theme Words:

earthworm, worm,  out, rain, soil, surface, ground, water, ring, cat, thin, body, rains, end, oxygen, fact, moist, under, environment, heavy, site, skin, form, crawl, drown, eco, animals, burrows, related, below, air, high, sure, come, through, their, after

Theme Phrases:

out in the rain, come to the surface, surface of the soil, out of the ground, out to the surface, travel to the surface, above the ground, through their skin, amount of oxygen, male and female, heavy rain, their skin, social sciences, carbonic acid, after rain, blood vessel, earthworm reproduction, worms surface, carbon dioxide, moist environment, worm composting, blood vessels, male gonopores, reproduction process, worms breathe, worm cocoons

... and the results?

First lets look at how well the top 69 pages are themed.  Most of these are highly relevant to the search term.

.. and if we look at batches of 10 pages in various areas of the SERPs?  Remember, this is where theming should stay high if my theories are correct.

OK; now I know what you are thinking.  Those results down in the 400 – 409 range are only have a theme score of 36.6 %.  Well, remember that the Theme score is an algorithm built into Web Content Studio, so is not precise.  Its there to give an idea of how well themed the articles are.  You can see from the % theme words used that even these pages ranking 400 – 409 used over 60% of my 39 theme words!  That’s quite impressive I think.

Since we started out this by considering the theme words associated with Dog Grooming, let’s do it as a final case study.

Google actually only returns 516 web pages that it considers to be “the most relevant results”.

Using Web Content Studio I narrowed down my initial list of 112 theme words to just 48 theme words.  Here they are:

 Theme Words:

bath, breeds, brush, brushing, care, clean, clippers, coat, cut, dog, dry, dryer, ear, eyes, face, first, groom, groomer, grooming, hair, head, health, help, how, important, leg, lie, mat, nails, neck, owner, pet, rate, remove, rinse, shampoo, side, sit, skin, start, table, tail, tips, tools, trim, use, vet, water

I won’t bore you with the 61 theme phrases I found for this topic, since those are not used in the theme calculations anyway.

So let’s look at the theme analysis for this search phrase against my chosen theme words.  First the top 69 pages:

.. and a wider spread of data through the relevant search results:

OK; something is going on here.  Look at those pages in the 300 – 409 range.  Only 35 – 39% of theme words used.  Does that through my theories out the window?

Well, actually no.  I think this is perhaps more of a Google issue because when I looked at the pages in that area of the search results (those used in this analysis), I found several wedding related pages.  Why weddings?  Well I assume it was because I had the word “groom” in my search phrase “How to groom a dog”.  Perhaps some of those pages in Google’s Serps use the words “groom” and “dog” to describe someone at the wedding J

I think the results of these tests (and many more I have conducted on a range of other keyword phrases) provides a lot of evidence that Google is looking at word combinations on the page to determine relevancy. 

For naturally written content by an expert, this theming of the content is occurs naturally, because the author NEEDS to use the theme words to accurately and adequately write the article.  

NOTE: To help those suffering in the aftermath of Google's latest changes, I have updated my Creating Fat Content Course (150+ meaty pages) and made it available for free. Download Creating Fat Content for 2011. This book shows you how and why you should create quality content.

You can also read a couple of other Hubs that I have written on this subject called Google Competition and Google "Authority Sets" and Writing Themed Web Content.

Finally, if you have any comments on this hub, or any of my other stuff, please leave a comment at the end.