These aren't what you think

Notice anything odd about these two web results? They've got Google's new author profile pictures in, right?

How you normally make those photos appear

Well, no. Normal websites have to go through a slightly complicated process to get photos of the author of a page showing up in Google's web results (the old process, the new process). This basically involves cross linking your Google profile and either the page in question or to an "author page" that the page in question also links to.

There's a "by Malcolm Coles" link next to the photo

But when you do that, Google's results show not just a photo - but also a link to your Google profile.

And this link to the Google profile is missing from the screenshot above (but you can see it in the picture here).

So what's going on?

It turns out that those two web results from the Guardian aren't showing photos because of Google's normal process for making them appear.

There are no links from the author profiles on the Guardian to the journalist's Google profiles. And their Google profiles don't link to their author pages. (Here is Charles Arthur's author page on the Guardian and his Google profile. Here is Jemima Kiss's author page on the Guardian and her Google profile.)

However, both those author pages link to the journalist's twitter accounts - as do their Google profiles. And it appears that Google has decided that on the basis of this, it can presume that the photo on their Twitter profile picture is a picture of the article's author - and is showing the photo.

I think we can assume that this isn't going to work for most sites ... Thanks to @yesiamben, @ysekand, @danbarker and @yoast for helping work all this out yesterday.

But it turns out that you can dominate Google's search results in the short term via duplicate content - and here's the explanation, based on a talk I'm giving at Brighton SEO.

Look at all the traffic I got

Here's a graph that showed how I got loads of traffic to this blog by exploiting duplicate content. Don't look any further if you're easily offended or love the royal family.

WOW. Where did all that traffic come from? Where did it go?

I took this blog from a few hundred page views a day to 30,000 at its peak by exploiting the fact that Google was looking for new content when it comes to suddenly popular or brand new search terms. Then I stopped and I went back to normal levels.

I'm sorry to report those search terms involved Pippa Middleton's arse and underwear and super injunctions and twitter.

So I waited for something to happen that I knew would trigger lots of searches. After the Royal Wedding it was Pippa Middleton's arse. Then a few days later it was her underwear. Just as that died down, the whole Ryan Giggs / Twitter / superinjunction thing kicked off.

Each time something happened I would throw a short blog post together and publish as quickly as possible - my ambition each time was to beat the mainstream news sites to publishing something.

Then I pulled a trick - for a few stories, I republished the story shortly afterwards on a different URL and 301ed the first URL to the second one. Then I did it again.

(This trick only works on trending news topics - so search terms that are suddenly popular and which make Google think that it should throw away its usual search results and replace them with pages that have only just been published (to correspond with the new interest in the search term).)

Look, two results for the same content ...

Not sure what I'm on about? Well, look at this picture.

Step one: two results for the same content.

Hopefully at the bottom you can see that I have two URLs for my Pippa Middleton underwear pictures (NOTE: there are no pictures of this kind. She's in swimwear - blame the tabloids).

What I did

I took the first URL which was http://www.malcolmcoles.co.uk/blog/pippa-middleton-underwear-pictures-google-updates-autosuggest and I changed it to http://www.malcolmcoles.co.uk/blog/pippa-middleton-underwear-pictures-google-updates-autosuggests - the same URL but with an S on the end. I also tweaked the HTML title a bit (took the word its out).

How WordPress helped

Now a quirk of WordPress means that it tries to match partial URLs and will redirect accordingly. What this means is that if you tried to access the old -autosuggest URL you would get automatically redirected to the new -autosuggests one.

Try it with this URL: http://www.malcolmcoles.co.uk/blog/pippa-middleton-underwear-pictures-google-updates-auto and see where you end up. That's not a manual redirect, it's WordPress doing it automatically.

How this fooled Google

What Google sees is as follows:

• It sees some content on a subject everyone is searching for but no on has published on lately.
• So it shoves my result on page one.
• Then it sees I've published another URL.
• It's still desperate for new content so it gives me 2 results.

Of course, the 2nd one is 99% the same as the first one but (1) it hasn't noticed that yet and (2) it hasn't revisited the first URL to see that it redirects to a new one.

Now I've got three!

But if I can pull that trick once ... Here's another screenshot.

Hat trick

Yes, I've now got three results on page one. I'd changed the URL again. This time from -autosuggests to -autosuggests2. I didn't bother changing the title.

Again, Google is still desperate for new content. Everyone is searching for this term because American sites published the pictures. But very few UK sites have. So it gives me three results.

As a searcher, however, if you click on the old -autosuggest or -autosuggests URLs, which no longer exist, then WordPress will just match those to what it thinks you're looking for which is -autosuggests2. So you always end up at the same place.

Boo. Back to two.

Eventually, Google wised up to the fact that the first URL was 301ing (or was maybe the same, who knows). And it took me back to two results.

Back to two

As you can see, the first -autosuggest has now vanished.

Eventually, all the duplicates were filtered out and my current URL is now top of Google for a search on her undies. Oh good.

But for a crucial few hours, when loads of people were searching, I had the same content several times on page one of Google's results.

Conclusion

One side effect of this is that people think I'm a pervert. That aside, for spiking news terms you can insert the same piece of content into Google's results. It will eventually notice that the earlier URLs redirect or are duplicates of the later ones. But by that time everyone will have stopped searching.

To be honest, you probably shouldn't do this. It breaks Google.

You also need to be on the ball. You need to beat big news sites (if you're dedicated, this is possible). And it only works on spiking news terms (these news data tools might be handy).

And don't cock it up. Duplicate content is bad. Ask the Express. Or the Indy.

This is the last of my posts or conference talks on Pippa Middleton (here's my earlier talk on Karen Gillan's underwear). I hope you enjoyed them. Don't blame me for the fact that nation is a bunch of perverts.

In the past, they used to be shoved over on the right hand side (EG see the screenshot here). Now they are integrated into the search results - with a bit prominent link to my google+ profile. See this screenshot of a site: search for this site (I see similar on other sites, too).

Author pictures integrated

Anyone else see this? You can add author pictures by following these instructions (which were recently simplified).

Here's one example.

No results, just refinements when I search for Kentish Town

Here's another.

No results, just refinements when I search for Hyde Park

Or maybe it's a bug!

They have one version of their headline in their HTML title (what Google shows in its web results) and news sitemap - this includes "Pippa Middleton arse" at the front: "Pippa Middleton arse: Motorist paid price for bum comments to girlfriend". This is also the URL they use: http://www.dailymail.co.uk/news/article-1386280/Pippa-Middleton-arse-Motorist-paid-price-bum-comments-girlfriend.html

But the version of the headline on their page just says "'Is Pippa's bum still better than mine???' How motorist with Middleton envy paid the price for comments to girlfriend".

Here's how their story looks in Google.

Saucy

And here's how it looks when you get to the Mail.

No arse in the on-page version

Cheeky, one might say. They haven't gone as far as optimising for Pippa's underwear yet (and they need to do a bit more as the caption on the last picture says "write caption here").

Update: They've removed the word arse! New screenshot:

Booooo

Pah.

To infinite URLs - and beyond

Using rel = canonical properly

You use the rel=canonical command to tell Google that a given URL is actually a version of another URL - and that the search engine should treat the second version as if it was that main URL.

It's useful if you have multiple copies of a page in different directories, have lots of versions of the same page due to EG WordPress making 2 versions of every page, or allow anyone to rewrite your URLs so it looks like your insulting Pippa Middleton's sister.

Make a mistake with rel=canonical, however, and it can wipe your website off the face of the internet.

Using rel = canonical to make infinite URLs

The Express site's CMS is creating a duplicate version of every single page via the rel=canonical tag. And then a 3rd version, and then a 4th ... and it's never stopping until it gets to infinity.

Take a sample page like this one: http://www.express.co.uk/features/view/244786/AV-referendum-Why-we-must-vote-NO-to-the-new-voting-system

If you look at the HTML code, you can find:

<link rel="canonical" href="http://www.express.co.uk/features/view/244786/AV-referendum-Why-we-must-vote-NO-to-the-new-voting-systemAV-referendum-Why-we-must-vote-NO">

The CMS has miscoded the canonical URL to include the first bit of the URL relating to the individual page (the AV-referendum-Why-we-must-vote-NO bit) twice.

If you visit that supposedly canonical URL, you see this, with the page-specific bit in there three times.

<link rel="canonical" href="http://www.express.co.uk/features/view/244786/AV-referendum-Why-we-must-vote-NO-to-the-new-voting-systemAV-referendum-Why-we-must-vote-NOAV-referendum-Why-we-must-vote-NO">

Go to that URL, and you find it there 4 times. Etc.

I got bored at http://www.express.co.uk/features/view/244786/AV-referendum-Why-we-must-vote-NO-to-the-new-voting-systemAV-referendum-Why-we-must-vote-NOAV-referendum-Why-we-must-vote-NOAV-referendum-Why-we-must-vote-NOAV-referendum-Why-we-must-vote-NOAV-referendum-Why-we-must-vote-NOAV-referendum-Why-we-must-vote-NO

but this will never stop. Each time you visit the canonical URL, a new canonical URL is created.

All these URLs are working pages because the Express only looks at the number in the URL to decide what content to show. So http://www.express.co.uk/features/view/244786/AV-referendum-Why-we-must-vote-NO-to-the-new-voting-system is the same as http://www.express.co.uk/features/view/244786/vote-YES is the same as http://www.express.co.uk/features/view/244786/who-exactly-specced-this-CMS.

Dozens of URls for each Express story

Sometimes these duplicate canonical URLs aren't in Google's index (I guess as each one is cancelled out by the next one). Although you can find them. This search, for instance, has this URL showing up: http://www.express.co.uk/posts/view/242092/DEBATE-Is-Britain-a-soft-touch-for-benefit-spongers-DEBATE-Is-Britain-a-soft-touch-for-benefit-spongers-DEBATE-Is-Britain-a-soft-touch-for-benefit-spongers-DEBATE-Is-Britain-a-soft-touch-for-benefit-spongers-DEBATE-Is-Britain-a-soft-touch-for-benefit-spongers-DEBATE-Is-Britain-a-soft-touch-for-benefit-spongers-

Even worse, the first URL that appears for that search is the printable URL of the page with no adverts on!

One paragraph, 55 results ...

And as that search, with 55 results, reveals, the Express has a massive problem with duplicate content.

The Express then makes the problem even worse ...

This is a problem it makes worse via its use of Tynt to add URLs when you copy and paste content. So if you copy and paste the first sentence from this URL: http://www.express.co.uk/features/view/244786/AV-referendum-Why-we-must-vote-NO-to-the-new-voting-system, what you end up with is this:

"BY the time you read this you will have probably already voted No to AV in today’s referendum.

Read more: http://www.express.co.uk/features/view/244786/AV-referendum-Why-we-must-vote-NO-to-the-new-voting-systemAV-referendum-Why-we-must-vote-NO#ixzz1LW2s00ge".

The Express uses Tynt to add the read more bit and the URL to what you've copied.

But, yes, the code they are adding contains the wrong URL with two versions of the page slug. Follow that link and copy a sentence and you end up with this:

"BY the time you read this you will have probably already voted No to AV in today’s referendum.

Read more: http://www.express.co.uk/features/view/244786/AV-referendum-Why-we-must-vote-NO-to-the-new-voting-systemAV-referendum-Why-we-must-vote-NOAV-referendum-Why-we-must-vote-NO#ixzz1LW31La3e"

Yup, another new URL created by the system that's designed to channel links to the main story.

You can see this in action on this page on the Daily Mail where someone has copied the opening para from some other bat shit story, and the Tynt URL is to http://www.express.co.uk/posts/view/244206/EU-wants-to-merge-uk-with-franceEU-wants-to-merge-uk-with-franceEU-wants-to-merge-uk-with-france#ixzz1LCIcD5jI.

This might explain why the Express can't rank in first place for a paragraph from its own story.

To sum up

The Express isn't appearing top of Google's results for searches using their own content and Google is serving up versions of its pages with no adverts on - all because Google can't work out which page is the correct one because the Express constantly points to yet another URL for every single page - even the made up ones.

My head hurts.

Image credit.

Pippa Middleton's arse

Google Insights gives you a unique insight into how people search the internet. An often depressing insight ...

I can reveal that by Saturday, the day after the Royal Wedding took place and the most recent day for which Google is currently showing data as I type this, searches for "Pippa Middleton arse" were running at 72% of the volume of those for "Pippa Middleton dress".

I'll keep the graph below, updated ... (there's another 3% relating to Pippa Middleton ass, not shown in the graph).

Interest in the dress has peaked but is still rising in the arse.

The good old British people.

Pippa Middleton's arse is suggested

Interestingly, it's treating ass and arse as synonyms and showing results from pages using either.

(Update: A I now appear to be the top result for a search on "pippa middleton's arse". But not ass. Which is nice.)

I've written before about the problems Google Instant has with deciding what to filter and not filter when it comes to rude words. Maybe arse isn't as rude in the USA ... (Update: interest in Pippe Middleton's arse vs dress)

So this story (about a jelly bean with Kate Middleton's face in) can also be reached here with a URL of http://www.independent.co.uk/life-style/food-and-drink/utter-PR-fiction-but-people-love-this-shit-so-fuck-it-lets-just-print-it-2269573.html (Update - this rude version has had 1,476 retweets compared with 33 for the original version!)

People are busy changing this to anything they like on Twitter. Google has already indexed the two versions:

Sweary URLs ...

How many more?!?

Update: While writing this post, a third one had been indexed at http://www.independent.co.uk/life-style/food-and-drink/you-can-say-anything-you-like-in-this-bit-including-read-the-guardian-instead-of-the-indie-2269573.html.

More to come I'm sure ... You can test for yourself how many Google has indexed by running this search (Google is at least recognising them as duplicate URLs).

Update 2: Google has the sweary version on page one of its results when you search for kate middleton jelly bean. (And the sweary version is first for a search on Kate Middleton bean)

Update 3: There are now 12 URLs indexed for this story, including the following:

• http://www.independent.co.uk/life-style/food-and-drink/benjamin-gray-is-the-true-heir-to-the-throne-2269573.html
• http://www.independent.co.uk/life-style/food-and-drink/OMFG-i-can-write-ANYTHING-I-WANT-in-here-and-create-FAKE-TWITTER-MEMES-2269573.html
• http://www.independent.co.uk/life-style/food-and-drink/Load-of-old-wank-2269573.html
• http://www.independent.co.uk/life-style/food-and-drink/unfortunately-you-can-change-the-url-on-any-indy-story-to-whatever-you-like-2269573.html

Update 4: 17 indexed now. And counting ...

Update 5: .The Head of Digital Audience & Content Development at this Indy has tweeted to blame the Guardian for the whole fiasco.

Jack Riley blames a Guardian sub

Maybe try the IT department too?

Update 6: By the end of the day, Google had indexed 62 different URLs for this story, nearly all via being made up and spread on twitter.

New ones included:

• AllAboutTheGames-is-the-best-gaming-site-on-planet-earth-2269573.html
• middleton-flicks-bean-as-she-waits-for-wedding-night-2269573.html
• what-first-attracted-you-to-royal-heir-prince-william-2269573.html

And a load of others I'm too tired to check.

Update 7: the next morning. Google has indexed 64 URLs (not all via Twitter. Reddit was having fun too ...).

It's no longer treating them all as duplicates.

Take a sentence from the story (which was not originally produced by the Independent) and add the word Independent and you are shown  5 versions of the story. Even without the word Independent, you see three versions on page one of the results and the option to see lots more.

The Indy's written a blog post apologising (for some reason).

We are however acting urgently (as are now some of our major competitors) to find a way to avoid such abuse and hope that will be in place shortly.

The solution's fairly simple. Do what other sites do with the same CMS and redirect to the canonical URL when someone types in gibberish in the part of the URL (the keywords) that isn't actually parsed.

I say he wrote it. It's mostly a restirring of an original source, an article in the Atlantic. But his main complaints are that journalists are turning into sheep, with the herd all charging in the same direction. I can't see anything different about how online news works compared to how newspapers work. Here's why.

Ewe don't have to follow the herd

His issues are as follows.

News organisations are writing about what their readers are interested in

"Market-driven news [means] news agendas that are driven not by some professional assessment of what's important and relevant, but by research into what viewers like and respond to."

There are five sorts of stories that news organisations can report on:

1. News they uncover themselves (scandals like MPs' expenses)
2. News that they and everyone else knows is going to happen (a sports event or a court hearing)
3. News that's sent to them (press releases)
4. News that someone else has uncovered (copying someone else's scoop)
5. News that wouldn't be news but for the fact that it's related to things people are interested in and actively searching for.

With any of these, I fail to see what's wrong with taking account of what viewers like and respond to. If you can't get interested anyone in your story, what's the point of writing it?

There are sites that only do (4) and (5). We tend to call them content farms and dislike them. A couple have made a success out of it. But as a long-term strategy, I wouldn't bet your business on it - you're at the mercy of Google's next algorithm update.

What's more, newspapers have always done this. They have an editorial line - it's based on what their readers expect. The fact that there are tools that make feedback more precise online, doesn't mean it's any different to what they get up to in print.

News organisations are writing about what their readers are interested in

"Somewhere in [online newsrooms'] fancy open-plan offices is a guy who is watching the second-by-second audience for every page on the site. He's the chap who knows what's "trending" now and if your stories aren't figuring then – depending on the editorial culture – you may eventually feel the heat of managerial disapproval.

Again, this surely applies in print as well. If you can't uncover or write about stories that are of interest to people, or make them sound interesting or compelling, you're not going to last long. There are tools that make it more obvious with online - but good.

Also, what's trending doesn't have to be a passive thing. You can shape the news agenda, you can drive interest in a topic by the quality of your journalism.

News organisations are writing about what their readers are interested in

AOL asks its journalists to consider "Traffic Potential ("How many page views will this content generate?"); Revenue/Profit ("What CPM [cost per thousand impressions, a measure of online advertising effectiveness] will this content earn?"); Turnaround Time ("How long will it take to produce?"); and finally Editorial Integrity ("Will this content conform to AOL's editorial standards?"). In that order."

I'm not really sure the order matters does it? But I'm struggling to see anything wrong with this as a set of questions. Newspapers are not a public good - they survive on the basis of the audience they deliver to advertisers (or not). Maybe journalists were never asked this as explicitly before - but is this really a bad thing?

To sum up

I'm not denying the pressure on journalists, the toll on news rooms caused by cost cutting or the dumbing down of the news agenda.

But none of this is caused by the measurement of online news per se - the new tools just give news organisations a more accurate way to measure what they've always done.

Photo credit.