How to
SEO advice
Categories: Featured, Internet, Newspapers. Tags: ,

Want an SEO job? Check out the Daily Mail’s robots.txt file …

August 24, 2010 33 Comments

Want a job as SEO manager at the Daily Mail? Check out their robots.txt file (just don't tell them you saw it here ...). In the middle it says:

# August 12th, MailOnline are looking for a talented SEO Manager so if you found this you're the kind of techie we need!

Daily Mail's robots.txt file contains an SEO job advert

Daily Mail's robots.txt file contains an SEO job advert

Genius! You could also contact Mail Online MD James Bromley on Twitter.

For those who don't know, the robots.txt file is how you tell search engines which pages they can and can't crawl on your site to include in their index.

In the past it was worth occasionally checking out newspapers' robots.txt files as they listed the URLs of stories that they've had to withdraw for legal reasons (or joke Polish editions). Sadly, they don't seem to do that so much these days (and they'd get lost in the Mirror's massive file). Plus there's no easy way to check if they've been updated - Google Reader's ability to track changing webpages doesn't work with robots.txt files. Boo.

If you liked this post, I've got lots more about SEO (try this story analysing Google's keywords tool). I also blog about newspapers a lot - such as the Daily Mail (this story was popular recently on iDosing) and lately a lot about the Times paywall.

You might also like
  1. Whisky hidden in robots.txt file
  2. Hugely embarrassing: Daily Mail jumps gun on “Amanda Knox guilty” story
  3. Michael Jackson’s kids made the Daily Mail the most visited UK newspaper site in June
  4. Daily Mail readers pro Brown over condolence letter
  5. Change your logo: the shame of the Daily Mail’s poppy refusenik stance

Share this post

Follow me on Facebook or Twitter


Leave a comment!

Add your comment below, or trackback from your own site. You can also subscribe to these comments via RSS.

Be nice. Keep it clean. Stay on topic. No spam.

You can use these tags:
<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

This is a Gravatar-enabled weblog. To get your own globally-recognized-avatar, please register at Gravatar.