The importance of a 404 status code on your 404 page
The ABCe site shows the importance of a proper 404 status code, as this search for abce demonstrates. The explanation of how they can avoid this is below.
The problem - the third result
ABCe: not found
The solution
The usual thing to do with known links that no longer work is to 301 redirect them to a different page (EG when you know a URL has moved). For all other URLs that don't work (EG when someone mistypes one), you return a 404 code.
These status codes help Google, for instance, work out what is going on. If it sees a 404 code it knows there's nothing there so it needn't return the page in search results.
I've written before about the awful abce site and the terrible abce redesign. It appears when they did the latest redesign, they decided to 301 redirect any broken links to this page: http://www.abce.co.uk/404.aspx.
However, that page, even though it it called 404.aspx and has a 'page not found' title, returns a 200 code - telling Google that it is a genuine URL.
The end result is that all links to old ABCe URLs are redirected to the new 404.aspx page. The 301 tells Google to count links to the old pages as if they belonged to the new 404.aspx page. Because this is a lot of links and a lot of pages, Google has ended up thinking the 404.aspx is the most important page on the abce site.
All of which means that if you type abce into google, you get the identical abc site first, and then you get the 404 page (with a 200 code) for the abce site. Obviously, this looks completely stupid.
This should be super easy to fix, even if you're using ASP
Let's hope they stop by to realise that!
This is a pet hate of mine, so many people neglect to return the correct status codes because of misguided SEO advice from people who don't know what they are doing.
Managing redirects and error codes is part of an SEO's job and if you're not doing it your failing in part of your role. Just like it wouldn't be acceptable for a designer to leave your footer unfinished.
Half of its stems from poor advice on htaccess syntax, obviously not in this case but in general
I wonder if they've ever actually typed abce into google. Maybe they just type abc and don't realise they've got two identical sites ...
Hi Malcolm, I just found your blog. Nice work!
This post and the one regarding 9 sites who don't catch their site.com traffic as well as http://www.site.com traffic caught my eye.
These are exactly the types of things that can be simply fixed, if you have an Application Delivery Controller (ADC) sitting in front of your site. Most organisations have load balancing, but need to move up to ADCs, so they can quickly fix these (and more complex problems) as they crop up.
[...] The importance of a 404 status code on your 404 page (malcolmcoles.co.uk) [...]