Regardless, the main the main argument:

"And it's not like it has overlooked simple security measures and is exposing its database on a public-facing server"

Well no, it isn't. The database isn't accessible whatsoever.

telnet http://www.pcc.org.uk 3306
Trying 83.138.130.133...

A little knowledge is a dangerous thing

Poor article

I've got plenty of thoughts about ways the PCC could better handle traffic like that in the future, i'll be sure to include some of the points here - hopefully I can get them to some receptive ears.

I'd like to contrast what the PCC's response has been with my own response when 3,000 people visited my blog in about 30 minutes and my host refused to answer my emails and phone calls when my monthly limit was exceeded.

It took me about 12 hours to get the site moved to a new host and everything up and running again.

Little about the PCC setup has changed from friday until earlier today as far as I can tell. Maybe they've hurled some load balancers in there that I can't see. But they seem to be having little effect until recently. The PCC site was slow earlier today, the other sites on the server were slow (things do seem to have improved now).

The PCC, unlike my blog, is the body that self-regulates the press. It has a duty and a responsibility to allow people to make complaints. 21,000 may be lots - but how many more would have complained if they could have got on the site?

As you say - a wake up call. We can agree on that.

"more complaints in a single weekend than the regulator has received in total in the past five years.

This would explain why the PCC website ran so slowly on Friday and today. Actually, no it wouldn't. 21,000 isn't that many."

Right. I'd like to take issue with your numbers, here: "21,000 isn't that many",

1/ Except that it is relative to the regular traffic.

2/ And that's assuming that everyone that visited the page actually made a complaint, more likely a huge number of people followed blind bit.ly/tinyURL links and didn't do anything - but those are still hits to the site.

3/ Each of those 21,000 complaints is at least 4 or 5 clicks to complete the complaints process. Let's assume 5% of people who hit the link made a complaint (reasonable considering the follower count of fry, brown, etc - more so that they have many non-UK followers), that's actually much closer to half a million requests.

A more reasonable estimation - no?

I'd say the security issue is pretty much a non-issue, in the scheme of things. I would agree some more forward planning, particularly failover process should have been done. The problem is, this is such a huge increase in traffic it's difficult to account for.

Just because the PCC is the press watchdog doesn't mean they have an infinite budget to prepare for something like this, not to mention justifying it.

I imagine this will be a bit of a wake up call for their tech team and hopefully things will improve in the future. Things going tits up on a Friday never helps either.

There are plenty of quick and easy alternatives to expensive dedicated boxes, such as RackSpace Cloud Servers/Sites/Files, which can all be set up in minutes, and don't require expensive long term commitment.

The site is now pretty quick for me, although I'm on a fast connection - I'm now wondering if the problem was with the webserver, or whether it came down to poorly written SQL/unoptimised DB tables.

At the end of the day, it's horses for courses. I've stuck sites on single boxes and seen them withstand a torrent of abuse. I've also launched sites on heavily load-balanced servers and seen them yo-yo under the pressure.

You can't forsee a massive spike in traffic, but you can always make an effort to minimise the impact once it does.

(Plus the PCC isn't really an 'average' site - it is the press's watchdog!)

There's stability, then there's pissing away money overprovisioning to the point where you're able handle 1825 days of traffic in one day, just on the off-chance Charlie Brooker and Derren Brown decide to Stephen Fry you.

Shared servers are common in the world of hosting, and often operate perfectly adequately come rain or shine. Obviously, they're not going to stand up to a slashdotting/digging, but then again, neither would most single dedicated boxes, unless well configured.

Secondly, having the database server on the same box as the webserver is perfectly acceptable for the vast majority of websites out there - sure, it's not scalable, and it's certainly not very resiliant, but for the average site it does what is required of it.

Having a DB server on the same box as the webserver also doesn't make it any more insecure - if you compromise the site and get hold of the DB login credentials, you'll be able to get access to the database, regardless of whether the server is local or remote.

As you said above, "more complaints in a single weekend than the regulator has received in total in the past five years." - this site has clearly been happily running away in its own little shared environment, fit for purpose until the day it gets an unexpectedly large volume of traffic.

That's not to say that the PCC aren't at fault for having provisions in place for the worst case scenario - regardless of whether the site is on a single box or on a load balanced cluster, if it's not well built, and hasn't had any thought applied to scalability or optimisation, there's trouble ahead. Clearly in this case, little thought was given by either the PCC, or their hosting partners, who by now should have pulled the site onto a box which could cope with the demand.

Just my 2c.

Looking at their press releases, this year it seems they've dealt with less than 50 complaints - http://www.pcc.org.uk/news/press-2009.html

What's interesting is how the co-hosting model can be adapted to the "Slashdot Effect". When my website was featured on Boing-Boing, I had to rapidly pay for extra bandwidth or face being cut off halfway through the month. Is there any sensible way for those who share servers (rightly or wrongly) to manage sudden and infrequent bursts of interest?

T