How to
SEO advice
Categories: Bad reviews, How to. Tags:

New research: 100% of surveys are rubbish

August 11, 2009 8 Comments

In my survey, 100% of polls I looked at are rubbish. Therefore I conclude that all surveys are meaningless.

Brand Republic

So, according to Brand Republic, "less than a third of consumers would be willing to pay for online news content ... according to a survey of more than 1,000 UK consumers'.

Blimey, sounds impressive. It's not. It's a 'survey' that The Media Blog set up online in which anybody could vote. To get people to take part, they sent a tweet.


This is how to get a representative sample.

So they didn't go out and conduct a survey, weighting the answers to make sure they ended up with a representative sample. They sent a tweet. To their followers. Who are mostly journalists I imagine.

So I think that should read "... according to an online vote among anonymous people on Twitter who could vote more than once and who followed an account that's wholely about the media". This opinion poll is as far from being representative of the general public as it's possible to be.

It's bad enough the Media Blog claiming it was 'research' among 'UK consumers'. For Brand Republic to uncritically report is as news is even worse.

The Independent

The Independent is currently seeking feedback when you go to its site - a little box appears asking your opinion. Rubbish. (Screenshot got deleted in error, sorry).

The question is what type of content you'd like to see more of.

So what happens? They go running into the editor's office and go '23% of people want more business. Sack the sports desk and go headhunt me some business reporters'.

Or is the survey designed to get you to sign up to some emails and promotional offers. Hmm?

The Express

It's a bit cheap and easy, but honestly. Who dreams up these questions?!?

So 99% of people think Labour's treatment of the elderly is a disgrace. 1% don't. Nobody isn't sure.

Alongside that, 97% of people think the Olympics should be scrapped. and 98% think Labour has wrecked the NHS (aren't waiting lists about a quarter of what they were when Labout came to power?)


A disgrace

97:3 - some dissent?

97:3 - some dissent?

At least they're consistent

At least they're consistent

You might also like
  1. Ofcom: we need action not research on broadband speeds
  2. ITV shows why websites for mobiles are RUBBISH
  3. Google’s results for MPs’ expenses are rubbish
  4. Replace suspension cost: more rubbish UK search results
  5. Bing xRank – it’s rubbish as it can’t distinguise volcanos from 90s bands

Share this post

Follow me on Facebook or Twitter


  • 100 % of your readers who have left a comment totally agree with you.

  • Malcolm, thanks for the link to The Media Blog. Just to clarify a couple of points. One slight inaccuracy, the survey could only be completed once per IP address, multiple entries were blocked unless people wanted to vote at home, at work, from an internet cafe etc. In which case I admire their passion for this debate.

    I appreciate the findings also contradict your own view on the subject, which I understand is inconvenient and probably relevant to your criticism. But anything that fuels this debate is healthy in my eyes.

    And while you're probably right, by polling people via Twitter and among Media Blog readers and followers there were a lot of media consumers / contributors among the respondents - which makes sense to me, given the subject - I think you'll find if we'd asked people at random, on the street etc, or via focus groups representative of wider society, the results would have been badly skewed by the number of people who don't currently pay to consumer media in printed form, or consume media free of charge online, so we wouldn't have been examining a shift/trend and attitudes towards it, just a general malaise or disinterest in the subject.

    Totally take on board your criticism though and thanks again for the link and taking time to keep this debate alive.

  • Will, thanks for the comment (while you're here, did you get my ping/trackback on your original article yet..?).

    I don't think it's true that you can only vote once - I've just checked again, and I voted twice - once using Safari and once Firefox from the same computer. I am in a large office currently, so who knows what that does to IP blocking.

    I thought the survey was a useful snapshot contribution, and it was impressive it got so many responses.

    But there were only 5 questions, it wasn't representative of the public, and the questioning assumed a paywall-round-current-content type of charging (whereas, as you know, I've been arguing that the content will need changing to justify a paywall).

    To be honest, it was the reporting of it that annoyed me most - Brand Republic quoting it as 'a surevy' of 'more than 1,000 consumers'. It wasn't a survey (with its connotations of representativeness etc), it was a poll. And it wasn't of UK consumers. As you say, it was largely of media types.

    I then saw someone report the Brand Republic article - by which point it had become fact that less than a third of people would pay.

  • Also: "I appreciate the findings also contradict your own view on the subject, which I understand is inconvenient and probably relevant to your criticism."

    Ouch. Talk about playing the man, not the ball! I didn't get that personal when you misrepresented my points here.

  • Ade MacLeod says:

    I couldn't agree more. It is all very well saying the response to an ill-conceived survey is interesting, but it distorts the truth.

    I don't think the question "would you pay for newspaper content online?" is valid unless people understand what they are getting and how much they will have to pay. And I don't think they will know that until the papers are doing it.

    If you had asked me a year ago "would you pay to have car racing game on your phone?" my answer would have been an unequivocal NO. Yet... you guessed it... more than a fiver.

    So, if every one of the 1000 respondents is between 0 and 100% wrong, how useful is the survey?

  • [...] At the time the survey of 1,000 Media Blog readers was criticised in some quarters for being self-selective, a not unreasonable challenge. [...]

Leave a comment!

Add your comment below, or trackback from your own site. You can also subscribe to these comments via RSS.

Be nice. Keep it clean. Stay on topic. No spam.

You can use these tags:
<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

This is a Gravatar-enabled weblog. To get your own globally-recognized-avatar, please register at Gravatar.