Viewpoint: Opinion Polls - less of a problem for research, more of a teaching aid

Nick Moon, GfK

One thing that I can confidently predict will have happened between me writing this and you reading it is that, somewhere in the print media or online, there will have been a ‘Polls Apart’ headline. In fact I’m surprised I haven’t seen one recently, given that the arrival of yet more new pollsters on the block has led to some consistent disparities.

Comparing pollsters has been made slightly more tricky since YouGov’s switch to daily polling, but we can dampen the effect of this by looking at only one of its daily polls for the Sun each week. On this basis, then, according to the estimable UK Polling Report, there have been 53 polls so far in 2010. If we take the Conservative lead asan admittedly crude marker for each poll, then the range is from 2% to 16%. Over two and a half months this is hardly exceptional, but what is striking is that there are 17 polls with a double-digit Tory lead, and 10 of these come from just two firms: Opinium and Angus Reid. And what is even more striking is that while the lowest lead on all of Opinium’s polls is 4%, the lowest lead on Angus Reid’s is 12%.

The most recent Angus Reid poll had a lead of 13%, while YouGov and ICM, at almost exactly the same time, had leads of 3% and 7%. This is not to say that Angus Reid is wrong and the other two are right, but they can’t all be right. It appears that the main driver of the difference is that Angus Reid is using a different approach to weighting, and specifically to weighting by past vote. This means that one of three things will happen: it will carry on doing this and will be wrong in its final poll while others are right; it will carry on doing this and will be right in its final poll while others are wrong; or it will change its approach to weighting and come into line with the others. But what it underlines for those not involved in polling is that weighting, and in particular weighting with a degree of subjectivity about it, plays a far larger role than in most surveys.

As well as leading to predictable headlines, polling gives those of us involved in survey research of any kind the chance to educate and inform those who understand little about the principles or surveys. In a series of polls, we would expect a degree of scatter (in fact we would expect more scatter than we usually get), with one poll every so often way out of line, so the occasional ‘Polls Apart’ is a good way to explain sampling error. But what we have at the moment is a systematic difference between Angus Reid and YouGov, despite them both being online pollsters, and this is a good way to explain the difference between error and bias.

People of course remember that the polls did very badly in 1992, but in recent elections the performance of the polls has been strong enough to provide support for the theoretical basis of surveys, rather than something that had to be explained away. At the last election, NOP’s final poll had all three main parties absolutely correct, and the other polls were very close as well. Apart from anything else this proves that it is possible for a sample of a mere 1000 people to represent the views of an electorate of 40 million. I’m sure that people reading this journal will already know about the bowl of soup analogy – you need only a spoonful to know what flavour it is – but I would like to share with you a wonderful new analogy I heard recently. Next time someone tells you they don’t believe a small sample poll can possibly tell you anything, just say to them ‘OK then. Next time you have to have a blood test, why don’t you ask them to take the whole lot?’

With the election likely to be very close indeed, the polls will also provide a good tutorial in how the media work. All we know about error and bias tells us that if the polls have been very consistent and then one poll suddenly comes out with a very different result, then this is probably the 1 in 20 that isn’t inside the 95% confidence limit – it’s a ‘rogue poll’, and the best thing to do is not to react until you see another poll saying the same thing. But this can be difficult to do, because the media understandably make far more fuss about big changes in polls. You will, for example, see headlines shouting ‘NOTHING HAS CHANGED’ far less often than you see ones like ‘TORIES STREAK AHEAD’ or ‘TORY LEAD HALVED’. The no-change polls get buried on page 94, while the ones that show big changes are guaranteed the front page, and yet the former are more likely to be right. Our profession will come under great scrutiny during the election campaign, but we should see polls as a good thing for the industry, not a bad thing. At the last election I had the opportunity to conduct a survey and immediately afterwards have external proof that it was right. How many other researchers can say that?

International Journal of Market Research 52(3), 2010

Would you like to respond to this Viewpoint? Or perhaps you have an idea for another? Responses and new submissions are welcome. They should be emailed to the IJMR, where they’ll be considered for publication.

What's New - Membership - Company Partner Service - Members' Area - Code/Guidelines - Qualifications - Training - Awards
Events - Networking - Publications - Media Info - Market Research - Search - Site Map - A-Z Directory - Contact Us - Home