Tuesday, September 28, 2004

Opinion polling

I found a rather natty opinion poll simulator. It's set up for a small sample of 20, from a population of 8,000, but you can change it to reflect say UK opinion polls. I think it rather nicely shows how margins of error work.

Thus e.g taking a populaton of 44 million, and a sample size of 1000, which is typical for the UK, assume that Labour support in the country is 36%. Let's take ten opinion polls.

Now you probably can't see on this small picture, but if you click here it'll be bigger, and you should be able to take 10 squares. These represent ten different polls (taken with the same methodology, at the same time).



As you can see no poll gets exactly 36%, which is the correct figure. None is miles out though, ranging from about 32.5% to 38.5%.

If we up the number of polls taken, to say 1000, we get this result (again click here).



Now with each dot again representing a poll one can clearly see the clustering around the actual figure, with around 95% of the polls within 3% of it (hence the 95% margin of error).

Of course in the real world you don't get multiple polls taken at the same time with the same methodology, and you don't know the real figure. So all you will have is one of these dots.

[There's lots of other things the site does -- e.g. show the distribution of polls, and how changing the sample size changes the margin of error (though the code for this is broken)]

Update: Thinking about it some more, what I have shown above, doesn't seem quite the same as the commonly understood 'margin of error'. The chart directly above shows that the results of 95% or so of polls will be within plus or minus 3% of the true value. However as commonly understood 'margins of error' say that true value will be within plus or minus 3% of our sample value. Does this amount to the same thing? It's too late...