YouGov mayor polls
YouGov has been a tremendously successful company - just look at its share price. I'm sure there are many reasons for this, but one must be that its polls have been consistently accurate. I have no reason to believe this is not the case in the London Mayoral election - the latest shows Johnson 6% ahead with 45% to 39%.It has been noted (by Adam Boulton) that their weighting for the over 55s was very strange. In their poll conducted between the 12-14 March, out of 1005, the poll was weighted so 382 represented the over 55s, which is 38%. The one on the 20-28 March had 1051 people, which was weighted to 408, which again is 38%.
Apparently this was an error, and YouGov then changed their weighting, so the 2nd-4th April poll had 1003 people, of which 281 were over 55, and the latest one, between 9th and 11th April, 1031 people, of which 290 were over 55. This is 28%.
Peter Kellner told Political Betting that the error made no real difference to the results, as other weighting factors (such as Party ID) compensated for it.
Fair enough. In other words it shows that the age weighting is only part of the weightings and doesn't do much on its own.
However another thing I noticed was that this error seems to have not only crept into the weighting, but the raw sample. Now YouGov's panel, who answer email questions, is not designed to be perfectly representative, and even if they ask a selection of people who are, they cannot guarantee those who reply will be. Nevertheless clearly it is helpful to get a reasonably similar age distribution in responses as you intend to weight to, as it means less statistical manipulation is required.
So looking again at the over 55s:
Respondents to YouGov Mayoral surveys who are over 55
2007
19/12 = 498/995
2008
19/02 = 517/1003
12/03 = 503/1005
20/03 = 557/1051
02/04 = 380/1003
09/04 = 335/1031
These are actually who replied to their survey. So for the first four London Mayoral polls, more than 50% (53% on the 20th March!) of respondents were over 55, not far off double London's real figure and much higher than even their wrong weighting they were aiming for (38%).
Now one can imagine that the near-retired and retired have more time to fill out the surveys, and this would explain why, since the weighting mistake has been pointed out, the no. of over 55s replying to surveys (or presumably who have been asked) has fallen quite sharply, but not back to 27%.
Nevertheless one has to wonder how much statistical manipulation has to be done to a poll when those surveyed are half over 55, when the proportion of the population (in London) is 27%. This again is not necessarily wrong - it is to some extent the way in which the company works. Yougov polls are not random samples, in which the idea is that it is going to be representative of the country, but instead (as far as I understand it) occasionally repeated samples of people of whom they know key characteristics, and therefore from that they can extrapolate about the opinions of others.
Labels: Polls
posted by Matthew @ 4/15/2008 08:06:00 AM
Comments: 0
