How old men being available on Friday nights to do online polls might be skewing results

How old men being available on Friday nights to do online polls might be skewing results

boon

Very early responders to poll invites might not be representative

After YouGov’s methodology changes last week ICM have announced their own measures as we approach the big day.

This is the firm’s Martin Boon he explains it on the pollster’s website:

“..Interviews tend to build up quickly on each Friday night, probably because certain types of people are more readily available and willing to participate. Indeed, there is a remarkable consistency across our online polls, with big Leave leads being built up in each hour from 4pm to 9pm on a Friday, partially mitigated by big Remain In leads every hour thereafter until the survey closes, ostensibly by Monday morning for data delivery to clients.

We believe it likely that the weight of interviews generated before 9pm on a Friday has the effect of consolidating a Leave lead as a result of the survey process itself – demographic quota cells fill up and ‘close’ once the target number has been hit. If a specific cell, such as 65+ men, is filled early with people disproportionately likely to support Leave, no additional 65+ men will subsequently be allowed on the survey. As a result, interviews with 65+ men are unlikely to be politically or attitudinally representative of all such 65+ men even though in demographic terms they are identical. But they are not, and their presence possibly introduces a small skew into in favour of Leave (or UKIP, depending on the question looked at).”

As a result the pollster is to stagger the release of invites to take part in its political polls and also to introduce a new weighting.

“..However, it is unlikely that process change outlined above will solve the problem other than partially. Respondents more inclined to Brexit may be equally fast to respond to their invite at other times during the weekend, thus still affecting the data but less overtly. As a consequence we are overlaying a new weighting scheme to reflect the profile of response by quickness to participate.

We will not publish full technical details of this weighting scheme, for fear of conditioning its power. However, we will be applying a “time of response weight” to reflect disparity in response between early responders and late responders. The net effect of this weight, so far, has been to reduce the Leave share by up to 2-points, with a corresponding increase in the Remain share by up to 2-points. It is entirely possible that the strength and direction of this weighting effect will change, if the pattern of response changes on any individual survey.”

The latest poll sees IN and OUT level pegging following a 4% OUT lead in last week’s poll.

Mike Smithson



Comments are closed.