Dr Rob Ford (University of Manchester), Dr Will Jennings (University of Manchester), Dr Mark Pickup (Simon Fraser University) and Professor Christopher Wlezien (Temple University) reflect upon models and methods for projecting the vote at the 2010 British election.

In this post we consider the performance of our projection model and the other models published in the run-up to the May 2010 UK Election, discuss the methodological issues involved, and consider some of the future directions for projection both in the UK and elsewhere.

There were a number of different forecasts of the 2010 election result. The models differed quite a lot and the performance did too. There were some notable similarities, however. First, all of the models over-predicted the number of Liberal Democrat seats, by at least 24 seats. Second, all of the models under-predicted the number of Labour seats, by at least 23 seats in all but one case. Third, the models tended to best predict the number of Conservative seats.

Our PoliticsHome Poll Centre projection was the only model to exactly predict the number of Conservative seats. Like the other models, however, our projection underestimated Labour seats and overestimated Liberal Democrat seats. Our model relied on recent voting intention data and a variant of uniform swing. The model that was closest across the three parties was in fact the Hix-Vivyan pooling-the-polls model that assumed a uniform swing, one of three alternative models they offered—see the table above. The superior performance of these models, which relied upon uniform swing or some variant on it, contrasted with the claims of some other forecasters, who argued in favour of proportional swing. Interestingly, the FiveThirtyEight projection model, based on estimation of proportional swing, was the worst performing of all the models.

Projecting the Vote

As noted both on pb.com and elsewhere, our method for estimating electoral sentiment pooled all the currently available polling data, while taking into account the estimated biases of the individual pollsters (‘house effects’), and the effects of random sampling, adjusted for the effects of the sampling decisions pollsters make, which mean their samples are not truly random (‘design effects’).

Using this method, our final projected Tory share of the Great Britain vote, excluding Northern Ireland which is not surveyed by pollsters, was accurate within 0.1% at36.9%. The projection underestimated Labour’s share of the vote by over two percentage points (projected 27.6%, actual 29.7%) while also significantly overestimating the Lib Dem share (projected 27.2%, actual 23.6%). The model had a lower projection for the share of the vote gained by other parties (projected 8.3%, actual 9.8%).

By giving extra weight to the last day’s polls, we also reduced the error on our forecast by successfully detecting the late swing to the Conservatives and Labour evident in these polls – our final forecast upgraded the Conservatives’ share by 2 percentage points and Labour by 1 percentage point.

Our vote projections significantly underestimated Labour’s share and but reflected the general tendency of the polls to overestimate the Liberal Democrats’ share. The British Polling Council statement on the accuracy of the final polls reflected that the polls ‘nevertheless told the main story of the 2010 election – that the Conservatives had established a clear lead. All but one of the nine pollsters came within 2% of the Conservative share’. This explains why the error on seat projections was lower for the Conservatives than for Labour and the Liberal Democrats for all but one of the projection models. The British Polling Council also noted, however, that

‘The tendency at past elections for polls to overestimate Labour came to an abrupt end, with every pollster underestimating the Labour share of the vote, though all but one were within 3%. However, every pollster overestimated the Liberal Democrat share of the vote.’

This election demonstrated – with a number of new and relatively unknown pollsters in the field – that the simple averaging of all polls can present difficulties. While there was a general tendency to underestimate the Labour vote share, this was greater for some pollsters, such as Angus-Reid and Opinium, which had the effect of distorting the vote share projections – across all the projection models noted above. Simply pooling the pools to estimate an average did not resolve this problem, so our attempt to control for house effects was vindicated.

Another problem, little discussed in the pooling of polls was averaging of rolling surveys, such as by ComRes in the final weeks of the election campaign, which double-counted respondents and introduced a ‘moving average’ into the time series.

From Votes to Seats

The PoliticsHome Poll Centre’s projected outcomes in individual constituencies were correct in 86% of seats. Although our model performed well, there was a large slice of luck involved. We were in fact wrong to assume that the Tories would outperform in the marginals, but this was balanced by Lib Dem underperformance everywhere to deliver roughly the right result.

There were very clear patterns of differential swing in Scotland, as we predicted, although the differences were even larger than the polls had suggested. There were also differential patterns in Wales and in seats with large ethnic minority populations. These would both have been near the top of our list of expected differential effects, but we had no polling evidence on them so did not incorporate them in our model.

The massive over-estimate in Liberal Democrat support caused us (and others) to substantially overestimate the number of Liberal Democrat seats. This had less effect on our model and others relying on uniform swing as this gave the Liberal Democrats less chance of winning over a large number of seats even with large swing.

The big story, though, with regards the UNS vs. differential swing debate is that the pattern of swing was remarkably uniform:

The change in Conservative vote varied by less than two percentage points moving from their weakest to their strongest areas, and they actually underperformed somewhat in their weakest areas relative to the average

The change in Labour vote varied somewhat more, but there was no systematic relationship with prior strength – if anything the party performed worse in areas where it started off somewhat weaker.

The change in Liberal Democrat vote showed more evidence of proportionality, falling back three points in the strongest areas while rising in the weaker areas. But even here the evidence of proportional swing was weak and patchy at best.

Given the lack of any clear relationship between prior strength and outcomes, we expected proportional swing based models to perform quite poorly, and so it has proved.

By contrast, the PoliticsHome Poll Centre model performed very well given the poor performance of the polls. Indeed, had the polls been exactly right, our forecast would have been very close to the actual result —Conservatives 305, Labour 249, Liberal Democrats 65. This would have given a total seat error of 19 seats. Only the election night exit poll analysis run by the BBC, which built in demographic and political predictors of deviation from uniform swing, did better.

Visit the Poll Centre.

Remember their 10-point guide to CGT avoidance?

Three days before Gordon Brown became Prime Minister the Telegraph published this guide to property owners wanting to avoid capital gains tax. One choice bit was:

“Become a butterfly and flit between homes
Or in the jargon, switch “principal private residence” exemptions between properties. All gains on property are taxable with the exception of the home you live in which the taxman calls your principal private residence. However, if you own more than one home you can elect which you wish classed as your primary residence, provided there is some evidence that you have actually resided there, albeit shortly. If you live for even a matter of weeks at any stage in your “second” home, this enables you to write off the last three years of capital gains when you come to sell…”

Hat-tip to Scott P on the previous thread.

Mike Smithson

HMRC – Capital gains relief

Are they trying too hard to find a story?

There was a lot of heated argument on the overnight thread about the basis of Telegraph’s splash lead attack the new Treasury Secretary, Danny Alexander.

This is all about capital gains tax on property which, as most are aware, does not apply to your main residence. It also, as I myself discovered some time back, does not apply for the final three years of ownership – even if you are not living at the property as long as it has been your only main home at some point. (see final paragraph reproduced above)

That was the case with Mr Alexander who essentially is being accused of not paying a tax when no tax was due.

Over the past fifteen months the Telegraph has done a great job with its MPs expenses investigation and there’s a sort of assumption that if you are named in a splash lead like this then you are “guilty”.

This case is not about flipping or some of the other practices that were revealed.

I believe that the paper was trying too hard and that could undermine its credibility with further exposes.

Mike Smithson

Telegraph

Tonight’s “revelation” from the Telegraph

The target is the new Treasury Secretary, Danny Alexander who, according to the report, did not pay capital gains tax on the sale of a property.

Has he operated illegally? Apparently not. So what’s the big the deal?

Is the paper suggesting that citizens, not just MPs alone, should so arrange their affairs to maximise the tax they pay? That’s getting into a dangerous and contentious area.

It will be interesting to see what impact this story will have.

Mike Smithson

Will a big line-up make it harder to predict?

Above is a clip from the Guardian on the day that voting packs went out on the last occasion that Labour’s election machinery was used – the June 2007 deputy election.

At this point, just seventeen days before the result was announced, the firm favourite at 6/5 was Hilary Benn. Behind him in the betting came Alan Johnson (2/1), John Cruddas (4/1), Harriet Harman 6/1, Hazel Blears 16/1 and Peter Hain 33/1.

So with not that much time to go the eventual winner was fourth in the betting while the favourite finished up fourth.

I raise this to make the point that I’ve emphasised before – because the Labour leadership election is an exhaustive ballot then second, third or even fourth preferences could be critical.

And the more candidates that are in the race the more difficult prediction becomes.

If a fourth, and possibly a fifth, does gather enough nominations then , I’d suggest, David Miliband’s task becomes harder.

Mike Smithson

“..I think I will save the events of the last 45 mins for my memoirs. I wish I could tell u what’s going on, but if I did I’d have to kill u..”

Just over half an hour after Tweeting with news that David Laws was planning to resign Iain Dale published another Tweet, featured above, which I have been puzzling over ever since. This came out shortly after 7pm and before the news of the resignation got picked up by the mainstream media.

It sounds like he was involved, in some way, with the dramatic events but was not, and is not, in a position to tell us what this is about.

Yet from his appearances an hour or so later on the BBC News channel he seemed to be hinting quite strongly that an early return for Laws might be on the cards.

Such a scenario also fits with the resignation statement itself, the letter that David Cameron sent and the public comment that Nick Clegg made in the aftermath.

Putting the pieces together I wonder whether during these critical hours Laws was being advised in some way by Dale. He was acting as a sort of “prisoner’s” friend.

Both men are, of course, gay and the blogger would be an obvious person for the former minister to turn to. Dale had gone through a similar challenge about telling his parents of his sexuality and only did so when he was 40.

My theory, for what it is worth, is that Downing Street saw that carrying on was impossible and that Laws was being given assurances that he could be back quite quickly depending on the reports of the Standards Commissioner. It was that, I surmise, that Dale and him discussed.

BETTING UPDATE:
The latest prices on When, if ever, will Laws return as a minister are:
During 2010 – 25%
H1 2011 – 25%
H2 2011 – 30%
After that or not at all – 50%

Mike Smithson

I’m hoping that there’ll be a betting market up quite soon on whether David Laws will return to government as a minister.

Certainly the warm tone of David Cameron’s letter seems to indicate that this might be possible.

The PM concluded: “I hope that, in time, you will be able to serve again as I think it is absolutely clear that you have a huge amount to offer our country”.

Peter Mandelson, it will be recalled, had several comebacks. Clearly a lot depends on the Standard Commissioner’s report on the Laws case.

UPDATE: Betting opens on “Will Laws return as a minister”