Did the betting markets fail in Chesham & Amersham and Batley & Spen?

Two big by-election shocks in the space of two weeks. Did the pundits and betting markets underestimate the eventual winners as the Conservatives failed to win both Chesham & Amersham and Batley & Spen?

First, an explanation of what I mean by betting markets. The days of only ‘bookies’ deciding what the odds should be are long gone. The odds you’ll see on exchanges like Smarkets are the result of the supply and demand of the market and reflect the ‘wisdom of the crowds’ rather than the views of any one particular analyst. Indeed the odds you’ll see offered by standard ‘fixed-odds’ betting sites are generally linked to these estimates.

How ‘wrong’ were they? Well, in Chesham and Amersham some punters managed to back the Liberal Democrats at 28/1 at one stage, although they were back down to around 7/1 on the day of the vote. The top price you could have got on Labour in Batley and Spen was around 6/1 on polling day.

A couple of outsiders winning back-to-back doesn’t necessarily make me think you can dismiss betting markets out of hand as a way of forecasting elections. If our market says something has a 16% chance of winning, you should expect that to happen roughly one in every six times. If the favourite won on every single occasion, there really would be something badly wrong with the markets.

I accept that’s an easy get-out though — we don’t really have enough elections to determine how accurate the markets really are versus other forecasting methods. It’s not like sports where we can observe huge numbers of, say, tennis matches and verify that players given a 30% chance of winning do actually win 30% of the time in the long run.

Still, we can say that of the 17 British by-elections held since the 2015 general election (with betting markets available), the favourite — or most likely winner — was beaten in four. There’s not enough data here to draw any firm conclusions, although it’s interesting that three of the unexpected results were the Conservatives failing to win seats and the other was a Brexit Party flop (in Peterborough). Observers have often suggested that the betting markets have a tendency to overestimate the performance of right-wing parties, particularly of the populist variety. I guess this is a small piece of evidence one could use to back that up.

I suppose on reflection one could argue that people probably under-estimated local issues in both of the most recent contests. Based on national polling, there was no way the Lib Dems were going to win Chesham but it seems like the voters wanted a chance to vent over HS2 and planning reforms. Recent local election results also indicated they might have had a decent chance.

In Batley, maybe Kim Leadbeater’s personal strengths were underestimated. At least one shrewd punter from the area had told me that she would prove a very hard-to-beat candidate from a long way out. The story of Matt Hancock’s affair couldn’t have helped the Tories very much, although the betting markets hardly moved at the time. Predicting a close election or even a Labour win wasn’t illogical. George Galloway getting over 20% AND Labour winning? I wouldn’t have taken 100/1 had you offered it beforehand.

Matthew Shaddick

Matthew Shaddick is Head of Politics at Smarkets