In this article I compare the pre-election polling data with the vote share each candidate received to assess how accurate polls were before the first round of the presidential elections. I compare the actual election results to the vote share gained within Romania because opinion polls do not reach those who live and vote abroad.
Polls somewhat overestimated Iohannis’ popularity
As the first figure shows, the polls accurately predicted the relative ranking of the major candidates. The largest error occurred in the case of Klaus Iohannis, whose actual vote share was about four percentage points lower (within Romania) than the trend revealed by the polls predicted. The popularity of Iohannis among diaspora voters brought his election results slightly closer to the polls than the figure below suggests. Still, with literally all polls suggesting that he would win more than he actually did, either a systematic polling error occurred in his case, or his support dropped quite a bit in the end for unknown reasons.
A closer look at Iohannis’ polling numbers reveal two rather curious patterns. First, there is a difference between the predictions by established polling houses (those marked with colour-filled symbols in the figure) and polls with a more dubious origin – mostly allegedly private polls produced for the parties and then leaked (see symbols without a colour fill). The latter were more accurate and hewed more closely (with less variance) to the overall trend revealed by all published polls. Indeed, their figures varied so little regarding Iohannis’ vote share that it was as if they had no random sampling error. These patterns suggest that the leaked polling figures may simply be based on the results of other studies in their predictions.
The data regarding the popularity of Dăncilă followed a less systematic pattern except for an upward trend during the campaign. IMAS tended to have lower estimates for Dăncilă and higher estimates for Iohannis than other polling companies. CURS did the opposite and was thus ultimately more accurate regarding the two leading candidates’ vote share. SOCIOPOL tended to publish above-average estimates for both Iohannis and Dăncilă, while SOCIODATA’s figures hovered around the trend of all companies combined.
Barna’s polling numbers had a relatively high variance, little in the way of change over time, proved relatively accurate at the end, and showed no obvious sign of house effects. In other words, no company appeared to have a systematic tendency of indicating higher or lower support for him than other pollsters did around the same time.
Diaconu’s numbers are also worth mentioning, given his rapidly falling numbers after announcing his candidacy. As in the case of Iohannis, leaked polls seem to have less variation regarding his standing than polls by established companies. Among established companies, the last IMAS poll stands out by massively over-estimating the popularity of Diaconu compared to other studies and the vote share he ended up receiving.
House effects: how various pollsters performed
As a rough measure of accuracy, I calculate the standard deviation of each company’s predictions around the vote share received by the four leading candidates (Iohannis, Dancila, Barna, and Diaconu). This time I show the deviations from both the candidates’ domestic and total vote share (the latter also includes the 650,000 votes cast abroad).
As the figure reveals again, the leaked private polls – about whom we do not even know who did them or what sample they used- seem to be more accurate than the polls published by identifiable polling houses. Among the polling companies with a longer history, IMAS and SocioData were the least successful, and CURS and SOCIOPOL were the most successful. The studies by the former two deviate on average by 6-8 percent, while the predictions of the latter two deviate on average by 2-3 percent compared to the final vote share of candidates.
Overall these deviations seem fairly small, but the fact that all polling companies were better in predicting the total vote share of the different candidates than their domestic vote share is intriguing, since for purely methodological reasons the opposite could be expected. Unfortunately, the methodological choices different companies make about sampling and weighting are not transparent, and information about them is not publicly available.
A natural comparison could be made with the polls conducted in the context of the 2014 presidential elections. To this end, I reviewed the polls from a similar period – the beginning of August until the first round of the 2014 presidential election – and found that the list of companies publishing polling numbers was very different, with only one of them being present in 2019 as well (SOCIOPOL). Even though some of the ones from 2014 are still running political polls (e.g. IRES), they were conspicuously silent in the lead up to the 2019 presidential race. If we calculate the same measure as the one presented above for the polling numbers of the four most popular candidates in the 2014 race (Ponta, Iohannis, Tariceanu, and Udrea), the average deviation (prediction error) across companies for the four most popular candidates was only 2 percent, half what can be observed in 2019.
To sum up, the analysis reveals a mixed picture. On the one hand, the polls predicted the relative order of candidates and were thus informative for voters who may have wanted to vote strategically. On the other hand, the market for political polling is characterized by a changing cast of companies, something that does not help the establishment of track records and brand name credibility. Leaked polling numbers from unidentified sources and a severe dearth of information about methodological details make it difficult to assess the quality of the polling and to understand why polls are more or less close to the reality. The parallel with 2014 also illustrates that some of these problems show no signs of abating.
Endre Borbáth is a postdoctoral researcher at the Freie Universität Berlin and Wissenschaftszentrum Berlin für Sozialforschung and a Median Research Centre collaborator.
Leave a Comment