THERE was excitement in Josefina Vázquez Mota’s campaign team this week when an opinion poll put her within seven percentage points of Enrique Peña Nieto, long the front-runner in Mexico’s presidential election. Ms Vázquez, who represents the ruling National Action Party (PAN), was 20 points behind Mr Peña in a poll in January carried out by the same organisation. For the Institutional Revolutionary Party (PRI) fake oakleys ebay, which Mr Peña represents, July’s contest is no longer looking like a done deal.

But how reliable are Mexico’s polling organisations? Accusations of bias and incompetence fly back and forth every time a poll comes up with an unexpected result. This month a detailed analysis (in Spanish) in Nexos magazine looks at how Mexico’s pollsters have fared in recent elections. Leo Zuckerman, a journalist and political scientist at CIDE university, considered 44 polls taken by 17 companies in 17 elections for state governors during 2010 and 2011. The results were a useful reminder of how wrong surveys can be.

In the state of Sinaloa, for instance, there was an upset in 2010, when none of the four polls taken before the election correctly picked the winner. Polls by GCE, El Debate, Reforma and Diario Noroeste all predicted victory for the PRI; in the end an alliance of the PAN and the left-wing Party of the Democratic Revolution (PRD) won by a healthy six points. Voters also surprised pollsters in Puebla (2010) and Michoacán (2011), in both of which three out of four polls got it wrong. Puebla, like Sinaloa, saw the PRI overturned by a PAN-PRD alliance, which seemed to confuse pollsters. Michoacán was simply very close.

Mr Zuckerman comes up with a reliability index for the 17 pollsters, based on how often they picked the winner and how accurately they measured the gap between first and second place. Of the main pollsters, he puts the newspaper El Universal first (it picked the winner in all of its five polls), and its rival Reforma last (it was correct only two times out of five).

Two notes of caution. First, the samples involved in constructing this index are unavoidably small: most of the firms analysed by Mr Zuckerman carried out only one or two polls during the period, and even most of the bigger players did no more than five. A more serious problem is that the companies did not run polls in the same elections. For instance, El Universal did not take part in the Sinaloa election, which confounded all those that did, nor the tricky elections of Michoacán or Puebla. It therefore had a relatively easy ride. And because Nexos’s index ranks pollsters mainly according to whether they pick the right winner, one could get good marks in the ranking by doing polls only in places where the result was not in doubt. (In Coahuila, for instance, it did not require a crystal ball to know that the PRI would win by a mile last year. In the end the margin was 27 points.)

One way to eliminate this problem would be to remove the bonus given to pollsters that pick the winner correctly, and instead assess them solely on how accurately they predict the gap between first and second place. That would mean that a poll that was wrong by only a couple of percentage points in a very close race, thus predicting the wrong winner, would get higher marks than one which accurately predicted the winner in a landslide but was wildly wrong about the size of the victory. It would mean that pollsters were not penalised for carrying out research in close races foakleys, and that there would be no easy marks to be won by predicting the winner in obvious cases such as Coahuila. Either way, the Nexos piece provides a great data set and has kicked off a valuable debate. Let us know in the comments if you can think of ways to improve it further.

I leave you with a handy link to a poll of polls. Here you can see that although estimates of support vary widely, there is consensus on the overall trend: Mexico’s election is getting much closer.