Can you trust TripAdvisor and Yelp, or does ‘wisdom of crowds’ theory fail when applied to online review sites?

It is fashionable now to think that if you take a large number of opinions on a topic and average them out, the result will better reflect the truth than any single opinion. This idea is often referred to as the wisdom of crowds, and it is particularly popular with supporters of websites that are driven by customer reviews.

The idea of the wisdom of crowds was popularised by New Yorker finance writer James Surowiecki in his 2004 book of the same name, but it was conceived a century earlier, by gentleman-scientist and polymath Francis Galton.

In 1907, scientific journal Nature published a short paper by Galton titled “Vox Populi”, an account of his visit to the West of England Fat Stock and Poultry Exhibition, in Plymouth, the previous year.

Galton had been in search of data relevant to his views on selective breeding. But he happened upon a “guess the weight” competition in which participants paid sixpence each to enter a secret estimate of how much the meat on a particular ox would weigh once the animal had been slaughtered and its flesh dressed. There were prizes for those who came closest to the final weight, inducing nearly 800 people to enter. Some were animal dealers and butchers, but most were day trippers with no relevant professional experience.

Afterwards Galton persuaded the organisers to loan him the entries so he could make a statistical analysis. No fan of the common man, and as convinced as any modern cynic that common sense was anything but common, he was surprised at the results. The median – the point at which half the guesses were too high and half too low – was just 9lbs out, or less than 1 per cent away from the actual weight.

“This result is, I think, more creditable to the trust­worthiness of a democratic judgment than might have been expected,” he wrote.