Is there a fairer way to rank Olympics success?

Comment: You may have heard, Aotearoa New Zealand came away with an impressive medal haul from the Olympic Games in Paris, and with the most golds we’ve ever won.

If you look at the rankings based on the total medals won, New Zealand (population about 5.27 million) is ranked 12th-equal with Brazil (population around 217.6 million), with each country scoring a total of 20 medals. But how do you rank a country by its medal haul, when China has a population of 1.4 billion to choose its athletes from, and St Lucia has less than 200,000?

There are many ways to rate each country’s medal rankings at the Olympics, but which way is most fair is an issue statisticians spend hours investigating.

Which country came ‘first’ in the Olympics, can also be ranked by the number of gold medals won, in which case we’d sit at number 11 with 10 gold medals, just below Germany, a country with more than 80 million people and 12 golds.

Both ranking systems are flawed, with large countries like the US and China, which have hundreds of millions (in China’s case, 1.4 billion) of people typically topping charts.

Researchers have suggested it would be better to look at medals per capita. That is, divide the total number of medals by each country’s population. This would work in New Zealand’s favour. If we consider the Tokyo 2020 Olympics (held in 2021) on a per capita basis, we’d rank fifth behind San Marino, Bermuda, Grenada and the Bahamas. But this wouldn’t be a particularly revealing ranking system either. If a tiny country manages to win a medal (as Julien Alfred did in the100m sprint, winning Saint Lucia with a population of less than 200,000, its first medal in Olympic history) it looks very good in the medal rankings.

So while the total medals rankings give large countries a disproportional advantage, the per capita rankings do the same for small nations.

A fairer way of ranking how well countries perform at the Olympics has been developed by two US researchers, and according to their method, New Zealand rises to 6th place overall in the Paris 2024 Olympics, and Australia knocks the US out of the number one spot.

The two researchers, Robert Duncan and Andrew Parece, recently published their population-adjusted national ranking in the Journal of Sports Analytics.

Suppose the citizens of a particular country were all equally able to win an Olympic medal, they posit. They then calculate the proportion of the world’s population living in that country and multiply that by the total number of medals to get an idea of how many that country should win.

At the time of the Tokyo 2020 Olympics, the world population was 7.2 billion and the US had a population of 337 million. Using the population proportion, the US should have won approximately 50 medals. (Not that they necessarily could have even if they should have; in 2020, 1080 medals were up for grabs, but due to various country quotas, a country could claim 579 at most.)

What Duncan and Parece do at this point is use a mathematical formula to identify how likely or unlikely it is that a country will win more than this proportion. They then consider how many medals a country should have won based on its population, the actual number of medals won and how probable or improbable this is, which Duncan and Parece call the ‘probability ranking index’.

If we use this probability ranking index, then at the Paris 2024 Games the top five countries would be (in order): Australia, France, Great Britain, Netherlands, and US. At Tokyo in 2020, the top five countries would be Australia, Great Britain, Netherlands, New Zealand and Hungary. In the 2016 Rio Olympics, the top countries would be: Great Britain, the US, New Zealand, Australia and France.

Of course, this ranking is problematic too, which the authors acknowledge – it doesn’t account for nations’ relative wealth in terms of per capita GDP, for instance. But, they argue, the population-adjusted method is still better than the current ranking system.

Duncan and Parece conclude by drawing attention to three countries that rank among the top ten in the 2020, 2016 and 2012 Games, using either a per-capita approach or a probability ranking approach. These are New Zealand, Hungary and Jamaica. (Incidentally, I corresponded with Andrew Parece and he said, as noted in the paper, that it was Pete Pfitzinger of Athletics NZ who encouraged them to develop this ranking.)

Their adjusted ranking system does shed light on how countries are performing in this quadrennial celebration of human athletic prowess.

But as Kiwis, we can and should feel rightfully proud of what our athletes have achieved at the Olympic Games, no matter how the rankings are tallied.