Working with statistics can be difficult. Mathematical jargon is often confusing. If you don’t pay close attention it’s easy to make mistakes when interpreting numbers.
But every year, Africa Check’s fact-checkers still find themselves looking at claims and wondering: “How did they get it so very wrong?”
Let’s look back at some of the simplest statistical mistakes and most egregious errors we fact-checked in 2021.
(Africa) Check before you share
One of the first things you should do to check the reliability of a statistic is look it up! Fact-checking organisations, such as Africa Check, do the hard work of verifying numbers so that you don’t have to.
You can save yourself the embarrassment of sharing a bogus statistic by simply Googling to see if it’s already been debunked.
In April 2021, we spotted a number of Facebook users sharing the claim that South Africa is “the rape capital in the world” and that “a woman born in South Africa has a greater chance of being raped than learning to read”.
If the people hitting the “share” button had taken a moment to look it up, they would have found an article we published on the topic more than seven years ago.
But we weren’t even the first on the scene. Tom Moultrie, professor of demography at the University of Cape Town, debunked the claim that a woman in South Africa is more likely to be raped than to learn to read as far back as 2010, describing it as a “statistical horror”.
Prices go up? That’s inflation
It’s not just Facebook users who get it wrong. South Africa’s ruling party, the African National Congress, should have checked our guide to adjusting for inflation before claiming it had “increased the total values of goods and services produced in South Africa (GDP) 10-fold” in 25 years.
Ignoring the effects of inflation when calculating GDP growth results in, as one expert told us, “completely distorted figures” which make any comparison “pretty meaningless”.
We also checked a similar claim about South African GDP growth in 2019.
Bad surveys = bad stats
Some people are tricked into sharing terrible stats published by predatory journals or from a deceptive press release. But some surveys make their methodology and limitations clear.
Take the survey quoted by South African media outlet Independent Online, when it claimed that six South African cities had been ranked among the world’s 20 most dangerous. The survey was conducted on Numbeo, a site which asked visitors to rate how safe they felt in various cities.
While several South African cities were ranked as particularly unsafe, these results were based on the perceptions of the less than 3,000 South Africans who took the survey. The actual crime rates in those cities were not considered.
The original source had not hid these limitations or presented the survey as more than it was. Numbeo’s chief executive even told Africa Check that his company had never claimed that their index ranked the world’s most dangerous cities.
Runaway numbers hard to corral
One media outlet quoting statistics without verifying them is bad enough. But the worst-case scenario is when a mistake gains momentum as it is repeated and republished by multiple media outlets and public figures.
This is what happened to one faulty estimate of the number of Twitter users in Nigeria. A polling organisation used a sample of 1,000 phone-owning Nigerians aged 18 and older to put the figure at 39.6 million.
Statistics experts told Africa Check that it was wrong to make generalisations about the entire Nigerian population based only on this sample.
Mbe Nja, a statistics professor at the University of Calabar in southern Nigeria, told Africa Check: “For it to be applicable to the entire population, all Nigerians must have equal opportunity to participate in the survey. Many people in Nigeria don’t have phones and a large part of our population is younger than 18.”
But in June, the number was widely reported by both local and international news outlets and even shared by a prominent US diplomat.
There are more than 27 countries in the world
Here’s a tip that might seem obvious: when reporting that a country tops a world ranking, make sure the ranking includes all the countries in the world.
For example, don’t claim that a study found South Africa to be the country with the second highest rate of firearm-related deaths “in the world”, when the study only looked at 27 countries.
That figure appeared in a draft bill published by the South African police services. We found that more reliable datasets ranked South Africa much lower.
Want another example? South Africa does not account for “50% of global mining deaths”.
This claim was based on reporting by only 28 mining companies, which represented only about a third of the global mining industry. It also included no data on countries like China, which have especially large mining sectors.
How can you avoid statistical errors?
Everyone makes mistakes. Africa Check has a strict corrections policy in case we slip up. But not all statistical errors are complicated and most of them are easy to avoid.
In many cases, you can save yourself the embarrassment of sharing a bad stat by just taking the time to check trusted sources and doing some basic fact-checking before you share.
Or read our guides to help you understand common mathematical terms and avoid basic mistakes when reading and reporting numbers.
Look out for us in 2022!
Add new comment