Damn Lies and Statistics
Tom Jagtenberg
January 2018
When Janet Albrechsten told ‘a story of a faithless age’ (‘Faith, hope and celebrity at the polls’, The Weekend Australian, January 13-14, p.19) my fact-checking antennae went wild. One might agree about widespread loss of confidence in institutions (organised politics in particular), but the extent of the decline is probably exaggerated by Albrechsten’s sloppy use of statistics to back up her story. Indeed the statistics used appear so exaggerated that they will drive narratives about institutional decline in very unwarranted directions. This is another example of how the combination of journalists’ need for sensational stories and their misuse of statistics contribute greatly to the creation of new mythologies and directly to loss of trust in institutions. The journalistic use of statistics to describe society and culture may often be inappropriate, but statistics remain an important bulwark at a time when science and facts are not sufficiently respected – eg. the reception of climate science by Australian politicians. For those reasons, it is worthwhile further analysing Albrechsten’s story.
Citing a 2017 Gallup poll, Albrechsten asserted that ‘two-thirds of Americans don’t have much confidence in institutions that play a critical role in the life of America [. . . ]in a post-truth world, the only truth we seem to agree on is we don’t trust those who brandish institutional power’. What Albrechsten was keen to conclude was, basically (in my words), ‘no wonder we got Trump, and guess what? . . . in this climate Oprah Winfrey might be the next President!’ This is not an original line of thought and, persuasive as it may appear, the main effect of Janet Albrechsten’s journalistic appropriation of one very limited statistical poll does raise big questions that, seems to me, are very misleading.
For instance, to what extent have Americans become faithless? Are Australians and other western societies increasingly faithless? Is loss of faith in institutions caused by declining religious faith, or are their other reasons? Should religious tuition be encouraged to restore faith in institutions? And so on. These questions express something of one possible drift in a reactionary conservative take on the subject – a drift one assumes from journalists writing for the Australian - but they are questions that need a firm response, nonetheless. Not really, is the general short answer, for reasons I will continue to address in this blog space. Albrechtensten’s biggest claim, that we live in a ‘post-truth world’ has already been the subject of an earlier blog – my answer being, of course, it’s not that simple, even though some journalists might wish it so.
If we turn to the source of Albrechsten’s statistics, the Gallup poll, it is clear the findings can easily be read less sensationally. For instance, when asked about confidence in ‘the church or organised religion’, the direct findings are that 25 per cent of those polled had ‘very little’ confidence in those institutions as opposed to 23 per cent who had a great deal of confidence in them. This is not the same as saying that 59 per cent of respondents don’t trust the church. About half of the respondents opted for a more middle position on a five-point scale of trust (from ‘very little’ to ‘quite a lot’). One of the major problems with giving survey respondents a range of choices is that respondents will generally opt for a middle position. When big and vague questions are posed (such as in the Gallup poll), extreme responses are probably more indicative of ‘loss of faith’. In that light, the Gallup poll findings were not as sensational as ‘two-thirds don’t trust’ might signify. Another point about the actual poll findings is that not all institutions were mistrusted: the military and small business were highly trusted; no surprises there. More generally one has to wonder what actually goes through a typical respondent’s mind when asked about their trust in, say, political systems, or health systems or trade unions, whatever. Do people actually distinguish between corrupt or criminal individuals and the systems they work in? This distinction could not be more important in the case of secular liberal democracies in the western world where, arguably, this is as good as it gets - despite the criminality and bad behaviour of so many individual politicians.
One interesting finding that Albrechtsen neglected to report was 73% of respondents didn’t have much confidence in newspapers either (using Albrechtsen’s dubious calculus of 100%- the sum of percentages of those having ‘a great deal of trust’ and those having ‘quite a lot of trust’). Given Janet Albrechtsen’s uncritical haste to use poll findings that are actually more nuanced than she declares, we can see how a journalist might contribute to loss of trust in newspapers.
The bigger point that needs to be made is that Janet Albrechtsen is obviously wrong in her conclusion that we live in an age of ‘tumbling trust in major institutions’ – what she also calls ‘a faithless age’. That is, aside from only partial reporting of findings made on a five-point scale (six, if one includes ‘don’t know’) which should not be reduced to a binary don’t trust/do trust outcome, her uncritical acceptance of the Gallup poll findings misunderstands the still very high levels of faith-based thinking that occurs across the planet. Not only do very large numbers of people believe in God, religiously inspired cultures encourage passive fatalism in the face of real and imminent threats. For example, we continue to elect governments that downplay climate change, ecological devastation and the importance of science. As all advocates of secular democratic societies would say, ‘we’ urgently need to be replacing faith with reason. And we do need to be wary of ‘fake news’, aka sensationalistic journalism.
Data about confidence in institutions is very hard test. At least with polls taken about voter intentions, their results can be tested against the outcomes of elections. But how do you test general assertions about ‘loss of confidence’? One way is to look for more evidence – such as changes in economic indicators such as spending and investment patterns, or social indicators such as patterns of use of professional services or membership of clubs, organisations, societies and groups, or cultural indicators such as popularity of television shows, content of social media, and so on. Delving more deeply into the nitty-gritty of daily life will, I believe, support my assertion that people generally remain highly faith-based in their approach to evidence and in their reliance on most professionally based institutions such as health, education and the law. Market research for spending on consumer items is still the most regular critical thinking that people do.
Journalists over-rely on polls as evidence for sensationalistic stories. Rarely if ever do they back up their arguments with other statistics, or demonstrate any sensitivity about methodological shortcomings of any of their data sources. This flawed approach to telling the truth may actually be a post-truth methodology, but let’s hope it doesn’t become too popular.