On my bookshelf is a copy of Francis Wheen's How Mumbo-Jumbo Conquered the World (2004), in which cults, quacks and New Age mystics are hilariously exposed as charlatans. By contrast, in areas where it really matters, such as science and medicine, there is a solid bedrock of published evidence from which reliable judgments and opinions can be inferred.
But not always. Recently, and in quick succession, I have encountered three publications that should cause any health professional to reflect on the importance of critically appraising that which purports to be fact.
First, writing in the New York Review of Books, Dr Marcia Angell (2009) stated: ‘It is simply no longer possible to believe much of the clinical research that is published, or to rely on the judgment of trusted physicians or authoritative medical guidelines.’ A startling observation, since Dr Angell's conclusion was ‘reached slowly and reluctantly over my two decades as an editor of the New England Journal of Medicine.’
Second, Ioannidis et al (2017) advised readers on ‘How to survive the medical misinformation mess’, identifying four problem areas: the unreliability of much published medical research; the lack of awareness that most health professionals had of this problem; health professionals' lack of skills to properly evaluate evidence; and a lack of accurate evidence to inform decision-making for patients and families.
Third, an American study by Desrosiers et al (2018), which is at first glance freighted with grim portent, given the title: ‘Low carbohydrate diets may increase risk of neural tube defects’.
Having previously demonstrated that dietary guidelines were not evidence-based (Harcombe et al, 2015; 2017), Dr Zoë Harcombe (2018) undertook a detailed scrutiny of the findings of Desrosiers et al (2018). However, by the time Harcombe's analysis revealed that there was insufficient scientific rigour to support the claims made, media outlets around the world had repeated their flawed conclusion.
So what happened? Noting the well-established link between maternal folate and neural tube defects (NTDs) in infants, Desrosiers et al (2018: 1) state that restricting carbohydrate consumption: ‘entails abstaining from the very foods that are fortified with folic acid, such as cereal, pasta, and bread. Indeed, adherence to low-carbohydrate diets is associated with lower than recommended intake of several micronutrients including folic acid.’ They analysed data ‘from 1740 mothers of infants, stillbirths, and terminations with anencephaly or spina bifida (cases), and 9545 mothers of live born infants without a birth defect (controls) conceived between 1998 and 2011’, and undertook a telephone survey to estimate carbohydrate and folic acid intake before conception, concluding that: ‘women with restricted carbohydrate intake were 30% more likely to have an infant with anencephaly or spina bifida.’
Harcombe (2018), however, pointed out that the article had, extraordinarily, compiled a characteristics table that compared controls with controls and not controls with cases. This meant that the authors had incorrectly adjusted for some factors and not adjusted for material differences, such as women with birth defect pregnancies being less affluent, more likely to be obese and more likely to smoke. Notwithstanding these flawed adjustments, Harcombe (2018) also highlighted that one table could be interpreted to conclude that ‘94% of NTD-affected pregnancies occurred in women not restricting carbohydrate’. Harcombe (2018) wrote to the authors and to the editor of the journal in which it was published, but at the time of writing neither has replied.
Perhaps we are all guilty of not being sufficiently critical in our sifting of facts before arriving at an opinion. Clive James observes that criticism is not a science, but although his consideration of criticism was in the context of the humanities, James (1982: 205) also observes that: ‘Criticism is not indispensable to art. It is indispensable to civilisation—a more inclusive thing.’