dinsdag 20 november 2012

Bad Science - Ben Goldacre

Ben Goldacre is één van mijn helden. Dit is Ben Goldacre:





Hij heeft een column in The Guardian, die Bad Science heet, net zoals zijn eerste boek, en zijn eigen website. Ik las dat boek, en toen at ik een koek. Hieronder enkele fragmenten:

==========


Before we go any further into homeopathy, and look at whether it actually works or not, there is one central problem we need to get out of the way. Most people know that homeopathic remedies are diluted to such an extent that there will be no molecules of it left in the dose you get. What you might not know is just how far these remedies are diluted.
The typical homeopathic dilution is 30C: this means that the original substance has been diluted by one drop in a hundred, thirty times over. In the ‘What is homeopathy?’ section on the Society of Homeopaths’ website, the single largest organisation for homeopaths in the UK will tell you that ‘30C contains less than one part per million of the original substance.’ ‘Less than one part per million’ is, I would say, something of an understatement: a 30C homeopathic preparation is a dilution of one in 10030, or rather 1060, or one followed by sixty zeroes. To avoid any misunderstandings, this is a dilution of one in 1,000,000,000,000,000,000,000,000,000,000,000,000,000,000, 000,000,000,000,000,000, or, to phrase it in the Society of Homeopaths’ terms, ‘one part per million million million million million million million million million million’. This is definitely ‘less than one part per million of the original substance’. 

For perspective, there are only around 100,000,000,000,000,000,000,000,000,000,000 molecules of water in an Olympic-sized swimming pool. Imagine a sphere of water with a diameter of 150 million kilometres (the distance from the earth to the sun). It takes light eight minutes to travel that distance. Picture a sphere of water that size, with one molecule of a substance in it: that’s a 30C dilution. * At a homeopathic dilution of 200C (you can buy much higher dilutions from any homeopathic supplier) the treating substance is diluted more than the total number of atoms in the universe, and by an enormously huge margin. To look at it another way, the universe contains about 3 × 10 80 cubic metres of storage space (ideal for starting a family): if it was filled with water, and one molecule of active ingredient, this would make for a rather paltry 55C dilution. 

==========

Let’s imagine we’re talking—maybe even arguing—with someone who thinks that homeopathy works, someone who feels it is a positive experience, and who feels they get better, quicker, with homeopathy. They would say: ‘All I know is, I feel as if it works. I get better when I take homeopathy.’ It seems obvious to them, and to an extent it is. This statement’s power, and its flaws, lie in its simplicity. Whatever happens, the statement stands as true. But you could pop up and say: ‘Well, perhaps that was the placebo effect.’ Because the placebo effect is far more complex and interesting than most people suspect, going way beyond a mere sugar pill: it’s about the whole cultural experience of a treatment, your expectations beforehand, the consultation process you go through while receiving the treatment, and much more. 

==========

Similarly, many illnesses have what is called a ‘natural history’: they are bad, and then they get better. As Voltaire said: ‘The art of medicine consists in amusing the patient while nature cures the disease.’ 

==========

So, we take our group of people coming out of a homeopathy clinic, we switch half their pills for placebo pills, and we measure who gets better. That’s a placebo-controlled trial of homeopathy pills, and this is not a hypothetical discussion: these trials have been done on homeopathy, and it seems that overall, homeopathy does no better than placebo. And yet you will have heard homeopaths say that there are positive trials in homeopathy; you may even have seen specific ones quoted. What’s going on here? The answer is fascinating, and takes us right to the heart of evidence-based medicine. There are some trials which find homeopathy to perform better than placebo, but only some, and they are, in general, trials with ‘methodological flaws’. This sounds technical, but all it means is that there are problems in the way the trials were performed, and those problems are so great that they mean the trials are less ‘fair tests’ of a treatment. 

==========

Does randomisation matter? As with blinding, people have studied the effect of randomisation in huge reviews of large numbers of trials, and found that the ones with dodgy methods of randomisation overestimate treatment effects by 41 per cent. In reality, the biggest problem with poor-quality trials is not that they’ve used an inadequate method of randomisation, it’s that they don’t tell you how they randomised the patients at all. This is a classic warning sign, and often means the trial has been performed badly. Again, I do not speak from prejudice: trials with unclear methods of randomisation overstate treatment effects by 30 per cent, almost as much as the trials with openly rubbish methods of randomisation. 

==========

studies which don’t report their methods fully do overstate the benefits of the treatments, by around 25 per cent. Transparency and detail are everything in science. 

==========

a phenomenon which we see over and over again with CAM studies: most of the trials were hopelessly methodologically flawed, and showed positive results for homeopathy; whereas the couple of decent studies—the most ‘fair tests’—showed homeopathy to perform no better than placebo.

==========

These patterns are reflected throughout the alternative therapy literature. In general, the studies which are flawed tend to be the ones that favour homeopathy, or any other alternative therapy; and the well-performed studies, where every controllable source of bias and error is excluded, tend to show that the treatments are no better than placebo. This phenomenon has been carefully studied, and there is an almost linear relationship between the methodological quality of a homeopathy trial and the result it gives. The worse the study—which is to say, the less it is a ‘fair test’—the more likely it is to find that homeopathy is better than placebo. Academics conventionally measure the quality of a study using standardised tools like the ‘Jadad score’, a seven-point tick list that includes things we’ve been talking about, like ‘Did they describe the method of randomisation?’ and ‘Was plenty of numerical information provided?’

Geen opmerkingen:

Een reactie plaatsen