Bad Pharma by Ben Goldacre

Author:Ben Goldacre [Goldacre, Ben]
Language: eng
Format: epub
Publisher: HarperCollins
Published: 0101-01-01T00:00:00+00:00

Dodgy subgroup analyses

If your drug didn’t win overall in your trial, you can chop up the data in lots of different ways, to try and see if it won in a subgroup: maybe it works brilliantly in Chinese men between fifty-six and seventy-one. This is as stupid as playing ‘Best of three…Best of five…’ And yet it is commonplace.

Time and again we have come back to the same principle in this chapter: if you give yourself multiple chances at finding a positive result, but use statistical tests that assume you only had one go, then you vastly increase your chances of getting the result you want – if you flip a coin for long enough, you will eventually get four heads in a row.

A new way of doing this is the subgroup analysis. The trick is simple: you’ve finished your trial, and it had a negative result. There was no difference in outcome – the patients on placebo did just as well as those on your new tablets. Your drug doesn’t work. This is bad news. But then you dig a little more, do some analyses, and find that the drug worked great for Hispanic non-smoking men aged fifty-five to seventy.

If it’s not immediately obvious why this is a problem, we have to go back and think about the random variation in the data in any trial. Let’s say your drug is supposed to prevent death during the duration of the trial. We know that death happens for all kinds of reasons, at often quite arbitrary moments, and is – cruelly – only partly predictable on the basis of what we know about how healthy people are. You’re hoping that when you run your trial, your drug will be able to defer some of these random unpredictable deaths (though not all, because no drug prevents all causes of death!), and that you’ll be able to pick up that change in death rate, if you have a sufficiently large number of people in your trial.

But if you go to your results after your trial has finished, and draw a line around a group of deaths that you can see, or around a group of people who survived, you can’t then pretend that this was an arbitrarily chosen subgroup.

If you’re still struggling to understand why this is problematic, think of a Christmas pudding with coins randomly distributed throughout it. You want to work out how many coins there are altogether, so you take a slice, a tenth of the pudding, at random, count the coins, multiply by ten, and you have an estimate of the total number of coins. That’s a sensible study, in which you took a sensible sample, blind to where the coins were. If you were to x-ray the pudding, you would see that there were some places in it where, simply through chance clumping, there were more coins than elsewhere. And if you were to carefully follow a very complex path with your knife, you would be able to carve out a piece of pudding with more coins in it than your initial sensible sample.

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.