
Research on natural medicine is often reported differently than research on pharmaceuticals, and not just at the level of the press.
“Warts and All”
Why Research on Natural Medicine is “Better” and “Cleaner” than Research on Pharmaceuticals.
On April 14, 2015, the World Health Organization of the United Nations issued a statement calling for “the disclosure of results from clinical trials for medical products, whatever the result.” [emphasis added].
This is a huge deal – or at least it would be, if it accomplished anything. Chances are, though, it won’t. Because no matter what the WHO says, pharmaceutical companies are still under no obligation to publish or even admit the existence of trials in which their drugs don’t work.
From a Business/Law standpoint, this makes sense. They pay for the data, they own the data, and they can do whatever they want with it.
But from a Science/Ethics standpoint, it’s absolutely criminal. They conduct a trial where a drug doesn’t work… so they brush it under the rug. Then they design another trial, differently this time, and it still doesn’t work… so they brush it under the rug. Then they design another trial, again differently, and now it does work, sort of. So they publish the trial, and proudly proclaim that it’s effective in 100% of published trials[1].
And the doctors that prescribe, the journalists who report, the government officials who set policy – none of them know any better. This is 100% legal.
The Emperor’s New Drugs
I first ran into this issue the way many of us did: through the work of Irving Kirsch, a psychologist and professor who researches the placebo effect. In 1998, Kirsch and a colleague published a paper which pooled data from 19 published trials on antidepressant drugs, and found that the drugs were, on average, only slightly more effective than placebo[2]. The paper was well-researched and statistically sound, but its conclusions were challenged on the grounds that the data were too limited.
So in 2002, Kirsch published a second paper: “The Emperor’s New Drugs: An Analysis of Antidepressant Medication Data Submitted to the U.S. Food and Drug Administration.”[3] For this paper, Kirsch did something revolutionary: he didn’t just aggregate data from published trials; he looked for unpublished trials as well.
How did he find them? Well, before a drug is approved in the U.S., clinical trials conducted on it here have to be registered with the FDA. Since the information is held by a government agency, Kirsch was able to request access under the Freedom of Information Act. He had to fight for it (his request was strongly contested in court), but he finally got his hands on the trial data. It turned out, there were literally twice as many studies on file with the FDA that had never seen the light of day. In more than half of them, the drug was no better than placebo.
As an eager, science-minded young man, I remember being shocked and dismayed by this. Sure, the media and the advertisers could get the story wrong – even doctors and scientists could get it wrong – but not me, because I was reviewing the actual data! It hadn’t occurred to me, I was only reviewing half of it…
Things have gotten a little better since then, but only a little. In 2005, the International Committee of Medical Journal Editors issued guidelines for member journals, where clinical trials needed to be logged in a public registry before subjects were recruited, as a precondition for publication in participating journals. But this was only a recommendation. In 2014, a survey published in the Netherlands Journal of Medicine[4] found that less than 50% of the journals that claimed to adhere to this policy even went so far as to tell their authors about it – and less than 20% actually checked.
The Truth, the Whole Truth, and the Tamiflu™ Truth
Meanwhile, Roche Pharmaceuticals continues to advertise Tamiflu™ on television despite withering criticism from the Cochrane Collaboration, the British Medical Journal, the New York Times, etc.[5], for failing to even disclose the existence of the largest clinical trial ever conducted on the drug – and then, after its existence was accidentally discovered, promising to share the data – but then, years later, still not disclosing it.
And this, and the antidepressants, are what we know. The whole problem here is what we don’t know.
So, why is research on natural medicine “better” and “cleaner”? Because, for the most part, we get to see it, warts and all.
Nobody “owns” vitamin C, garlic, or fish oil. So nobody is designing studies to make them look as good as possible. Nobody is hiding millions of dollars’ worth of research if it doesn’t look good. If adults in a clinical trial on turmeric for arthritis got more heart disease, you’d better believe it would be reported. Yes, you’ll absolutely see trials where zinc doesn’t fight colds, glucosamine doesn’t help arthritis, and green tea doesn’t prevent cancer. But that’s the point: you see them. They are not hidden. What you see is what you get.
Update: since I wrote this piece about two months ago, things have gotten more interesting. In September of 2013, England’s Health Research Authority (the government agency that approves clinical trials on humans) began to require researchers to register all new trials, so their results would be available, good or bad. That was a huge step. But it still didn’t bring historical data to light.
In April of this year, HRA began to require that scientists disclose all past trials they were involved with, before they’d be allowed to experiment on humans again. This, again, was a huge step. But now it’s being challenged in U.K. courts by a company called Richmond Pharmacology, which conducts clinical trials for pharmaceutical companies. Richmond has filed for judicial review, but beyond that, they have no comment. Their website makes no mention of the legal challenge, although they do still proudly proclaim that they are “fully committed to quality and transparency.” So there’s that. If you want to get involved, http://www.alltrials.net/get-involved/ is a good place to start.
[1] The psychologist Robert Rosenthal coined the term “File Drawer Effect” in 1979 to refer to studies that are “lost in the file drawer.”
[2]Kirsch I & Saperstein G. Listening to Prozac but Hearing Placebo: A Meta-Analysis of Antidepressant Medication. Prevention & Treatment. 1998 June 26: 1.
[3] Kirsch I, Moore TJ, Scoboria A, Nichols SS. Prevention & Treatment. 2002: July 15: 5. Kirsch later authored a book also called The Emperor’s New Drugs.
[4] Hooft L, Korevaar DA, Molenaar N, Bossuyt PM, Scholten RJ. Endorsement of ICMJE’s Clinical Trial Registration Policy: a survey among journal editors. Neth J Med. 2014 Sep;72(7):349-55.
[5] See http://www.bmj.com/tamiflu and http://www.bmj.com/open-data , with links to other resources.