Credit: Flickr user Brooks Elliott
Placebo-controlled clinical trials are the gold standard of medical research. When doctors and other researchers want to know if a potential new drug is actually effective in treating disease, they design a controlled clinical trial comparing the drug to an ostensibly inert placebo. As I wrote last year, it’s critical to compare the drug to placebo, because otherwise it’s impossible to know whether the patients would have gotten better (or worse) anyways. Patients who believe their illness is being treated will often improve faster than those who are not confident that the treatment (or lack thereof) is actually helping. This is the placebo effect, and it’s a cornerstone of modern medicine. True placebos don’t have any power to treat disease; they just allow us to compare a proposed treatment to a neutral alternative.
But what if the placebo itself isn’t actually neutral? What if, for example, patients were allergic to some ingredient in a placebo? Might researchers be led into believing that the proposed treatment is more effective than it really is, since the placebo they’re comparing it to actually harms some patients?
You might argue that if this was the case, it would be a simple matter to take a look at the contents of placebos in a field of research and assess their potential side effects. Unfortunately, last week, UK medical writer Helen Jaques examined a recently published report suggesting that investigating the composition of placebos would not be a simple matter. A team led by Beatrice Golumb reviewed 176 clinical trials published in four highly-regarded medical journals in 2008 and 2009 and found that surprisingly few reports disclosed the contents of placebos used in the research. Only 23 percent of the trials gave full reports of the makeup of the placebos they used, and even counting partial reports, the number only increased to 32 percent. Studies involving pills (as opposed to shots or other treatments) disclosed even less: Just 9.3 percent of trials of pills gave full descriptions of the placebos used. The investigative report was published in Annals of Internal Medicine.
The report led one enthusiastic commentator, NaturalNews.com editor Mike Adams, to proclaim that the results of “thousands of clinical trials” must now be thrown out as invalid. Really?
Not so fast, the cancer surgeon and medical researcher who blogs as “Orac” responded last week. Orac points out that while the lack of placebo reporting is a concern that should be addressed, it doesn’t mean that placebos were never documented. While ideally the placebo’s composition should be reported in a formal research report, clinical researchers do document these things in lab notebooks, and are required to report placebos when submitting potential studies to ethics review boards. Any study that might be questionable could be verified after the fact.
Orac also wondered how frequent actual problems with placebos are in the real world. As one example, Golumb’s team identified lactose as a potential problem in placebos for cancer treatment, given that cancer patients in some cases tend to have higher-than-average levels of lactose intolerance. Orac looked up the references cited by the researchers and found no evidence that lactose in placebos resulted in any problems in the actual group of patients in the study in question.
That said, Orac concedes that there are potential problems with undocumented placebos, just not problems on the calamitous scale suggested by Adams. As Orac points out, Adams’ post is rife with errors, such as mistaking the .05 p value required for statistical significance for a drug “5% better” than a placebo (I should add that Adams also misquotes Shakespeare—which is almost as big a sin for former English majors like me). Adams claims that the problem with placebo documentation is a conspiracy on the part of pharmaceutical companies in order to market ineffective medication, but the study authors don’t even hint at this possibility. As Jaques points out, even Golumb says that the problem is not due to “willful manipulation,” it’s just an issue that needs to be addressed.
None of this is to say that pharmaceutical companies and the independent researchers they fund should be given a free pass. As the neuroscientist who blogs as “Neuroskeptic” wrote in September, drug companies have huge financial incentives to overstate the effectiveness of their products. In an article published in Clinical Pharmacology and Therapeutics, researchers proposed filtering out results of clinical trials from clinics where the placebo effect was unusually strong. As a result, the drug being tested would appear more effective—but in the process, Neuroskeptic argues, the clinical trial would no longer be truly random. This type of “cherry-picking” isn’t good science, and researchers shouldn’t be allowed to get away with it.
Dave Munger is editor of ResearchBlogging.org, where you can find thousands of blog posts on this and myriad other topics. Each week, he writes about recent posts on peer-reviewed research from across the blogosphere. See previous Research Blogging columns »
Originally published November 3, 2010
