Credit: amy_b
A lab is a lab is a lab. A mouse is a mouse is a mouse. So if you test genetically identically strains of mice in the sterile, controlled, homogeneous confines of a lab in Toledo, Ohio and run the same experiment again in a lab in Berlin, you should get the same results. Not necessarily, according to a just published study in the British science journal Nature Methods, which presents strong evidence that researchers’ current practice of trying to limit variations in laboratories is a fundamental flaw of experimental design, one that increases errors in early drug development tests and in turn drives up the cost of pharmaceutical development.
In their study, Joseph Garner, a Purdue assistant professor of animal sciences and his colleague professor Hanno Wurbel of the Justus-Liebig University of Giessen in Germany observed behavioral differences in three different strains of mice in three different labs in two different types of cages at three different times of year — variations that are not controlled for or currently expected to affect the outcome of an experiment. The team found 10 times more false positives, where one strain appears to act differently from another at rates higher than expected.
To determine the origins of the false positives, the researchers reevaluated the data, picking a mouse of each strain from each lab, and found only the same number of false positives as would be expected by chance, indicating that the mice were indeed responding to subtle environmental cues. “This is hugely significant and completely outside the realm for it to be a freak occurrence” says Garner. “It tells us that something is creating these false positives.”
“Phenotypic plasticity,” an organism’s ability to activate different traits or characteristics in response to its environment, has long been acknowledged in mice. In fact, mice are the one species that has been able to follow humans and successfully exploit the environments we create. “Not only do mice share much of our genetic code, they also share our fundamental adaptability, which is why they’re such good subjects for experimentation,” says Garner. Clinical trials account for human phenotype plasticity, employing a broad range of subjects and environments to determine how a drug or other treatment affects different types of people. But scientists use genetically identical mice in lab testing, which is thought to negate environmental factors. Unfortunately, Garner argues, environment is never irrelevant for mice, either. “The fundamental, underlying logic of how scientific experiments are set up assumes that normal variation does not occur, or, if it does, it doesn’t matter,” he says.
The danger of homogeneous environments in labs, Garner says, is that a treatment or drug could appear to work in the lab, and then fail in further animal and/or human clinical trials at the cost of millions of dollars. “A single drug isn’t expensive to make. It’s the cost of the hundreds of drugs that tank in clinical trials that are at issue,” says Garner, who hopes his findings will spur better-designed experiments — such as testing in environments that vary by food type or bedding, for example, or at the least account for the inherent variability in labs, such as the cage type used. “We’re not advocating for complete randomness. Rather, we suggest that a well-designed animal experiment contain an element of ordered heterogeneity,” says Garner, who points out that cutting down on false positives could also lower the number of mice used in laboratory testing. “If we can help out a few mice along the way that wouldn’t be such a bad thing either.”
Environmental standardization: cure or cause of poor reproducibility in animal experiments?
Nature Methods March 30, 2009
Originally published April 9, 2009
