Eureka Once, Eureka Twice

See Inside

Biology is making it harder for scientists to reproduce one another's experiments

Science works by iteration.' Scientists repeat their peers' work and build on their findings. The literature of peer-reviewed scientific papers is the record of this step-by-step process. In recent years, however, prominent reports have suggested that many scientists are not able to replicate others' published results. Is scientific progress going wrong on an unprecedented scale? Before we jump to that conclusion, it would help to consider the changing nature of science itselfparticularly biology.

Basic biomedical research and its translation into therapeutic interventions to cure diseases are at the center of this issue. In an ideal world, academic scientists identify targets for drugstypically proteins involved in diseaseand industry scientists look for agents that interfere with those targets' function. In reality, more often than not, industry scientists find that they cannot replicate the effects seen by academics in a sufficiently robust way to justify drug development. Worse, many promising drug candidates fail in phase II clinical trials when their efficacy is put to the test.

The world seemed simpler in the 1970s, when molecular biology brought us concepts such as gene A leads to protein B, which leads to function C. Thinking this way, scientists uncovered amazing mechanistic insights and, sometimes, designed effective drugsthe cancer drug Gleevec is the poster child of that reductionist approach. Wouldn't it be nice if drug discovery always went this way?

Those first drugs, however, were low-hanging fruit. Biology is much more complicated than simple schematics. Biological processes do not work in linear ways independently of one another but in tightly interconnected networks. In each branch of these networks, layers of regulatory controls constantly change the nature and abundance of the molecular players. We know little about the inner workings of human cells.

To illustrate how little, consider how genes are controlled. The modern study of gene regulation started in the 1950s, but researchers only started to unravel the complex array of histone modifications that fine-tune chromatin control of gene expression 20 years ago. The fact that RNA interference, another mode of gene regulation, is pervasive has only been realized in the past 10 years. What else don't we know yet?

Laboratory biologists deal with complexity on a daily basis. Mice bred with identical DNA behave differently. Two cells growing side by side in a petri dish cannot be considered identical. In the variable environment of the cell, it is difficult to distinguish a change that is meaningful to a process from one that is unrelated. Working in a modern lab also entails using sensitive apparatuses, rare technical skills and biological reagentsantibodies and enzymes, for examplewhich are themselves variable.

In such noisy systems, it is easy to mistake a chance observation for a robust, biologically meaningful effect. Biologists have to undertake large studies that can guarantee the statistical significance of observations, and they need self-critical analysis to avoid inadvertent biases. Scientists cannot be too careful to avoid falling prey to their own enthusiasm.

In that regard, they need the support of their institutions and the journals that publish their results. Some journals, such as Nature, have introduced checklists to ensure that scientists consider and report key information about experiments. (Scientific American is part of Nature Publishing Group.) Still, research institutions should provide more training and supervision of younger scientists. Institutions and funders should manage their incentive systems to limit undue pressures on researchers and promote best practices.

Go here to see the original:
Eureka Once, Eureka Twice