advertisement-vertical Download Proto magazine app
Social Icons

The Problem of Replication

icon-pdfpdf icon-printprint

Some scientists have expressed reservations about such efforts, citing potential conflicts of interest that could arise from a private company acting as gatekeeper. Others suggest getting funding agencies such as the NIH to support development of technology to make biomedical research data electronically available.

Indeed, reproducibility is a high priority at the NIH, says Lawrence Tabak, the agency’s principal deputy director. Some NIH institutes are looking for ways to improve peer review processes for grant applications and to provide better training in research methods for scientists. Tabak also says the agency is considering how it could support the validation of preclinical studies linked to proposals for large, expensive clinical trials.

Medical institutions could also help reform the replication process, suggests Bruce Booth of Atlas Venture. Technology transfer offices, which universities have set up to support researchers in patenting their work and creating private companies, might redirect some of their resources to research replication, Booth says. “If they could show third-party data supporting a lab’s findings, the prospects for funding would increase significantly, and failure rates could fall,” he says.

Last May, when Science published the technical comments disproving part of the Alzheimer’s study, they sparked publicity that might encourage other researchers to undertake the often thankless task of attempting replication. But the same issue also included a response from Gary Landreth, a neuroscientist at Case Western Reserve School of Medicine and lead author of the original study, who speculated that the replication failures might be related to how researchers prepared and administered the drug. Meanwhile, other groups are still trying to replicate the study’s results, according to Landreth, who says that findings presented at recent conferences have confirmed bexarotene’s impact on memory in mice. The original research has also spawned investigations into whether the drug might be helpful in treating other diseases.

Nor has the controversy surrounding the lab’s original findings deterred investigations into whether bexarotene could help human Alzheimer’s patients. In one small clinical trial, Landreth and his lab are looking at the drug’s effect on the brains of healthy human volunteers, while the Cleveland Clinic Lou Ruvo Center for Brain Health in Las Vegas is recruiting patients with moderate Alzheimer’s for another trial. These lines of inquiry would ideally be based on much more than that one tantalizing result. Yet pursuing them while replication efforts continue is better than nothing, many would argue—and even if researchers are met with failure, failure still counts as a result.

previous // next
icon-pdfpdf icon-printprint

1. “Why Most Published Research Findings Are False,” by John Ioannidis, PLOS Medicine, August 2005. In this seminal study on replication, Ioannidis, a Stanford University epidemiologist, uses statistical models and other key factors to demonstrate why the vast majority of published research findings are false.

2. “Believe It or Not: How Much Can We Rely on Published Data on Potential Drug Targets?” by F. Prinz, T. Schlange and K. Asadullah, Nature Reviews Drug Discovery, September 2011. In an industry analysis, three Bayer HealthCare scientists report that in-house experiments over four years failed to replicate two-thirds of 67 research studies in the fields of oncology, women’s health and cardiovascular diseases.

3. “Drug Development: Raise Standards for Preclinical Cancer Research,” by C.G. Begley and L. Ellis, Nature, March 29, 2012. A chronicle of scientists at American drug company Amgen who tried to replicate 53 studies they considered landmarks in the basic science of cancer—and were able to replicate only six.

Protomag on Facebook Protomag on Twitter