The Problem of Replication
Some scientists have expressed reservations about such efforts, citing potential conflicts of interest that could arise from a private company acting as gatekeeper. Others suggest getting funding agencies such as the NIH to support development of technology to make biomedical research data electronically available.
Indeed, reproducibility is a high priority at the NIH, says Lawrence Tabak, the agency’s principal deputy director. Some NIH institutes are looking for ways to improve peer review processes for grant applications and to provide better training in research methods for scientists. Tabak also says the agency is considering how it could support the validation of preclinical studies linked to proposals for large, expensive clinical trials.
Medical institutions could also help reform the replication process, suggests Bruce Booth of Atlas Venture. Technology transfer offices, which universities have set up to support researchers in patenting their work and creating private companies, might redirect some of their resources to research replication, Booth says. “If they could show third-party data supporting a lab’s findings, the prospects for funding would increase significantly, and failure rates could fall,” he says.
Last May, when Science published the technical comments disproving part of the Alzheimer’s study, they sparked publicity that might encourage other researchers to undertake the often thankless task of attempting replication. But the same issue also included a response from Gary Landreth, a neuroscientist at Case Western Reserve School of Medicine and lead author of the original study, who speculated that the replication failures might be related to how researchers prepared and administered the drug. Meanwhile, other groups are still trying to replicate the study’s results, according to Landreth, who says that findings presented at recent conferences have confirmed bexarotene’s impact on memory in mice. The original research has also spawned investigations into whether the drug might be helpful in treating other diseases.
Nor has the controversy surrounding the lab’s original findings deterred investigations into whether bexarotene could help human Alzheimer’s patients. In one small clinical trial, Landreth and his lab are looking at the drug’s effect on the brains of healthy human volunteers, while the Cleveland Clinic Lou Ruvo Center for Brain Health in Las Vegas is recruiting patients with moderate Alzheimer’s for another trial. These lines of inquiry would ideally be based on much more than that one tantalizing result. Yet pursuing them while replication efforts continue is better than nothing, many would argue—and even if researchers are met with failure, failure still counts as a result.