The Problem of Replication
In 2012, when the journal Science published a study by a group of Cleveland researchers touting a cancer drug’s success against Alzheimer’s disease in mice, it looked like a potential breakthrough in a field long marked by setbacks and failures. As many as 5.4 million Americans have Alzheimer’s, and by one estimate, there’s a new case diagnosed every 69 seconds. While existing treatments can alleviate symptoms, there’s nothing to stop the progression of the ultimately fatal disease.
But in this study, researchers from Case Western Reserve University School of Medicine described how a drug called bexarotene appeared to clear the buildup of amyloid-beta plaques—protein deposits in the brain that are hallmarks of Alzheimer’s—in mice genetically engineered to have a condition similar to the human disease. The drug also seemed to improve the mice’s memory, cognitive abilities and social behavior. “The plaque reduction was an astounding finding,” says Sangram Sisodia, director of the Center for Molecular Neurobiology at the University of Chicago.
No other drug had attacked the plaques so rapidly. Bexarotene, marketed under the brand name Targretin, belongs to the retinoid class of drugs, related to vitamin A, that had been approved by the Food and Drug Administration in 1999 to treat a type of skin cancer called cutaneous T-cell lymphoma. According to the study in Science, bexarotene activated a gene that boosts production of apolipoprotein E, or ApoE, which can help break down amyloid-beta. Alzheimer’s patients can’t produce enough ApoE to prevent the protein from creating deposits. The drug also seemed to rouse a cellular mechanism that consumes the plaques.
Sisodia was one of many scientists who rushed to replicate the original study. “I think we all went back to our labs and tried to confirm these promising findings by repeating the initial experiments,” he says. Rudolph Tanzi, director of the genetics and aging research unit at Massachusetts General Hospital, also sought to repeat the research, spurred in part by physicians whose patients had heard about the apparent breakthrough and wanted to start taking bexarotene immediately. That was such a common response that in August 2012, The New England Journal of Medicine published an article warning physicians to wait for evidence from human clinical trials. “The results in mice certainly didn’t guarantee the drug would work in people,” says Tanzi. “And although it’s safe for cancer treatment, it can have severe side effects.”
But before testing whether the drug could help human Alzheimer’s patients, scientists needed to make sure it worked in mice—and while some labs were able to reproduce the drug’s effect on memory and cognition, the plaque reduction effect couldn’t be replicated. In four technical comments published in Science last May, several teams of independent researchers, including those co-authored by Sisodia and Tanzi, reported that their replication attempts showed no effect on plaque levels in lab mice treated with bexarotene. Months later, in August, a paper in the journal Molecular Neurodegeneration reported that researchers from Johns Hopkins had also failed to replicate either the plaque reduction or memory and cognition effects found in the Case Western research.