For improving scientific reproducibility, biological samples are key

From humble roots in antiquity, the scientific method has come to dominate and transform modern society. Our insights into the natural world, from bacteria to black holes, are fruits of the scientific approach to understanding, as are the life-saving medical innovations society relies on.

The advancement of science critically depends on the ability of researchers to trust the integrity of their experiments. Were all the necessary measurements performed with care? Were proper procedures used? Were precautions against contamination observed? Perhaps most importantly, does the experiment yield the same results when repeated?

Doing good science and producing results that are accurate and consistent can be a formidable challenge, particularly in the life sciences, where even slight variations in experimental protocol can yield markedly different results.

A recent correspondence in the journal Nature Methods takes aim at another potential source of scientific error: the handling of biological specimens used by researchers to conduct their experiments, according to lead author Joshua LaBaer, executive director of the Biodesign Institute at ASU and director of the Virginia G. Piper Center for Personalized Diagnostics. 

In recent years, several studies have raised serious concerns regarding the reproducibility and reliability of scientific data. The lack of consistency in the procurement and handling of biological specimens is an important contributing factor to this problem. As the authors note, this can result in billions of dollars of public research funding being spent on research that cannot be replicated, undermining confidence in research results cited in the scientific literature.

The letter in Nature Methods points to a climate of significant scientific concern with respect to the reproducibility of experiments, a foundation of the scientific method. Failure to replicate a variety of studies has thrown a good deal of preclinical research into doubt.

In order to carry out vital research, scientists rely on high-quality preserved tissue and fluid specimens. Such samples help scientists better understand disease mechanisms and develop early-warning diagnostics for disease, known as biomarkers. Patients contribute these samples in the interest of advancing scientific knowledge and improving the treatment of disease.

Yet surprisingly, methods of preparing and handling such specimens are not standardized and can vary considerably, depending on which biorepositories particular samples are derived from. When specimens are not treated in an equivalent manner in different studies, scientific conclusions are compromised. The particular set of processing steps used by a given biorepository can have profound effects on experimental outcome.

The authors sampled over 300 scientific papers to explore how particular variables in sample preparation affect experimental results. Such variables include temperature, the length of time tissues are chilled, the steps applied for freezing samples, and chemical buffers in use. It’s vital that such standard operating procedures (SOPs) are consistent between experiments to ensure reproduceable results. For example, Alzheimer’s disease may be diagnosed through the identification of a protein fragment known as amyloid- β, but accurate measurements are critically dependent on precise and stable temperature measurements.

Recent technological advances providing greater sensitivity in experimental tests have heightened the awareness of sample fragility and complexity. But ensuring accurate and appropriate specimen preparation is challenging.

Scientific journals rarely report on the preparation steps preceding a published experiment. In many cases, the researchers involved in the study are not aware of the specifics of sample preparation. Tracking down such information can be a frustrating exercise.

When the current study surveyed 300 biorepository websites, they found that 67 percent of them did not share their SOPs publicly. Some facilities refused to share such information even on request, though these biorepositories were often recipients of public or philanthropic funds. Further, neither journals nor funding agencies request information on sample preparation.

The study makes a number of recommendations to rectify these potentially harmful oversights. It proposes a streamlining of the information flow regarding SOPs, to give researchers immediate and easy access to this important information.

Under the new system, biorepositories would be asked to deposit their SOPs in a centralized database, in fully searchable text form, linking specific procedures to the sample collections that use them. Further, an ID number linking the sample-prep SOP could be easily inserted in the methods section of a journal paper, allowing for simple and efficient retrieval of this information.

This new standardized approach would help researchers identify particular processing steps for various sample collections and find those most appropriate for their own research. To this end, the authors have launched an open project: BiospecimenCommons.org, which lists biorepositories, the collections they store and the SOPs used to process their samples.

The site launch represents a significant advance in the standardization and availability of specimen SOPs, to ensure researchers worldwide have consistent and accurate information on their samples. Biospecimen Commons currently lists 300 biorepositories, with 154 sample-preparation SOPs published to date.

The authors encourage funding agencies to insist that recipients of award money provide their experimental protocols. The transparent processing of tissue samples in research will improve the reliability of scientific results while ensuring the best allocation of limited resources.

“The most important first step towards improving how clinical samples are handled is transparency," LaBaer said. "By sharing our methods with each other and comparing outcomes, we will learn the approaches that work the best for each experimental need. We hope that Biospecimen Commons will provide a forum for that kind of sharing.”