New ASU psychology professor studies how best to replicate scientific experiments

January 9, 2019

If a scientific finding cannot be replicated, can it be true? The replication of experimental findings is a crucial part of the scientific method and is first taught in elementary school science lessons. But researchers from psychology, economics, marketing and medicine have recently struggled to reproduce many findings, giving rise to the so-called “replication crisis” in science.

The reasons why findings might not replicate are numerous and complicated. Though a few instances of scientific fraud have received a lot of attention in the media, a more common reason why a finding might not replicate is the improper or ineffective use of statistics. Samantha Anderson is an assistant professor of psychology at ASU. Download Full Image

Most replication attempts rely on one kind of statistic to determine whether a finding reproduces. This narrow definition of replication can lead to problems, as Arizona State University’s Samantha Anderson knows. Anderson recently joined the Department of Psychology as an assistant professor, and one of the focuses of her work is how the sample size, or amount of data in an experiment, can affect whether a study replicates.

“There are several ways to assess whether a replication study is successful or if it has failed, but right now most people are deciding if a replication is successful in the same way,” said Anderson, who with her doctoral adviser recently won the 2018 Tanaka Award for the best paper from Multivariate Behavioral Research, the flagship journal of the Society of Multivariate Experimental Psychology.

“It is important to expand the definition of what it means to replicate a study.”

Anderson also works on improving how experiments are reproduced using traditional definitions of replication.

Planning appropriate sample sizes with an easy online tool

The statistic that most replication studies rely on is the “p-value,” which gives information about how likely a scientific result is. When researchers attempting to reproduce a study duplicate the original sample size and then compare p-values, the replication attempt can fail even if the finding is actually reproducible.

Figuring out how much data are needed for an experiment or replication might seem straightforward but actually requires thinking about terms like signal and statistical power.

“You want a sample size that is large enough to detect the signal you are looking for, among noise and even errors in the data,” Anderson said. “The probability of you finding that signal is the statistical power.”

Determining the necessary sample size for a study can be challenging because it requires information about the magnitude of the effect being studied, which scientists call the “effect size.” Sometimes scientists might not know much about the signal they are hoping to detect. To replicate an experiment, many researchers logically start by using the effect size reported in the original study because that is the best available information on the magnitude of an effect. But Anderson said using sample effect sizes from published studies at face value can actually cause unintended problems because of publication bias. Scientists rarely, if ever, publish experiments that did not work, so the sizes of published results tend to be skewed higher than they actually are.

Another source of unintended problems in figuring out sample sizes is uncertainty, which exists because experiments often have to measure an effect from just part of a population. For example, politician approval surveys use a small subset of the electorate, usually just a few thousand people, to infer how the American population feels.

To combat publication bias and unravel the best way to handle uncertainty in data, Anderson worked with Ken Kelley at the University of Notre Dame to develop an online tool to make sample size calculations easy for researchers.

The tool, called the “bias uncertainty corrected sample size” or BUCSS for short, is based on statistical methods that were published decades ago and several publications from Anderson’s doctoral research. The BUCSS tool can help researchers plan sample sizes for new experiments and replication studies.

“We developed software that accounts for biases and uncertainty that can result in incorrect sample sizes,” Anderson said. “The BUCSS tool is designed so people can use it without thinking about the equations and math behind it.”

Before coming to ASU, Anderson earned her doctorate at Notre Dame. She started graduate school pursuing her degree in clinical psychology but soon realized she loved methods research. Anderson completed her doctorate in quantitative psychology and won the 2017 Psi Chi/APA Edwin B. Newman Graduate Research Award, which acknowledges the best research paper from a psychology graduate student.

Science writer, Psychology Department


Better drug to save mothers’ lives during childbirth may be on the way

January 9, 2019

Postpartum bleeding is the world’s leading cause of death for women during and after childbirth, and the third-leading cause in the United States alone. Many doctors in developing countries have turned to the drug misoprostol to save more women from deadly hemorrhaging.

Misoprostol, although affordable, has dangerous side effects, including uterine cramping, heart attack, toxicity in the brain and spinal cord, fetal death and fetal heart abnormalities. Image Some important conditions are treated by misoprostol, including ulcers, bleeding inflammation, pain and labor. Illustrated here is misoprostol in the binding site of the hormone prostaglandin E2 receptor 3. Download Full Image

Development of a safer drug to counter postpartum bleeding may be on the horizon, based on new research by Arizona State University, the Bridge Institute at the USC Michelson Center for Convergent Bioscience and the U.S. Department of Energy’s SLAC National Accelerator Laboratory based at Stanford University. The work appears in the journal Nature Chemical Biology.

With the laboratory’s powerful X-ray laser, the scientists created a 3D map of the structure of a cell receptor as it binds to misoprostol. This is a key step for identifying the best potential molecules for drug development.

“Misoprostol is a key drug for women’s health, especially in countries that lack access to medical resources, where it saves many mothers’ lives at childbirth,” said corresponding author Raymond Stevens, director of the Bridge Institute and Provost Professor of Biological Sciences and Chemistry at the USC Dornsife College of Letters, Arts and Sciences.

“The development of newer therapeutics that are more selective with the tissues they target would be an impactful advancement in women’s health,” he said. “This research improves our understanding of how the drug works and provides a starting point for new drug discoveries.”

Hao Hu, Uwe Weierstall and Wei Liu — researchers from the Biodesign Center for Applied Structural Discovery (CASD) and ASU’s School of Molecular Sciences — join scientists and engineers at the USC Michelson Center who have come together from USC Dornsife, the USC Viterbi School of Engineering and the Keck School of Medicine of USC to research and develop new drug therapies, tests and devices to solve significant health issues.

New research has helped to illuminate a class of cell surface receptor proteins known as GPCRs, which are among the most important targets for new pharmaceutical drugs. Until recent advances in X-ray crystallography however, detailed structural understanding of such receptors at the molecular scale was severely limited.

“Determining the atomic structures of proteins in their active states is critical for developing effective drugs with far fewer side effects,” said Petra Fromme, who directs the CASD center. Her innovative work in femtosecond X-Ray crystallography has already helped unveil protein structures that may aid in the development of new drugs for diabetes, extend the understanding of human vision and help to pioneer new therapies against a broad range of diseases, including heart disease, cancer and neurodegenerative disorders.

ASU plans to significantly expand the scope of X-ray crystallography research with the development of the world’s first compact free electron laser. The device, to be housed in the newly completed Biodesign C building, will allow researchers to investigate molecular structure in new ways at a fraction of the cost of massive accelerators like Stanford University’s SLAC facility.

Not just for postpartum bleeding 

The global maternal mortality rate has declined in recent years to an average of 216 deaths per 100,000 live births, according to UNICEF. Rates are improving with awareness, better medical response and increased access to therapies, including misoprostol.

By comparison, the rate in the U.S. is 18 per 100,000 live births, according to the most recent report by the Centers for Disease Control and Prevention. Several news outlets in recent months have noted that is a high rate among developed nations.

Misoprostol, also used to control stomach ulcers or to induce labor, belongs to prostaglandin analogues, a class of drugs so named because they mimic the healing prostaglandin hormones. Prostaglandins are usually triggered into action by illness or injury. Misoprostol specifically targets the prostaglandin receptor EP3, which controls labor induction.

“The human body is built of a few trillions of cells, and these cells have to talk to each other,” said co-author Alex Batyuk, a scientist at SLAC’s Linac Coherent Light Source X-ray laser. “The way they communicate is through these receptors, which sit in the cell membrane and transmit signals in and out of cells.”

Most prescription drugs are designed to target these receptors. An accurate, detailed model of drug molecule-to-receptor interactions is key for researchers to fully understand the impact of a drug on the body.

To make the 3D model of the interaction, the scientists first create a crystal of the drug molecule as it binds to the receptor. When the laser’s X-rays are beamed at the crystal, they form patterns that scientists then use to reconstruct and map the structure. The 3D model enables scientists to conduct computer-modeled tests that then help them identify what the best drug configurations may be for specific health conditions or diseases.

“Trying to design new drugs without understanding the structure of the receptors they bind to is like trying to build a car from the ground up, without a manual,” said lead author Martin Audet, postdoctoral scholar at USC Dornsife. “This could enable us to get even more data about its molecular mechanics — how it moves and interacts with drug molecules — which will help us fine-tune other potential drug compounds and predict how they will impact the body.”

In addition to the human EP3 receptor structure determined by the USC and Stanford researchers, the same issue of Nature Chemical Biology highlighted the structures of the EP4 receptor by friends and colleagues So Iwata and Takuya Kobayashi in Japan, and the related TP receptor structure by friends and colleagues Wu Beili and Qiang Zhao in Shanghai, China.

The study’s co-authors included several researchers from the Bridge Institute and USC Dornsife: Kate White, Barbara Zarzycka, Gye Won Han, Jeffrey Velasquez, David Manahan and Petr Popov, as well as Vsevolod Katritch, assistant professor of biological sciences, and Vadim Cherezov, professor of chemistry, physics and astronomy. Other co-authors were from ASU, Domain Therapeutics, the GCPR Consortium and iHuman Institute.

The work was funded by USC Dornsife, National Institutes of Health grants R01 GM108635 (V.C.), R21 DA042298 (W.L.), R01 GM124152 (W.L.), the STC Program of the National Science Foundation through BioXFEL (No. 1231306) (U.W. and W.L.), the Russian Science Foundation (project no. 16-14-10273) and the GPCR Consortium. M.A. was supported by a Canadian Institute of Health and Research (CIHR) Postdoctoral Fellowship Award. C.G. acknowledges the Panofsky Fellowship from SLAC National Accelerator Laboratory and Stanford University for financial support. P.P. acknowledges the Russian Foundation for Basic Research (RFBR No.18-34-00990). Parts of this research were carried out at the LCLS, a National User Facility operated by Stanford University on behalf of the U.S. Department of Energy and is supported by the department’s Office of Science, Office of Basic Energy Sciences under contract no. DE-AC0276SF00515 and at the GM/CA CAT of the Argonne Photon Source, Argonne National Laboratory.

— Adapted from story by Emily Gersema