ASU research reveals 'weak' replicability of place-based research

Across the scientific community, the repeated testing of studies has always been central to progress. Reproducing and replicating research not only validates prior findings, but it also validates research methods and data that could then be applied to solve other elusive problems and accelerate future research. 

But compared with the scientific fields of physics, chemistry and biology, dialogue around the reproducibility and replicability of research in the social and environmental sciences, like geography, has been largely absent and focused on computation challenges. 

In a recent perspective paper published in the Proceedings of the National Academy of Sciences of the United States of America, Michael Goodchild, research professor in Arizona State University's School of Geographical Sciences and Urban Planning, and Wenwen Li, professor in the school, conduct a novel analysis, shedding new light on the challenges and opportunities the scientific community faces in replicating place-based research. 

“Reproducibility is new in geography,” Li said, whose research focuses on computational spatial science. “If a result is reproducible, we can accelerate science. Other researchers will be able to more easily leverage your data and your model to solve different problems. ... We want to build upon and stand on the shoulders of others.” 

Spatial heterogeneity and weak replicability 

In their research, Goodchild and Li, who are both faculty affiliate members of ASU’s Spatial Analysis Research Center, analyzed prominent studies on replicability across the geographic domain and examined why and how place-based methods affect replicability.

The team reviewed methodological arguments and geospatial approaches currently used in the social and environmental sciences and introduced new relevant methods of machine learning like geospatial artificial intelligence (GeoAI). 

Goodchild and Li concluded spatial methods only possess “weak replicability,” a new academia term they coined for its inability to be as replicable as other scientific laws like in biology or physics.

“Results in the social and environmental sciences don't easily generalize to other times and places, unlike those of physics or chemistry,” said Goodchild, a global leader in geographic information science and lead author of the paper. “In the environmental and social sciences things are more complex.”

Chiefly, the researchers argue the weak replicability of place-based methods is due to spatial heterogeneity, or the uneven distribution of variables unique to specific locations.

“When we are studying social or environmental phenomena we usually have study areas, like Phoenix or Tempe or other locations, so a lot of the methods that we develop are place-based,” Li explained. “Every region has different social and environmental factors like climate or population or culture; these factors cause spatial heterogeneity, and when we change the place, variables may well change.”

Advancing reproducibility in spatial science 

Goodchild and Li say that further research is necessary to quantify the variation of just how “weak” replication of place-based methods may be in varying geographies. 

The researchers are encouraging other spatial scientists to develop more research in this area and propose the creation of future “replicability maps” that could determine how reproducible certain methods are at specific locations.

“A study in Phoenix may be more replicable in Tucson, compared to other places like Colorado, because they are close to each other and share common characteristics,” Li said. “If we can do an analysis across the U.S. or the entire world, we would be able to generate a replicability map that could quantify the degree of replicability of solution methods.

“What we are calling for is some solutions; we are in the process of advancing reproducibility in spatial sciences.”