Usefulness, not perfection: Psychology professor studies statistical models to understand behavior and outcomes

If Arizona State University’s Michael Edwards had a slogan for his research, it would be a statement made popular by the British statistician George E. P. Box: “All models are wrong, but some are useful.”

Edwards, who was recently promoted to full professor in the Department of Psychology, uses the example of designing a car for how statistical models often work. An early model of a new car might be carved out of foam and be missing details like door handles, but the foam carving provides useful information about how aerodynamic the shape of the car is.

At ASU, Edwards studies and develops statistical models based on item response theory. These models capture the relationship between observable behaviors, like how someone responds to a question, and unobservable traits, like how depressed someone is. He also works on best practices for fitting models to data.

“When I’m creating a model, I start by asking the question, ‘How do we as scientists measure what we think we’re measuring?’” Edwards said. “And everyone should ask the question, ‘Does this model, simple as it may be, capture meaningful features of what is going on in reality?’”

These questions frame how Edwards designs the statistical models he uses. It is more important that the model capture something useful than it perfectly fit data. A perfect fit of a model to data is something Edwards likened to a purple unicorn because it is something he has never seen.

“A perfect fit is not what matters. What matters is that we define what the model measures — which we call a construct — in a way that allows us to get reasonable answers from the model,” Edwards said.

Tools in the toolbox

Michael Edwards 

Statistical models can be thought of as tools that can be customized to capture the construct the scientists want to measure, whether it be anxiety or something else like level of pain for medical patients. Edwards said the power of a good model does not come from it being an identical representation of a phenomenon like pain level, just that it accurately represents a phenomenon.

The measured construct is often subjective, like the experience of pain, so statisticians like Edwards make that part of the model.

“As long as we have a reproducible way of setting up the construct in the model, we’re not adding noise and can get meaningful answers. Statistics always has an error term; that’s life,” Edwards said. “We just make it as small as we can.”

Edwards recently worked on a model that tracks anxiety levels in children with autism spectrum disorder. Parents answered 72 questions about their children, and Edwards and his collaborators used item response theory to determine which questions were the most useful for understanding how autism-related behaviors and symptoms related to anxiety in the children. The end result was a 25-question survey that reliably measured anxiety in children with autism.

Edwards has also worked on models that test quality of life for medical patients, alcohol consumption in adolescents, the impacts of teenage cannabis use on memory and depressive symptoms in inner-city teenage medical patients.

Written by Kimberlee D’Ardenne