New ASU psychology professor suggests best practices for data analysis

Psychology is the study of human behavior, but not all psychologists directly study people. Some psychologists like Daniel McNeish, a new assistant professor in the Department of Psychology at Arizona State University, figure out the best ways for scientists to understand their data.

McNeish creates methods to answer questions when only a small amount of data are available. A small dataset can be the only option for scientists when studying a special group of people such as children with a specific learning disability, or when it is too expensive to gather a large dataset, like when scientists collect human neuroimaging data.  

Psychologists analyze their data using statistics, and most statistical methods assume there is a large dataset. To test which statistical methods work well with small datasets, McNeish creates simulated data, which he calls “his imaginary friends.” He creates a new small dataset and, because he knows what the answer is from the start, he can figure out which ways of analyzing the data are the most accurate. Some of his recent findings were published in the flagship journal of quantitative psychology, Multivariate Behavioral Research, and this paper was the most-read article in the journal during 2016.

McNeish is also interested in how findings from small datasets are interpreted. In a recently published article in Educational Researcher, he investigated how well standardized test scores, such as those used for college admission, predict performance compared to testing data acquired throughout a student’s education.

“Standardized tests give a snapshot of performance at one time point,” McNeish said. “Our proposed method looks at data from many time points, and you can get a better picture of student performance.”

Using publicly available testing data of test scores from kindergarten through eighth grade, McNeish and his collaborator indeed were able to better predict student performance. 

“If you look at the whole dataset over time, some criticisms of standardized tests, such as they are biased against low-income students, are reduced.” McNeish said. “The problem is not necessarily with the test itself, but with how scores are interpreted. A snapshot gives limited information, and if you can expand the scope [of the data], you can make more reliable inferences.”

From the Midwest to the desert Southwest

McNeish, who hails from Massachusetts, started his undergraduate studies at Oakland University in Michigan. He quickly realized that he enjoyed studying both math and psychology, so he asked his first-year statistics instructor for advice on how to choose between the two disciplines.

“She told me all about psychometrics,” McNeish said. “I transferred to Wesleyan University because they had psychometrics, and I basically did independent studies for two years.”

Soon after moving to Connecticut, McNeish started studying with Steven Stemler, associate professor of psychology at Wesleyan University. McNeish would go to Stemler’s office, and Stemler would ask what he wanted to learn about that day. They delved into topics such as item response theory and structural equation modeling as McNeish earned his undergraduate degree in psychology.

Next, McNeish moved to the University of Maryland to pursue graduate studies in statistics.

“Maryland has so many statistics departments because the government is so close by,” he said. “Some are very nuanced and focused, like the program in survey statistics. Many of those graduates go work at the census bureau.”

At Maryland, McNeish earned his master’s and doctoral degrees through the department of behavioral statistics. He worked as an assistant professor at Utrecht University in the Netherlands and as a research scientist at the University of North Carolina before joining the psychology department at ASU.  

“Coming to ASU was a no-brainer because the quantitative group is world-renowned,” McNeish said. “When I found out I had an interview with the ASU psychology department, I withdrew from my other interviews so I could put all my efforts into getting the job. Fortunately, it worked out!”