Rethinking research impact

The way we think about the impact of university research, according to one ASU leader, needs urgent and fundamental change.

“Since the beginning of the new administration, there has been an avalanche of opinions and perspectives on the need for a new social contract for science,” said David Guston, associate vice provost for discovery, engagement and outcomes, at an Oct. 8 event held by Arizona State University’s Consortium for Science, Policy and Outcomes.

How the university research community can prioritize planning for the impact of research was the focus of the event — the second in the consortium's Rethinking Outcomes series — which drew leaders from the research community and Capitol Hill to ASU’s Barrett & O'Connor Center in Washington, D.C., for a discussion.

“The past year has been tumultuous in science and tech policy, to put it mildly, but the tensions have, in fact, been building for much longer,” said CSPO Director Arthur Daemmrich when he opened the event. “Our view is that a path forward is a focus on impact-oriented science and technology policy, and that word 'impact' has come into wide use.”

Guston proposed an “impact catechism,” an idea inspired by the Heilmeier Catechism, an evaluation heuristic comprising eight questions that identify the context, costs and effort required to execute a project and which was made famous by DARPA.

But unlike the Heilmeier Catechism, Guston’s set of eight questions specifically focuses on perceived impacts of research. Ideally, Guston’s catechism could help academics truly guide change from the very beginning of knowledge production.

The eight impact-probing questions that make up Guston’s impact catechism:

1. What kind(s) of impacts are you aiming at?
2. What scope and depth of impact are you planning for?
3. What specific audience(s) are you addressing or constructing?
4. What (causal) model do you have in mind for creating impact?
5. What people or networks connect your outputs to impacts and outcomes?
6. How are you creating opportunities for impact?
7. How are you participating in, researching or keeping track of (intermediate) impacts along the way?
8. How will you tell the story of the impact that you have with humility and accuracy?

If used right, researchers could magnify the potential impact of their research from its inception, cultivating an “impacts and outcomes orientation by beginning knowledge production with these questions in mind,” Guston explained.

Guston’s presentation also explored the history of the social contract for science, whether universities should be understood as a "club good" (instead of a public good) and similarities between the society-shaping impacts of legislation and technology. Guston suggested that democracies should consider similar democratic practices for universities and institutions creating technology — or technological impact.

Attendees pressed Guston on his proposals, with some calling his presentation a “quite radical critique.” Sarah Rovito from the National Academies asked Guston to explain why now was the right time to be revisiting the social contract for universities.

In response, Guston claimed that the university research community has been off track since the 1980s.

“We have failed to be explicit about the value that we deliver and have failed to build institutions that would maximize the public value that we deliver from the work that we do in universities,” he said.

To Guston, being explicit about the value of university research requires answering all eight questions in his proposed impact catechism and moving away from an overemphasis on cost.

“We in the university research community need to be explicit. We need to be explicit about how we deliver benefits, the outcomes we're supposed to be delivering,” said Guston. “It is not enough for science policy to talk about the ‘How much?’ question. You also have to address the why and the how and the who questions.”

ASU, as a prototype for the direction of the American public research university, is dedicated to the simultaneous pursuit of excellence, broad access to quality education and meaningful societal impact.

Guston’s ideas are an essential piece of the puzzle as the sector rethinks how to design research for social impact — and how to communicate about it.

“At the core of these proposals is a different imaginary — in it, faculty, staff and students learn the skills to change the world, not only through research, publishing, patenting and profit-seeking outputs, but through public service, policy design and implementation, through public interest, technology development and knowledge exchange with community-engaged research,” said Guston.

“If universities want to deliver on the goal of having a beneficial impact on their community, their state, their nation or the wider world, they must find a way to be explicit about how their work is making an impact.”