Tapping Twitter data for in-depth analysis of public opinion

September 20, 2013

Social scientists and others seeking insight into public opinion and behavior might benefit from Twitter-based research being conducted by two Arizona State University computer science doctoral students.

Yuheng Hu and Kartik Talamadupula are developing computer models for analytical systems that can harness massive amounts of data generated by Twitter and organize it in a rapid, reliable and efficient fashion. Twitter-based public opinion analysis Download Full Image

Such a capability can help provide a sound basis for advanced statistical analysis of public opinion that develops in reaction to various events or to the emergence of social and political issues and controversies.

Hu and Talamadupula are doing the research under the supervision of Subbarao Kambhampati, a professor in the School of Computer, Informatics, and Decision Systems Engineering, one of ASU’s Ira A. Fulton Schools of Engineering.

Kambhampati says the students’ research demonstrates how computer science can provide more sound information to help political scientists, social psychologists, linguists, journalists and others in similar fields gain more certainty in exploring societal attitudes and trends.

Gauging public reaction

Hu has created a computer model that performs a process he calls “sentiment analysis,” using data from Twitter messages to determine public reaction to various events. He tested the model during the presidential candidates’ debates before the 2012 national elections.

Hu’s system enabled him, for instance, to quickly determine whether there was a more positive or negative public reaction on Twitter to various comments made by the candidates.

Closely watched events such as political debates and campaign speeches generate thousands of tweets that indicate how the public feels about various subjects. But usually an analysis of Twitter messages is done manually, which provides only a limited sample of tweets with which to determine trends in public opinion.

Analysts “might be able to manually sort through 300 tweets to study language or analyze public sentiment, but they lose out on a lot of information and the process is inefficient,” Kambhampati explains.

Hu’s model can quickly segment and align tweets according to the content of the messages and what they express about events and issues, thus providing a more accurate gauge of public attitudes.

Twitter-based linguistics  

Hu teamed up with Talamadupula for a second project to conduct a computational linguistic analysis of the language used on Twitter. They were curious about whether the language of Twitter most resembles the way people communicate in text messages, e-mails or in the more formal language of magazines.

“All forms of social media and written publications have their own linguistic expectation,” Kambhampati says. Text messages contain language-shortening techniques, abbreviations and slang more frequently than e-mails, for instance.

“Linguists have been debating about the implications of the language of Twitter for some time, but Hu’s and Talamadupula’s research brings a needed large-scale analysis to the discussion,” he says.

Hu and Talamadupula conducted a computational linguistic analysis that took a “snapshot” from a portion of the Twitter fire hose from June to August in 2011. In this snapshot of thousands of tweets they found that the language tends to resemble e-mail and magazine language more than the language used in text messages.

Hu said they found Twitter language “surprisingly formal,” revealing that people resist word shortening and slang despite having to limit tweets to 140 characters.

New social science tool

Kambhampati says both projects exemplify the emergence of “computational social science,” a powerful tool for those in social science fields to more accurately analyze large amounts of data and provide a more solid basis for identifying or predicting trends in societal attitudes and behavior.

“Human behavior is dynamic and often hard to understand, this research has been a great opportunity to know people better through social media,” Hu says.  

Kambhampati says the analysis of social media – something that is used millions of times each day – gives researchers a more vast trove of data than standard polls and surveys from which to derive empirical evidence of public sentiment.

The kind of large-scale data offered by the computational methods Hu and Talamadupula are developing “is just the beginning” of advanced analytics that can have far-reaching impacts on sociological research, Kambhampati says.

Hu and Talamadupula presented their research at the International Conference of Weblogs and Social Media in Boston in July and Kambhampati made a presentation at the International Joint Conference on Artificial Intelligence in August in Beijing, China.

Emerging research field

In addition, Kambhampati recently received a $55,000 Google Research Award to support work related to the Twitter Alignment project. It is Kambhampati’s third Google Research Award.

He says the awards keep coming because Google respects the work ASU researchers are doing in this area and the company has an interest in the emerging field of “computational journalism” tied to Twitter.

“Reporting on Twitter and analyzing what is said are new and emerging areas of journalism,” says Kambhampati. “Our research clearly relates to that emerging field.”

Kambhampati plans to use the Google grant to expand research in “ways that we haven’t even fully realized yet.”

In addition to presenting the research at conferences, Hu spent his summer doing a Microsoft Research Internship in Washington state. He performed similar research focused on using Twitter to predict users engagement in local events.

In collaboration with Srijith Ravikumar, who is pursuing a master’s degree in computer science at ASU, Talamadupula began research on new algorithms to rank Twitter search results according to the interests of specific users. He reported on the research at the Annual Association for the Advancement of Artificial Intelligence (AAAI) conference in Washington state in July.

Their paper has also been accepted for presentation at the Association for Computing Machinery (ACM) International Conference on Information and Knowledge Management conference in San Francisco in October.

Written by Rosie Gochnour and Joe Kullman

Joe Kullman

Science writer, Ira A. Fulton Schools of Engineering


ASU Foundation premieres new season of Presidential Engagement Programs

September 20, 2013

Since 1994, the ASU Foundation has offered an annual series called Presidential Engagement Programs. Known as PEP, the programs were created by Elva Coor, wife of former ASU President Lattie Coor, and expanded by Sybil Francis, wife of current ASU President Michael M. Crow. Their purpose is to offer a gateway to the community, showcasing the benefits of having a major university next door.

That showcase opens on Tuesday, Sept. 24 with an away-presentation in the community of Prescott: “On the Road to Recovery – Economic outlook for greater Prescott and Arizona.” Lee McPheters of ASU’s W. P. Carey School of Business will provide an up-to-the-minute review of economic conditions, historical background and obstacles still to overcome as Arizona continues on the road to recovery. Download Full Image

McPheters will devote particular attention to the greater Prescott area, including how the recent fire disasters will affect ongoing recovery. Other Arizona communities will also be examined, with a look at how Arizona’s economic health and performance compares to that of neighboring Western states. The program is open to the public in the Simmons Community Center, but registration is required. Visit asufoundation.org/pepevents for information.

Throughout the 2013-14 season, PEP is partnering with colleges, departments and institutes throughout the university to provide a broad vista of what ASU is and does. Senior development coordinator Vanessa Barrera de Leyvas manages PEP for the ASU Foundation, and she says these collaborations are rich opportunities for PEP patrons.

“Our partnerships throughout the university have produced some fascinating offerings,” Barrera de Leyvas says. “It’s a pleasure for us to show members of the community that the variety of opportunities available to them through ASU is nearly endless.”

Other programs this fall include “The Future of the Brain,” “An Evening with (early childhood expert) J. Ronald Lally,” “The American Way of War,” “Is K-12 Education Keeping Up?”, “Sophocles ‘Antigone’” and “The Two Koreas.” A full slate of events is also planned for the spring, including President Crow’s annual community conversation in January, and other special events will be added throughout the year.

PEP audiences tend to be curious, enthusiastic and loyal. Kyle Longley is the Snell Family Dean’s Distinguished Professor in the School of Historical, Philosophical and Religious Studies, and a regular PEP presenter. He’ll be featured in “The American Way of War,” beginning in October. Longley says, “It’s so much fun to be in front of an enthusiastic, well-traveled group that asks great questions. Members understand how rewarding it can be learning just for the sake of learning.”

Barrera de Leyvas says it’s gratifying to hear from PEP members about how much they enjoy the programs and that they’re going to tell friends. “One couple told us, ‘We routinely tell our friends about PEP and share the brochure with them in the hope they will enjoy it as much as we do.’ That’s how we know we’re successful,” she says.

The ASU Foundation’s Presidential Engagement Programs are offered from September through May, including single events and multi-week courses. All the events require advance registration and most have a charge for entry. A complete schedule is available at asufoundation.org/pepevents.

Erik Ketcherside, erik.k@asu.edu
Communications Manager | Editorial Services
ASU Foundation for A New American University