As an undergraduate, Tejaswini Yelamanchili would spend hours a day playing video games like Counter-Strike and Age of Empires. Time would speed by – hours seemed like minutes – as she focused on the process of gaming. Now a graduate student at Missouri S&T, she’s spending much of her time getting others into gaming as part of her research to better understand how the brain works when players are in the zone.
This sense of being fully immersed and energized in some enjoyable activity – whether it’s playing a video game, a musical instrument or a set of tennis – is known in psychology as “flow.” It is a mental state involving intense concentration and focus in which one can lose all sense of time and space.
Although the flow phenomenon has been studied for years, researchers don’t yet fully understand what’s happening in the brain when people are in a state of flow, says Yelamanchili, who is pursuing her master of science degree in information science and technology from Missouri S&T. Working with Dr. Fiona Fui-Hoon Nah, a professor of business and information technology at S&T, Yelamanchili seeks to fill that knowledge gap by studying how people engage with a classic video game: Tetris.
“It is very difficult to measure flow,” says Nah, whose research specialties include human-computer interaction. The most common approach is through questionnaires in which researchers ask subjects how they felt while or after engaging in some activity, such as playing a video game.
But the questionnaire approach is not very reliable, Nah says. People don’t always remember precisely how they felt during an experience, or they may downplay or exaggerate their feelings or even make false statements.
“This is a self-report measure,” Nah says of the questionnaire approach, “and with this type of approach, there are a lot of challenges. We want to have a more precise assessment of flow.”
Nah and Yelamanchili are looking for “the neuropsychophysiological correlates of different states of user experience,” Nah writes in a recent paper on the topic.
Specifically, they’re interested in how the brain behaves not only during flow, but also during two other states: boredom and anxiety. By comparing brain activity from these three states, they hope to determine how the brain functions in each. They also hope brain activity will offer a map that indicates when a player moves from one state to the next – from boredom to flow or from flow to anxiety.
Their research could also help designers, developers and usability specialists improve the overall user experience of computer-based systems, such as games, as well as educational products, smart devices and a broad array of other computerized systems.
To measure these different states of user experience, Nah and Yelamanchili have an ample supply of students who are willing to play a video game for the sake of science. The researchers put an electroencephalogram (EEG) headset on each subject to measure their brain activity while the students play Tetris.
By today’s gaming standards, Tetris, a tile-matching game that first hit the market in 1984, is not the flashiest or most advanced product around. But it works well for Nah and Yelamanchili because it allows them to measure how players react under different conditions, such as boredom (at the beginning of a game, when the tiles fall slowly), flow (when players experience a sense of mastery over the game yet still feel challenged) and anxiety (when the tiles are falling so quickly that players feel that they have lost control).
After enough experimentation, and by comparing the EEG results with other measures, they hope to be able to more precisely show what happens in the brain during states of flow.
In addition to relying on the EEG headsets, Nah and Yelamanchili record the players on video during game play so they can note facial expressions and correlate these visual cues with what the EEG tells them. They also show the videos to the students and ask them to discuss how they were feeling throughout the game play. The researchers’ goal here is to examine these recollections while considering the brain activity that was recorded by the EEG headset.
Nah and Yelamanchili are also using questionnaires as part of their research. This approach adds another dimension to the study by allowing them to compare what people say about their experience with what their brain activity shows.
“To fully understand flow, we need a strong understanding of the human brain, as well as an understanding of the behavioral and physiological aspects of flow,” Yelamanchili says. The video recordings and questionnaires will help her and Nah better assess these behavioral and physiological aspects, she adds.
Last May, Yelamanchili presented a paper on this topic, titled “Neural Correlates of User Experience in Gaming,” at the 2017 Midwest Association for Information Systems conference at the University of Illinois at Springfield. Co-authors of the paper are Nah; Dr. Keng Siau, professor and chair of business and information technology at S&T; and Dr. Langtao Chen, assistant professor of business and information technology at S&T.
Nah has published and presented several papers on the topic of flow in the context of her research on human-computer interaction. She presented a paper titled “A Review on Neuropsychophysiological Correlates of Flow,” which she co-authored with Yelamanchili and Siau, at the 19th International Conference on Human-Computer Interaction last July in Vancouver, British Columbia. Nah will present their latest work at the 24th Americas Conference on Information Systems this August in New Orleans, Louisiana.
This project has sparked the interest of other graduate students in pursuing creative research. For example, Chandana Mallapragada is also pursuing her master of science degree in information science and technology at S&T and will use machine-learning techniques to carry out predictive analytics with brain mapping. Mallapragada will be presenting her latest work this May at the 2018 Midwest Association for Information Systems conference at the University of Missouri-St. Louis.
Together, Yelamanchili and Mallapragada are applying multiple statistical and analytical approaches to identify neuropsychophysiological correlates of user experience. This research project has provided Yelamanchili and Mallapragada with “invaluable research experience as data scientists of the future,” says Nah.