Examining Minds Under Stress

You are a soldier running through a battlefield, ground rumbling beneath your feet as bombs explode nearby. You can feel the tension level rising in your body, your heart racing, saliva drying in your mouth. Your only hope is to follow a far-off beacon, floating in the sky and pointing the way to your objective, your team, and safety.

This is no video game or virtual reality amusement park—it’s a scientific lab at Tufts, with a very real goal: to develop new techniques that could boost performance and enhance survival of troops on the battlefield.

The Center for Applied Brain and Cognitive Sciences (CABCS) at Tufts is a unique partnership with the U.S. Department of Defense, engaging faculty in Tufts’ School of Arts and Sciences and School Engineering to help understand and equip soldiers of the future. The results of the research won’t just help the military, however—they are also applicable to any front-line personnel who work in stressful mental and physical environments.

“Medics, firefighters, law enforcement—we’ve had them all in here,” says Tad Brunyé, AG07, principal scientist and scientific manager for the center. He is standing in the middle of a  virtual reality (VR) cave, a large room with wraparound video screens and a raised metal “rumble platform” where test subjects experience simulations of a variety of missions taking place in many kinds of terrain.

“We’re really interested in how people orient themselves and adapt in complex urban and subterranean environments with constantly changing demands,” Brunyé says.

In addition to VR goggles and an air rifle equipped with white motion-tracking balls, participants wear a bio-harness that measures heart rate, respiration, and other physical characteristics so the researchers can monitor their physiological measures and motions.

Sometimes they also wear a shock belt that transmits a mild electric shock when they make a mistake, or just at random times to heighten their sense of stress and anticipatory anxiety. “Especially when we’re bringing in military personnel to participate, it makes it much more realistic and immersive for them,” Brunyé says.

The room is one of three VR caves in the lab on the ground floor of the new Joyce Cummings Centerin Medford; multiple team members can collaborate simultaneously in the same virtual environment so researchers can study teamwork.

By analyzing performance in virtual tasks, the researchers hope to determine, for example, what information is useful to provide to soldiers in a goggle display—like that virtual beacon pointing the way to a gathering point.

“You can imagine this in your forward field of view with an augmented reality system, giving you updated information to help you reach your objective,” Brunyé says.

A Collaborative Project

The VR caves are just one of several tools at CABCS dedicated to measuring, predicting, and improving cognitive performance of soldiers and emergency personnel under intense circumstances.

“Wherever there is a high-stress, high-stakes environment, we are looking at how we can monitor, optimize, and provide the necessary information for people to succeed,” says Holly Taylor, co-director of the center and a psychology professor who has long explored the field of spatial cognition.

More than a dozen years ago, Taylor started collaborating with researchers at the Combat Capabilities Development Command Soldier Center, a research lab in Natick, Massachusetts run by the U.S. Army that explores all aspects of soldier performance, nutrition, equipment, and technology.

Over the years, several of Taylor’s psychology graduate students have gone on to work as scientists at the Army center in Natick with Brunyé and Caroline Mahoney, AG02, a cognitive scientist and co-director of CABCS, including Aaron Gardony, AG16, Marianna Eddy, AG08, and Grace Giles, AG16. While working together on a number of grants, Taylor and Mahoney quickly realized how much better the Army scientists and Tufts faculty could collaborate together if they were in the same space.

“We wanted to figure out ways the university could benefit from understanding the applied challenges faced by soldiers, and we could benefit as well from the tremendous expertise of multidisciplinary Tufts faculty,” says Brunyé. In 2014, Tufts and the Army signed a five-year cooperative agreement, earmarking funds to build CABCS and support Tufts faculty on research.

In 2019, they renewed the agreement for another 10 years. The cognitive and psychological research is bolstered by strong collaborations with engineering faculty and students, led by Eric Miller, one of the lab’s scientific managers, a professor in the Department of Electrical and Computer Engineering, and director of the new Tufts Institute for Artificial Intelligence, who is spearheading development of processing methods for data coming off of new wearable biosensors and other equipment to better quantify  the connection between physiological state, brain activity, and performance.

At the same time, the collaboration provides a good environment for student research on the cutting edges of human-computer interaction.

“A lot of students are interested in those topics, but there aren’t a lot of other laboratories to support those interests,” Brunyé says.

Walking and Chewing Gum—the Extreme Version

Some projects explore how people regulate their emotions in intense environments. “In high-stress situations, brain systems that support goal-directed cognitive processes can start to shut down,” said Heather Urry, a professor of psychology and expert on emotion regulation and stress who is a scientific manager at the center. “That makes it difficult to react in ways that allow you to perform whatever task is at hand.”

Others explore how humans make sense of information when they are dealing with too much of it all at once. “In these high-stakes environments, when you are trying to process multiple streams of information simultaneously, you can suffer from pure cognitive overload,” says Taylor.

Still others look at how people’s brains function under physical stress. In the center’s Physical Activity and Brain Lab—nicknamed the PhAB Lab—sits a pair of treadmills and physiological monitoring equipment, where subjects attempt cognitive tasks while exercising—sometimes while wearing heavy backpacks to simulate conditions in the field. “It’s like trying to walk and chew gum at the same time—only a more extreme version of it,” quipped Brunyé.

Research has shown that some physical exertion can increase mental performance, while in other circumstances, it degrades it. “If we’re able to monitor you and know how long you’ve been moving and under what conditions, we can make a much better prediction about what’s going to happen to your performance,” Brunyé said.

That, in turn, can help determine what tasks—cognitive, memory, or control—might be enhanced by exercise, which should be delayed, and which could be shared across individuals and teams.

Other center lab spaces explore more sedentary activities. In the EEG (electroencephalogram) lab, test subjects sit in a chair and look at a TV screen while wearing a cap implanted with electrodes. The headgear can measure changing electrical activity from the brain at the surface of the scalp, in speeds down to the millisecond.

“We are really interested in understanding what’s happening in the brain with very high temporal precision,” said Brunyé. Some studies, for example, have combined EEG and eye tracking technology to test certain types of camouflage, detecting the moment that a subject spots a person hidden in the environment.

“We can start to understand and quantify the advantage of little, subtle changes in camouflage patterns and how well they blend into backgrounds,” Brunyé said.

Other studies can measure the amount of cognitive overload subjects feel when performing certain tasks, helping researchers develop new ways to detect mental exhaustion in the field.

“A lot of people really aren’t aware of what they can and can’t do,” said Taylor. “Having real-time metrics can help you know when somebody is overloaded and change something about their context—reallocating tasks to a different individual or administering some sort of performance enhancement techniques.”

Improving Facial Recognition with Neurostimulation

One of those techniques the center is evaluating in the lab room next door is neurostimulation, which basically takes the opposite approach of the EEG Lab. Instead of measuring electrical signals, a cap administers very low-intensity electrical pulses to stimulate neurons through the scalp.

“If you licked a nine-volt battery you would get much more of a shock than what this is doing,” said Brunyé reassuringly. Some studies have shown that fifteen or twenty minutes of such tiny stimulation can increase cognitive performance—though the data is far from conclusive.

“You have a lot of research showing it’s doing something, a lot of research showing it’s doing nothing, and a very small amount of research showing it actually impairing performance,” Brunyé said.

So far, the Tufts researchers have seen a variety of results. In one study, for example, they found that neurostimulation targeting a brain area associated with face processing improved facial recognition—something that could be helpful for soldiers in identifying enemies, or police officers in identifying criminals.

In another study, the researchers found that neurostimulation in fact made people less likely to learn information. By better determining the circumstances under which the technique improves or degrades outcomes, said Urry, the center can provide a valuable service in recommending to the Army whether to use it or not.

“If we’re going to select this from the range of potential enhancement techniques, we need to make damn sure as scientists that we’re confident with the efficacy of this for all people in all contexts,” Brunyé said.

Using Augmented Reality

That also goes for the kind of visual enhancements that the center is exploring for use in real-life situations. The nearby augmented reality lab is mostly a big empty space. Wearing AR goggles, however, subjects can see and interact with virtual objects at the same time they are seeing actual objects in the environment (as opposed to VR, in which they only see the virtual world).

Microsoft contracted with the U.S. military to use its HoloLens technology, and now the Tufts team has been experimenting with how it can be best used. “We can simulate a subterranean tunnel network in here,” Brunyé says. “A rat comes running out, and people literally jump.”

Such experiences could help train soldiers or emergency personnel in entering difficult-to-navigate environments. “How can you train firefighters who need to go into a burning building to find whether there are people still in there?” asked Taylor. “If you could give them information about the layout of the building very quickly on the way to the fire, they’re going to be a lot more successful than if they are randomly looking through a building and risking getting lost in a smoke-filled environment.”

Using augmented reality, a team of firefighters or soldiers could all look at a topographical map of an area together, even using eye-capture technology to make sure that everyone is looking at the same features.

Like many research projects, learning to use AR for mission planning has had its share of challenges, which in turn has produced some useful insights. In tests, the researchers have found that people quickly get overwhelmed and confused by a three-dimensional landscape.

“The complex spatial information actually seems to be too much of burden for people,” Brunyé said. Instead of taking advantage of a topographical map of a city or landscape, he said, many people tilt the viewer up to make it two-dimensional like a map. “They defeat the entire purpose of the augmented reality within the first two minutes,” he said. What’s more, they seem to actually perform better at orientation tasks with that view.

While it’s possible that viewers could learn to better navigate an augmented landscape with experience, the researchers have found that they can improve performance by providing less information, stripping a detailed landscape down to its essential elements.

“To make a quick decision in an environment, people only require a pretty low level of detail,” Brunyé said. In addition to making soldiers or responders more effective, simplifying the display could also result in reducing bandwidth by up to 80 percent when downloading satellite information in the field, potentially cutting costs and saving time in life-and-death situations.

Tipping Points

In addition to using AR for mission planning, the center is also exploring how to use it in heads-up displays during a mission itself. That’s where the VR caves come in, allowing the researchers to test information with soldiers and other personnel in a realistic environment, so they can tweak what will be most effective in a real-life situation. One potential application is for navigation through dense urban environments in which wars are increasingly being fought.

In those situations, soldiers can lose satellite navigation in urban canyons, or have cell service actively jammed by enemies, and need to rely on inertial navigation systems using gyroscopes–rotation sensors that calculate location by dead reckoning as they move. Those systems, however, can accumulate errors as a person walks.

“In short distances, it doesn’t matter much, but if you are going miles and you are off by 10 degrees, it is huge,” Brunyé said. In tests with soldiers, the researchers have introduced errors into the beacons, to see how subjects react and how much error they can tolerate.

“We were able to provide the exact tipping points when they stopped relying on the system,” Brunyé said. That information could help engineers better refine the technology. “It was a really nice way of taking research from the VR cave to do something in the real world.”

Enhancing Stress Management

The brain center is also using the VR caves to conduct studies on stress and emotions, giving subjects different strategies for managing stress before exposing them to intense simulations. In one study that is still ongoing, for example, researchers are comparing cognitive reappraisal, in which subjects learn to reframe the situation to reduce negative emotions, with mindfulness, in which they accept their feelings without fighting them.

By judging the subsequent performance in the VR environment, the researchers aim to determine whether training in these techniques could enhance performance in high-stress contexts. In upcoming work, Urry also plans to examine whether people’s typical emotion regulation tendencies might predict performance under stress.

“Maybe there are certain people who are well-suited to certain performance situations because they’re good at regulating negative emotions,” said Urry. “If you know that a person is good at using something like reappraisal, then that may predict their ability to manage their negative emotions and, thus, perform well in a high-stress context.”

In all of these kinds of studies, the researchers are helping to create a total picture of how people’s minds react in over-the-top stressful scenarios, and how they can better complete their missions and survive.

“The Department of Defense has come to realize that individuals and small teams of people are still the most integral part of an effective operation,” said Brunyé. “So, then the question becomes, what does it mean to have a highly capable human, and how do we get there?”

Michael Blanding is a Boston-based freelance writer.