New AI Framework Improves Teamwork Training Technologies

Monday - June 26 2023, 22:10 UTC - 1 year ago

Researchers have developed a new Artificial Intelligence (AI) framework with the goal of improving team training technologies by accurately assessing and categorizing dialogue between individuals. To test the performance of the new framework, the researchers compared it to two previous AI technologies using data from a training mission and found that the new framework performed substantially better than the previous AI technologies.

Researchers have developed a new artificial intelligence (AI) framework that is better than previous technologies at analyzing and categorizing dialogue between individuals, with the goal of improving team training technologies. The framework will enable training technologies to better understand how well individuals are coordinating with one another and working as part of a team."There is a great deal of interest in developing AI-powered training technologies that can understand teamwork dynamics and modify their training to foster improved collaboration among team members," says Wookhee Min, co-author of a paper on the work and a research scientist at North Carolina State University. "However, previous AI architectures have struggled to accurately assess the content of what team members are sharing with each other when they communicate." .

"We've developed a new framework that significantly improves the ability of AI to analyze communication between team members," says Jay Pande, first author of the paper and a Ph.D. student at NC State. "This is a significant step forward for the development of adaptive training technologies that aim to facilitate effective team communication and collaboration." .

The new AI framework builds on a powerful deep learning model that was trained on a large, text-based language dataset. This model, called the Text-to-Text Transfer Transformer (T5), was then customized using data collected during squad-level training exercises conducted by the U.S. Army.

"We modified the T5 model to use contextual features of the team—such as the speaker's role—to more accurately analyze team communication," Min says. "That context can be important. For example, something a team leader says may need to be viewed differently than something another team member says." .

To test the performance of the new framework, the researchers compared it to two previous AI technologies. Specifically, the researchers tested the ability of all three AI technologies to understand the dialogue within a squad of six soldiers during a training exercise.

The AI framework was tasked with two things: classify what sort of dialogue was taking place, and follow the flow of information within the squad. Classifying the dialogue refers to determining the purpose of what was being said. For example, was someone requesting information, providing information, or issuing a command? Following the flow of information refers to how information was being shared within the team. For example, was information being passed up or down the chain of command? .

"We found that the new framework performed substantially better than the previous AI technologies," Pande says.

"One of the things that was particularly promising was that we trained our framework using data from one training mission, but tested the model's performance using data from a different training mission," Min says. "And the boost in performance over the previous AI models was notable—even though we were testing the model in a new set of circumstances." .

The researchers also note that they were able to achieve these results using a relatively small version of the T5 model. That suggests that the approach can be used with limited resources.