Biocybernetic VR/AR Training System for De-Escalating Conflict

Information Technology and Software

Biocybernetic VR/AR Training System for De-Escalating Conflict (LAR-TOPS-377)

Adapts VR/AR environment to trainee ‘s or player’s biofeedback signals

Overview

NASA Langley’s innovative biocybernetic system employs a blend of hardware and software within a virtual/augmented/mixed reality (VR/AR/MR) environment. Its purpose is to provide psychophysiological self-regulation training, specifically geared toward helping individuals de-escalate conflicts in real-world scenarios. In this system, trainees or their avatars engage with simulated social situations involving conflict. The virtual characters they interact with are designed to present varying levels of threat or cooperation. What sets this approach apart is its continuous measurement of the trainee's cognitive and emotional states through biofeedback. The system's feedback mechanism is distinct, and drives the behavior of the virtual characters. Trainees are rewarded when they effectively achieve the desired mental and physiological states, resulting in desirable character responses. Conversely, ineffective states lead to undesirable character behavior, effectively penalizing the trainee. NASA has tailored this technology for de-escalation training, aiming to reduce physical responses in situations such as customer service interactions, flight attendant-passenger interactions, or police encounters. The system adapts the behavior of non-player characters based on the trainee's ability to attain the desired state, providing a dynamic and immersive training experience.

The Technology

NASA’s biocybernetic system is a cutting-edge technology designed to cultivate emotional regulation skills. It leverages the concept of biocybernetic adaptation, where the trainee engages with virtual entities, such as characters in VR/AR/MR environments, whose behavior dynamically responds to the trainee's physiological signals. This responsive system provides real-time feedback, incentivizing the trainee to attain a calmer physiological state. The key components of this VR innovation include: · Head-mounted display hardware · Physiological monitoring hardware, tracking heart rate, breathing, sweat, breath, and brain waves · Software, powered by the Biocybernetic Loop (BL) Engine, integrating physiological data into the VR simulation · Character response avatars · Integration of the trainee's biofeedback data with the VR environment This technology relies on two functional elements working in unison to adapt the behavior and appearance of VR/AR/MR characters. Inference of the trainee's emotional state from physiological signals requires the implementation of advanced machine learning and modeling techniques. A pattern comparator stores templates of physiological patterns and continually assesses the proximity of the trainee's real-time physiological activity to the desired patterns. The pattern comparator calculates a closeness score in relation to one or more reference patterns, transmitting this data to the VR/AR/MR environment components. Consequently, the level of threat or cooperation presented by virtual characters is dynamically adjusted in response to the closeness score, creating an immersive and adaptive training experience.

Benefits

Enhances Social Skills: Professionals like law enforcement officers dealing with agitated citizens or soldiers interacting with antagonistic civilians can improve their de-escalation and self-regulation skills.
Elevates Gaming Experience: In a VR setting, it empowers game characters/entities to adapt and respond to the player's physiological changes, making games more engaging.
Offers Real-Time Consequences and Rewards for Self-Control: It provides immediate feedback for managing one's mental state, making it a valuable tool for self-improvement.
Prepares for Sound Decision-Making in High-Stress Situations: Users can train to make effective decisions under extreme stress, which can be crucial in various fields.
Measures Mental State: This technology enables the objective measurement of physiological responses to stress, allowing for accurate benchmarking and tracking of improvements.
Proven Proof of Concept: At a Technology Readiness Level (TRL) of ¾, this technology is ready for adaptation into specific use cases, showcasing its viability and readiness for implementation.

Applications

De-escalation training: Law enforcement, military agencies for soldier training, airline training for pilots, flight attendants, and gate agents, and classroom teacher training
Virtual and Augmented Reality gaming: Enhancing interaction and system intelligence via real-time adaptations based on detected conscious or unconscious human neurological and other biophysical states.
Autonomous vehicles: Monitoring the psychological states of drivers to inform the design of the autonomous car experience “feeling of being safe” in their self-driving car to optimize the driver’s mental state.

Download Fact Sheet as a PDF

Ask a Question

Apply to License

Technology Details

Category Information Technology and Software

Reference Number LAR-TOPS-377

Case Number(s) LAR-19961-1

Patent(s) (pop-up blockers must be disabled)

Attachments

Papers

Tags:

de escalate deescalation deescalation training control your state biofeedback biocybernetic AR VR gaming adaptation feedback changes NPC changes background changes behavior avatar immersive social skills conflict resolution training calm your body

Similar Results

Pervasive Biocybernetic Adaptation

The system seamlessly integrates into real-life environments, including homes, cars, or cockpits for pilots. This enables users to train within their daily routines as they move through various work and leisure settings. The technology provides continuous physiological feedback, motivating users to regulate their cognitive and emotional states, with rewards being reflected in the environment. For instance, a driver or pilot can monitor alertness and see changes in their dashboard once a desired state is reached. The system comprises three key components: Talisman: Biofeedback sensors worn on the user's body. Artifacts: Receivers placed in the environment to receive biofeedback signals from the talisman. Software elements that facilitate changes in the operation and appearance of the environment. A pattern comparator stores physiological pattern templates and calculates the proximity of the user's current physiological activity to desirable or undesirable patterns. The reward calculator uses this data to determine when and what rewards are introduced or removed from the user's experience, both qualitatively and quantitatively. As of now, the system has a Technology Readiness Level (TRL) of 3, indicating it is a concept with proof-of-concept analysis.

Player preparing to practice putt wearing the VRZONE headset.

Apparatus and Method for Biofeedback Training

Measured values of physiological signals may be associated with physiological states and may be used to define the presence of such states. For example, in a physiological state of anxiety, adrenaline diverts blood from the body surface to the core of the body in response to a perceived danger. As warm blood is withdrawn from the surface of the skin, the skin temperature drops. Similarly, in a physiological state of stress, perspiration generally increases making the skin more conductive to the passage of an electrical current, thereby increasing the galvanic skin response. It is well known in the field of performance psychology that the peak performance of a task, such as, for example, putting in golf, foul shooting in basketball, serving in tennis, marksmanship in archery or on a gunnery range, shooting pool, or throwing darts, requires the presence of a physiological state, comprising one or more optimal measured values of physiological signals, coincident with the physical performance of the task. The presence of such an optimal physiological state in athletics is colloquially referred to as being in the zone. The technology provides: an apparatus and method of performance-enhancing biofeedback training that has intuitive and motivational appeal to the trainee, by tightly embedding the biofeedback training in the actual task whose performance is to be improved; and, an apparatus and method of performance-enhancing biofeedback training that is operational in real-time, precisely at the moment when a task or exercise, such as an athletic or military maneuver, is required to be performed. The feedback behavior of the physical environment provided by the present invention has the added benefit of providing aids to visualization that the trainee can use in the real-world skill performance setting.

System for Incorporating Physiological Self-Regulation Challenge into Parcourse/Orienteering Type Games and Simulations

Although biofeedback is an effective treatment for various physiological problems and can be used to optimize physiological functioning in many ways, the benefits can only be attained through a number of training sessions, and such gains can only be maintained over time through regular practice. However, adherence to regular training has been a problem that has plagued the field of physiological self-regulation limiting its utility. As with any exercise, incorporating biofeedback training with another activity encourages participation and enhances its usefulness.

ZONE (Zeroing Out Negative Effects)

The system uses perturbation feedback to help the athlete get into the zone through an original method of ZONE. The method allows a trainee to learn physiological self-regulation in order to modify the difficulty of the performance task and/or environment in which training is conducted. For example, better concentration leads to a variety of easier conditions on a training putting green. The technology incorporates software and hardware to provide real-time feedback to the athlete about how close his or her arousal and emotive responses are to an optimal state required to successfully perform the athletic task. This innovation presents the capability to extend current sports training and psychological practices of guided imagery visualization and cognitive reinforcement learning by systematically providing demonstrable and relevant feedback through the use of closed-loop, cybernetic feedback principles that provide immediate reinforcement of pyschophysiological self-regulation and translate into better skill-based performance.

Game and Simulation Control

The technology is constructed to allow modulation of player inputs to a video game or simulation from a user interface device based on the players psychophysiological state. The invention exploits current wireless motion-sensing technologies to utilize physiological signals for input modulation. These include, but are not limited to, heart rate, muscle tension, and brain wave activity. The current capability has been successfully prototyped using the Nintendo Wii console and wireless Wii remote. The experience of electronic game play may also be enhanced by introducing a multiplayer component in which various players collaboratively pursue the goals of the game. The device can also enhance multiplayer experiences such as a video game tournament, in which the skill set required in competitive game play is increased by allowing players to interact with the game, and compete with one another, on a psychophysiological level. This system is compatible with the Nintendo Wii, and prototypes have been designed and are being developed to extend this capability to the PlayStation Move, Xbox Kinect, and other similar game platforms.