Name: Video: Reconstructing Virtual Scenes Comprising 3D Avatars of Soldiers Using Image/Video Data
Uploaded: 2024-12-06T12:00:00
Duration: 4 min 34 s
Description: 861 Views. Reconstructing Virtual Scenes Comprising 3D Avatars of Soldiers Using Image/Video Data

reconstructing virtual scenes comprising 3d avatars of soldiers using image/video data

Virtual Scene Generation for Blast Wave Simulations in Military Weapon Training

During military training, service members and instructors are frequently exposed to low-level blasts with heavy and light weapons. Recent studies have shown that blast exposure could lead to decreased neurocognitive performance1,2 and alterations in blood biomarkers3,4,5,6. Repeated low-level blast exposure results in challenges in maintaining optimal performance and minimizing the risk of injury7,8. The conventional approach using wearable pressure sensors has drawbacks, particularly when it comes to precisely determining blast pressures on the head9. The known adverse effects of repeated low-level blast exposure on human performance (e.g., during training and in operational roles) exacerbate this problem. Congressional mandates (Sections 734 and 717) have stipulated the requirement for monitoring of blast exposure in training and combat and its inclusion in the service member&#39;s medical record10.
Wearable sensors can be used to monitor the blast overpressure during these combat training operations. However, these sensors are influenced by variables such as body posture, orientation, and distance from the blast source due to the complex nature of blast wave interactions with the human body9. The following factors affect pressure distribution and sensor measurements9:
Distance from the blast source: Pressure intensity varies with distance as the blast wave disperses and attenuates. Sensors closer to the blast record higher pressures, impacting data accuracy and consistency.
Body posture: Different postures expose various body surfaces to the blast, altering pressure distribution. For example, standing versus crouching results in different pressure readings9,11.
Orientation: The angle of the body relative to the blast source affects how the pressure wave interacts with the body, leading to discrepancies in readings9. Physics-based numerical simulations provide more accurate assessments by systematically accounting for these variables, offering a controlled and comprehensive analysis compared to wearable sensors, which are inherently influenced by these factors.
In response to these challenges, there has been a concerted effort to develop more sophisticated tools. In this direction, the Blast Overpressure (BOP) tool is developed. This tool is developed to estimate overpressure exposure under varying service member postures and positions around the weapon systems. There are two different modules under the BOP tool11. They are (a) the BOP tool SCENE module and (b) the BOP tool SITE module. These modules are used to estimate blast overpressure during weapon firing12. The BOP SCENE module is developed to estimate blast overpressures experienced by individual service members or instructors participating in a training scenario, while the BOP SITE module reconstructs a bird&#39;s-eye view of the training course, depicting the blast overpressure zones generated by multiple firing stations. Figure 1 shows a snapshot of both the modules. Currently, the BOP Tool modules comprise weapon blast overpressure characteristics (equivalent blast source term) for four DoD-defined Tier-1 weapon systems, including the M107 .50 cal Special Application Sniper Rifle (SASR), M136 Shoulder-Mounted Assault Weapon, M120 Indirect Fire Mortar, and Breaching charges. The term weapon blast kernel refers to an equivalent blast source term developed to replicate the same blast field surrounding a weapon system as that of the actual weapon. A more detailed description of the computational framework used for the development of the BOP tool is available for further reference11. The overpressure simulations are run using the CoBi-Blast solver engine. This is a multiscale multiphysics tool for simulating blast overpressures. The blast modeling capabilities of the engine are validated against experimental data from literature12. This BOP Tool is currently being integrated into the Range Managers Toolkit (RMTK) for use on different weapon training ranges. RMTK is a multi-service suite of desktop tools designed to meet the needs of range managers throughout the Army, Marine Corps, Air Force, and Navy by automating range operations, safety, and modernization processes.
<img alt="Figure 1" class="xfigimg" src="/files/ftp_upload/66575/66575fig01.jpg" /> 
Figure 1: Graphical user interface (GUI) for BOP tool SCENE module and BOP tool SITE module. The BOP SCENE module is designed to estimate the blast overpressures on service member and instructor body models, while the BOP SITE module is intended to provide an estimation of the overpressure contours on a plane that represents the training field. The user has the option to choose the height at which the plane is situated relative to the ground. <a href="https://www.jove.com/files/ftp_upload/66575/66575fig01large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
One limitation of the existing BOP tool SCENE module is its use of manually estimated data for building virtual service member body models, including their anthropometry, posture, and position. Manual generation of the virtual service members in the appropriate posture is labor-intensive and time-consuming11,12. The legacy BOP tool (legacy approach) uses a database of pre-configured postures to build the weapon training scene based on the image data (if available). Furthermore, since the postures are approximated manually through visual appraisal, correct postures may not be captured for a complex postural setting. As a result, this approach introduces inaccuracies in the estimated overpressure exposure for individual service members (as a change in posture can modify the overpressure exposure on more vulnerable regions). The paper presents improvements that were made to the existing computational framework to enable rapid and automatic generation of service member models using existing state-of-the-art pose estimation tools. This paper discusses the enhancement of the BOP tool, particularly emphasizing the development of a novel and rapid computational pipeline for reconstructing blast scenes using video and image data. The improved tool can also reconstruct detailed body models of service members and instructors at the moment of weapon firing, utilizing video data to create personalized avatars compared to the legacy approach. These avatars accurately reflect the service members&#39; posture. This work streamlines the process of generating blast scenes and facilitates a more rapid inclusion of blast scenes for additional weapon systems, significantly reducing the time and effort required for weapon training scene creation. Figure 2 shows a schematic of the enhanced computational framework discussed in this paper.
<img alt="Figure 2" class="xfigimg" src="/files/ftp_upload/66575/66575fig02.jpg" /> 
Figure 2: Schematic showing the overall process flowchart in the computational framework. The different steps include image/video data processing, virtual warfighter generation, blast scene reconstruction, and blast overpressure simulations. <a href="https://www.jove.com/files/ftp_upload/66575/66575fig02large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
The paper presents the automated approach being implemented into the BOP Tool, which represents a significant improvement in the computational tools available for estimating the overpressure exposure during training and operations. This tool distinguishes itself through its rapid generation of personalized avatars and training scenarios, allowing for immersive blast overpressure simulations. This marks a significant departure from the traditional reliance on population-averaged human body models, offering a more precise and individualized approach.
Computational tools used in the automation process 
The automation of virtual service member model generation is a multi-step process that leverages advanced computational tools to transform raw image or video data into detailed 3D representations. The entire process is automated but can be adapted to allow manual input of known measurements if needed.
3D pose estimation tools: At the core of the automation pipeline are the 3D pose estimation tools. These tools analyze the image data to identify the position and orientation of each joint in the service member&#39;s body, effectively creating a digital skeleton. The pipeline currently supports Mediapipe and MMPose, which offer Python APIs. However, the system is designed with flexibility in mind, allowing for the incorporation of other tools, such as depth cameras, provided they can output the necessary 3D joint and bone data.
Anthropometric model generator (AMG): Once the 3D pose is estimated, the AMG comes into play. This tool utilizes pose data to create a 3D skin surface model that matches the service member&#39;s unique body dimensions. The AMG tool allows for either automated or manual input of anthropometric measurements, which are then linked to principal components within the tool to morph the 3D body mesh accordingly.
OpenSim skeletal modeling: The next step involves the open-source OpenSim platform13, where a skeletal model is adjusted to align with the 3D pose data. Pose estimation tools do not enforce consistent bone lengths in the skeleton, which can lead to unrealistic asymmetry in the body. The use of an anatomically correct OpenSim skeleton produces a more realistic bone structure. Markers are placed on the OpenSim skeleton to correspond with the joint centers identified by the pose estimation tool. This skeletal model is then rigged to the 3D skin mesh using standard animation techniques.
Inverse kinematics and Python scripting:&#160;To finalize the pose of the virtual service member, an inverse kinematics algorithm is employed. This algorithm adjusts the OpenSim skeleton model to best match the estimated 3D pose. The entire posing pipeline is fully automated and implemented in Python 3. Through the integration of these tools, the process of generating virtual service member models has been significantly expedited, reducing the time required from days to seconds or minutes. This advancement represents a leap forward in the simulation and analysis of weapon training scenarios, providing rapid reconstructions of specific scenarios documented using images or video.

Author Spotlight: Development of an Automated Camera-Based System for Real-Time Blast Overpressure Monitoring and TBI Risk Assessment in Military Training

Generation of Warfighter Avatars from Weapon Training Scene Images for Blast Exposure Simulations

Reconstructing Virtual Scenes Comprising 3D Avatars of Soldiers Using Image/Video Data

reconstructing-virtual-scenes-comprising-3d-avatars-soldiers-using

Research

JoVE Journal

Engineering

861 Views. Reconstructing Virtual Scenes Comprising 3D Avatars of Soldiers Using Image/Video Data

Video: Reconstructing Virtual Scenes Comprising 3D Avatars of Soldiers Using Image/Video Data

Military personnel involved in weapon training are subjected to repeated low-level blasts. The prevailing method of estimating blast loads involves wearable blast gauges. However, using wearable sensor data,&#160;blast loads to the head or other organs&#160;cannot be accurately estimated without knowledge of the service member&#39;s body posture. An image/video-augmented complementary experimental-computational platform for conducting safer weapon training was developed. This study describes the protocol for the automated generation of weapon training scenes from video data for blast exposure simulations. The blast scene extracted from the video data at the instant of weapon firing involves the service member body avatars, weapons, ground, and other structures. The computational protocol is used to reconstruct service members&#39; positions and postures using this data. Image or video data extracted from service member body silhouettes are used to generate an anatomical skeleton and the key anthropometric data. These data are used to generate the 3D body surface avatars segmented into individual body parts and geometrically transformed to match extracted service member postures. The final virtual weapon training scene is used for 3D computational simulations of weapon blast wave loading on service members. The weapon training scene generator has been used to construct 3D anatomical avatars of individual service member bodies from images or videos in various orientations and postures. Results of the generation of a training scene from shoulder-mounted assault weapon system and mortar weapon system image data are presented. The Blast Overpressure (BOP) tool uses the virtual weapon training scene for 3D simulations of blast wave loading on the service member avatar bodies. This paper presents 3D computational simulations of blast wave propagation from weapon firing and corresponding blast loads on service members in training.

A computational framework was developed to automatically reconstruct virtual scenes comprising 3D avatars of soldiers using image and/or video data. The virtual scenes help estimate blast overpressure exposure during weapon training. The tool facilitates a realistic representation of human posture in blast exposure simulations in various scenarios.

Military personnel involved in weapon training are subjected to repeated low-level blasts. The prevailing method of estimating blast loads involves ...