This paper outlines the assessment of infants' gross motor performance with a multisensor wearable and its fully automated deep learning-based analysis pipeline. The method quantifies the posture and movement patterns of infants from lying supine until they master walking independently.
Developing objective and quantitative methods of early gross motor assessment is essential to better understand neurodevelopment and to support early therapeutic interventions. Here, we present a method to quantify gross motor performance using a multisensor wearable, MAIJU (Motility Assessment of Infants with a JUmpsuit), which offers an automated, scalable, quantitative, and objective assessment using a fully automated cloud-based pipeline. This wearable suit is equipped with four movement sensors that record synchronized data to a mobile phone utilizing a low-energy Bluetooth connection. An offline analysis in the cloud server generates fully analyzed results within minutes for each recording. These results include a graphical report of the recording session and a detailed result matrix that gives second-by-second classifications for posture, movement, infant carrying, and free playtime. Our recent results show the virtue of such quantified motor assessment providing a potentially effective method for distinguishing variations in the infant's gross motor development.
Early gross motor development is essential for higher-level neurocognitive performance that emerges later by supporting infants' exploration of the environment. Therefore, clinicians and researchers alike have a high interest in assessing early gross motor development1,2,3. To provide support for evidence-based medicine or scientific studies, it is essential that the gross motor assessments are quantitative, reliable, objective, and ecologically valid. However, there is a scarcity of such methods available for either clinical or basic science research.
A typical early gross motor development progresses through a predictable sequence of newly acquired skills. They are commonly observed in infants as reaching discrete motor milestones4, where standing and walking are often considered significant landmarks en route to more complex behavioral repertoire5. In addition to direct observation or parental surveys about motor milestones, several widely used standardized batteries have been developed6,7,8,9 for performing infants' assessments in the laboratory or hospital environment. However, these assessments suffer from multiple caveats: they need substantial expertise from trained professionals, they are partly subjective and categorical, and they assess infants' performance in an environment (hospital or laboratory) that is unnatural from an infant's perspective.
Recording infants' spontaneous motor activity over longer times in the native environment, such as their home, enables more relevant measures of motor abilities. In one such feasible method, the assessment is performed for the full sequence of the infant's motor ability development from lying supine to walking fluently with a wearable system, such as the MAIJU wearable (Motor Assessment of the Infants with a JUmpsuit)10,11,12. The MAIJU wearable system (Figure 1) involves a whole-body textile garment equipped with movement sensors to allow unsupervised out-of-hospital/laboratory assessments and recordings that are analyzed with an automated pipeline, providing a second-by-second assessment of posture and movement patterns. These algorithmic detections can be used for each posture and movement type separately, or they can be combined for a holistic assessment of the maturational level of the infant's motor abilities. A recently published, unit-free expression of such metric of motor maturity is BIMS (Baba Infant Motor Score)10,12.
This article will describe the assessment of infants' gross motor performance using a multisensor wearable suit; the rationale, practical performance, analysis pipeline, and potential future perspectives for using the metrics that can be obtained from the automated analysis pipeline available for recordings with a multisensor wearable10,11,12. The method is suitable for a detailed quantitation of spontaneous gross motor activities in all infants that exhibit motor abilities between supine lying and fluent walking.
The multisensor wearable system consists of three components: 1) the full body overall garment equipped with four movement sensors, 2) a mobile device using a custom-built iOS application, and 3) a cloud-based analysis pipeline (Babacloud the credentials for which can be obtained from the authors)11. The waterproof inertial measurement unit (IMU) sensors stream synchronized data (3-axis accelerometer and gyroscope) at 13-52 Hz sampling frequency to a mobile phone using a low-energy Bluetooth connection. The data is initially stored in the memory of (the sensor or) the mobile device, followed by an offline analysis in the cloud server after the recording is stopped.
The studies carried out with this system were reviewed by the Ethics committee of the New Children's Hospital, Helsinki University Hospital, and the hospital gave permission to carry out the research projects described in the original studies cited in this work. An informed consent was obtained to film the child in the video.
1. Preparing the suit for a recording session
2. Preparing and dressing the infant for a recording session
3. Recording session
4. Cloud-based analysis: upload of raw data and download of results
The presented method quantifies infants' gross motor performance by classifying the types of postures and movements for every second of the recording session. Therefore, the results package from the automated analysis pipeline includes a full classification matrix (Supplemental file 3) and a graphical summary (Supplemental file 4) from the entire recording session. Depending on the exact study question, these results can be inspected at different levels.
Results inspection was used for the development and validation of this method.
In the following, we present four levels of results inspection used for the development and validation of this methodology. Supplemental file 5 presents representative examples of the key validation experiments that were previously published in full detail10,11,12.
First, the automatic algorithms trained for movement and posture detections were validated against second-by-second-level human observations of infants' movement behavior. We used several parallelly trained experts who blindly reviewed the synchronized video recordings with the wearable recordings. All different posture and movement categories were compared separately to the individual human annotations, and we found a very high agreement between the algorithm and human for postures (average kappa 0.93); a substantial agreement was found for movement categories (sub-category specific kappa mostly at the range of 60-80%). See Supplemental file 5A for an example confusion matrix10. Also, interrater agreement levels were assessed to confirm that the algorithms perform at about human equivalent level10,11.
Second, we assessed how well the classifier-based quantification of movement and posture categories would match the corresponding quantification from the second-level human annotations. Example scatter plots shown in Supplemental file 5B10 demonstrate that several key categories have a nearly perfect match (correlation coefficient >0.96) between algorithmic and human visual quantitation. This directly supports the idea that the age-specific distributions of motility quantitation (Figure 3A,B) are reliable12.
Third, the idea of a holistic assessment of motor maturity was validated by training a developmental age prediction from the combination movement quantities (see above) that correlated very closely with the actual age in the typically developing infants (r=0.89; Supplemental file 5C). Subsequently, the age prediction was scaled to 0-100 as a unitless measure BIMS (Baba Infant Motor Score10), and its utility for building motor growth charts (Supplemental file 5D) was validated using a typically developing infant cohort highly showing age-dependent and predictable growth trajectories. We also validated its relative accuracy by showing that the accuracy in motor growth charts compares well with the well-known physical growth charts12.
Fourth, the potential for abnormality detection with the given metrics was validated in a proof of principle experiment where individual motor measures were shown to clearly differentiate between infants with poor and good motor performance, respectively (Supplemental file 5E)10.
Potential further study questions at different levels of analyses
Figure 3 shows examples of further uses for the information provided by the wearable suit and its automated analysis pipeline. First, the early development of posture and movement skills can be plotted as a function of age and compared to the age-dependent distributions (Figure 3A, "growth charts"12), or the development can be tracked over time for each individual (Figure 3B). Second, when a study question requires a more holistic gross motor assessment, one can use a combination of an individual's posture and movement proportions (as shown in Supplemental file 5D, computed from the time series in Supplemental file 5C) to generate an index like BIMS (Baba Infant Motor Score). Such measures support direct use in motor growth charts (Supplemental file 5C,D) and computation of statistical derivations like z score (Figure 3A). Using the full-time series of algorithm's detections (Figure 3C, and Supplemental file 3) allows studies on the detailed temporal structure of infants' motility with questions like "How many posture transitions does the infant perform in a unit time?" or "What is the distribution of standing epochs during spontaneous playtime?".
Figure 1: Overview of the multisensor wearable system and a typical study flow from the recording to the analysis. Figure 1 adapted from Airaksinen et al.12published under CC_BY license. The infant photograph is published with parental consent. Please click here to view a larger version of this figure.
Figure 2: Posture and movement categorization and an example visualization of raw data and analysis output (A) Posture and movement categorization scheme used by the algorithmic classifiers within the fully automated analysis pipeline for a multisensor wearable. This figure is reprinted from Airaksinen et al.10 (B) Example of a 10 min of raw accelerometer sensor data from each of the four arms as it comes out from the MAIJU recordings. The horizontal bars below depict automated classifier outputs for the posture (upper bar) and movement (lower bar) detections for the same 10 min epoch. Please click here to view a larger version of this figure.
Figure 3: Example results derived from the raw classifier outputs. (A) An example of a data comparison between infant age and Baba Infant Motor Score (BIMS). The S-shaped green curve depicts the developmental trajectory of BIMS in a larger population. The dot depicts an example individual measured at 14 months, with a BIMS ~74, corresponding to a little below the average age-typical level (the green line). (B) An example cohort with individual trajectories of motor development using the Baba Infant Motor Score over (BIMS). Each line represents an infant recorded at several age points (dots in the line). The lines are colored for the average deviance relative to the age-typical mean (blue S-shaped curve in the background; see also panel A. (C) Representative output matrix from the automatic classifier as it comes from the Babacloud pipeline. The first column depicts the elapsed time from the recording start (in seconds) for each analysis window in the classification (window duration 2.3s, with 50% overlap). The second and third columns show the classifier detection for posture and movement, respectively. The third and fourth columns are auxiliary classifiers depicting epochs when the infant was carried by someone else, and when the infant was engaging in autonomous play, respectively. (D) An illustration of a summary report. Panels A and B are adapted from Airaksinen et al.10. Please click here to view a larger version of this figure.
Supplemental File 1: A quick guide to recording with MAIJU wearable. Please click here to download this File.
Supplemental File 2: A quick guide to transferring data from Maijulogger to the analysis server in Babacloud. Please click here to download this File.
Supplemental File 3: An example of a detailed classification matrix that gives all second-by-second classifications for posture, movement, infant carrying, and free playtime. Please click here to download this File.
Supplemental File 4: A full PDF file example of a graphical summary report. This is taken from the algorithm file that includes, (A) basic background information about the recording session (subject ID and age, the recording date, duration, and the total amount of epochs used for the final analyses). (B) A graphical display of the full recording indicating distribution of postures over the full recording and epochs that are excluded from the quantitative assessment. C) Typical distributions displayed with violin plots for all six postures (left-hand side) and 12 movement types (right-hand side). Similarly, drawings on the right side depict the type of movements indicated by each posture, also showing the sequence of incrementally developing motor performance (the dots indicating the results from an individual recording and colorful violin plots indicating age-dependent distributions of posture/movement of a relevant dataset). Notably, the raw values indicated by the dots show actual amounts of the given motor performance, and they can be used directly in other contexts. Please click here to download this File.
Supplemental File 5: Validation experiments at different levels of analysis. (A) Confusion matrices showing agreement between human annotations (target class) and the algorithm's detections (predicted class) for both the posture and the movement categories. (B) Comparison of motor quantitations over full recording sessions between human annotations and detections derived from the automated classifier. (C) Correlation between developmental age prediction from the wearable data (left side Y axis) and its re-scaling to generate the BIMS score (right side Y axis). The actual age of the infant at the time of recording is shown on the X-axis. (D) Correlation of the age prediction and the actual age when using a fitted function. The values depict fit to the model when using group average at the depicted time windows (blue), all raw values (black), or when accounting for repeated measures of each individual (green). (E) Comparison of individual motor measures between groups of well and poorly performing infants suggests that several automatically detected motor metrics may differentiate these infant groups. Panel A,B,C is adapted from Airaksinen et al.10. Panel D is adapted from Airaksinen et al.12. Panel E is adapted from "Airaksinen et al.11. Please click here to download this File.
A quantified assessment and developmental tracking of infants' motor performance with a wearable solution, such as MAIJU, is technically simple to learn and perform, and it can be readily implemented into health care or clinical research practice10,11,12. Compared to the other existing motor assessment methods, this kind of at-home recording of infants' spontaneous motor activity improves the ecological validity of the assessment. Furthermore, it provides a quantified, transparent, and fully automated analysis of infants' motor performance. Most importantly, the metrics used in the analysis are intuitive and explainable, which enables their easy comparison with other clinical and research assessments, such as environmental factors, cognitive development, or psychosocial assessments. A holistic assessment of motor development provides an accuracy that compares well with the conventional physical growth measures12.
Critical steps in the protocol include careful preparation of the wearable suit. When preparing for a recording, choosing the correct size for the suit is crucial, as the sensor attachments in the sleeves and legs are required to sit tightly to obtain a reliable recording of body movements. Also, for a successful recording, it is essential to place the sensors in the pockets with a correct orientation, as indicated in the protocol. The sensor mounts will not allow sensors to rotate during the recording. However, the incorrectly oriented sensor records data that is difficult, if not impossible, to fix afterward. The infant should be encouraged to move freely and independently during the recording. The recording length may vary according to the given study questions. The multiple spontaneous movement epochs are combined to accumulate enough spontaneous movement for each recording session.
The flexible and practical operation of the MAIJU wearable solution allows its use in variable contexts in both supervised and unsupervised settings, such as research labs or homes. Recent results from our clinical trials show that fully unsupervised recordings conducted at home may provide comparable results with recordings that are done under full or partial supervision12. Still, a child's spontaneous motor behavior is potentially affected by several factors, such as the surroundings (e.g., playing outside vs. indoors, the layout of the space, furniture, and toys), the child's level of alertness, and the parents' involvement during the home recording. When the recordings are performed in unsupervised settings at home, it is important to encourage the child to play spontaneously, i.e., to play or move independently, without someone else carrying or holding the child if not necessary, and to keep the recording mobile phone at a Bluetooth range (in the same room)10. The majority of our current troubleshooting situations during the recordings are caused by loss of Bluetooth connection. Near-future advances in sensor technology will improve Bluetooth connectivity, and the upcoming introduction of a larger sensor memory will allow offline recording by storing movement data directly in the sensor memory.
Out-of-hospital recordings with a wearable solution of this kind are readily scalable and they may improve safety of infants, e.g., by enabling remote monitoring during circumstances such as a pandemic. Our present classifier algorithms were trained to specifically recognize the given motor abilities, postures, and movements shown in the motility description scheme (Figure 2A). These phenomena were identified as characteristic of infant movement during the first two years of life. Other types of movements or postures seen in older children, such as running or jumping, will require modified movement description schemes and respective algorithms to be trained to identify them. Posture-context dependent analysis is a potentially fruitful approach where an infant's motor activity is analyzed separately in different postures to support studying, e.g., developmental correlates of infant behavior5,6,7,8,9,13. Alternatively, a context-dependent movement analysis could also support assessing asymmetry in motor function when predicting the development of unilateral cerebral palsy10,12,14,15. Further, assessment of motor abilities with the MAIJU system may be combined with other study modalities, e.g., eye tracking, imaging, or video recording, to provide multimodal data, spanning it to different types and contexts. Multimodal data may be useful, e.g., in evaluating the effects of social interaction or the efficacy of therapeutic intervention.
For the success of novel wearable technologies in out-of-hospital monitoring environments with infants, certain limitations, challenges, and ethical concerns need to be addressed. Our analysis pipelines were trained and validated using typically developing infants in Finland10,11,12. The raw analysis outputs with pure postures and movements should be universal. However, their developmental trajectories may require adjustments for diverse cultures and geographical locations. According to parent feedback regarding wearable devices, they are viewed favorably due to infant-friendliness16. However, parents may raise concerns regarding privacy, data access, and family practicalities (e.g., multiple caregivers, visitors, and varying schedules). Dependency on the battery life of the sensors and the recording phone can be considered a limitation of the method. In our experience, the battery model (CR2025) typically lasts for the full day (12-24 hours) when using continuous data streaming. Notably, it depends on both the battery brand and the strength of the Bluetooth connection needed for wireless data transmission, which is continuously changing to maximize data transmission in the recording environment. For instance, a long distance between the infant and the phone or a wall between them would adjust the Bluetooth connection to significantly higher battery consumption. Notably, the batteries of most mobile devices are also drained within about the same time if using continuous Bluetooth streaming. In practice, the presently used continuous data streaming over Bluetooth connection implies that both the sensors and mobile devices need a daily recharge/battery replacement. The near future introduction of sensors with larger memory capacity will allow data storage in the sensor memory, supporting over a week of continuous recording. That will remove the need for power-consuming Bluetooth streaming, as well as carrying the phone within a Bluetooth range that may be perceived as restrictive in recording situations and is susceptible to human error.
Overall, tracking of early neurodevelopment needs methods that are sensitive to natural neurobehavioural variability. Gross motor development is an intricate process consisting of variations in the order and timing, both on individual and cultural levels4. Detection of atypical motor development is effective in recognizing infants at risk for an extensive range of neurodevelopmental disorders. Traditional test batteries with standardized neurodevelopmental assessments are performed in controlled environments, such as hospitals, and are at least partially subjective7,8,9. Current advances in sensor technology and signal analysis have enabled recordings of infants' spontaneous motor ability over extended periods in out-of-hospital settings and quantitation of the motor behavior at an accuracy comparable with human observers10,11,12. Novel wearable technology offers automated and scalable methods for monitoring movement and the efficacy of therapeutic intervention in infants in an ecologically valid and objective way. Furthermore, the novel neurodevelopmental index Baba Infant Motor Score (BIMS) enables the estimation of infants' maturity of motor ability by individual tracking of neurodevelopment10,12. It can be employed in a range of future applications, such as the development of infant motor growth charts12. By training the automated classifiers for other specific motilities (e.g., for older children or adults) with different kinds of movement description schemes and algorithms, the wearable movement sensors have the potential for clinical applications, such as movement disorders or follow-up on the effects of therapeutic interventions regardless of the developmental stage of the individual17. Currently, however, this should be viewed as an investigational methodology that should not be used to inform clinical diagnosis or treatment targets.
This work was supported by the Finnish Academy (314602, 335788, 335872, 332017, 343498), Finnish Pediatric Foundation (Lastentautien tutkimussäätiö), Aivosäätiö, Sigrid Juselius Foundation, and HUS Children's Hospital/HUS diagnostic center research funds.
Name | Company | Catalog Number | Comments |
iOS device (version 16.5 or higher) | Apple | n/a | |
MAIJU jumpsuit | Planno Ltd | n/a | customized for purpose |
Maijulogger (mobile application) and sensor firmware | BABA Center (www.babacenter.fi), Kaasa solutions GmbH | n/a | constructed by Kaasa Solutions, distributed by Baba Center |
Movesense movement sensor | Movesense (www.movesense.com) | n/a |
This article has been published
Video Coming Soon
ABOUT JoVE
Copyright © 2024 MyJoVE Corporation. All rights reserved