Our methodology adds important stages, selection of sensor combination, placement and classification, to simplify the fall detection system with a deep analysis. There are previous works that address some fall detection design issues, but there is no work that focuses on holistic methodology for overcoming all of these problems. This methodology can also be used for human activity recognition in assisted living, sport performance evaluation, physical therapy and rehabilitation applications.
When a dataset is created, challenges due to synchronization, organization, and data inconsistency can arise. A tradeoff between an accurate estimation and the model complexity should be taken into consideration. Demonstrating the procedure will be Jose Pablo Nunez Martinez, a research assistant, and Sofia Pacheco Ibanez, an undergraduate engineering student from our laboratory.
Begin by setting up the data acquisition system to facilitate subject data collection and storage. Select the types of wearable sensors, ambient sensors, and vision-based devices required as sources of information and assign an ID for each source of information, the number of channels per source, the technical specifications, and the sampling rate of each of them. To connect all of the sources of information to a central computer or a distributed computer system, first verify that the wire-based devices are connected properly to a single client computer and verify that the wireless-based devices are fully charged.
To set up each device to retrieve data, set the data acquisition system to allow data storage on the cloud and confirm that the data acquisition system fulfills the appropriate data synchronization and data consistency properties. Be sure to check that all of the sensors are acquiring data consistently and simultaneously and to include labels for identifying subject activity and traits. Collect sample data with the devices and store the data in a preferred system.
Query the database and determine if all of the sources of information are collected at the same sample rates. After considering the conditions required in the restrictions imposed by the goal of the system, set up the testing environment placing a mattress or any other compliant flooring systems in the center of the environment to ensure participant safety. Keep any objects at least one meter away from the mattress and prepare any necessary personal protective equipment for the participants.
Then set up the appropriate cameras and pair infrared sensors around the mattress as illustrated. Define the goal of the fall detection and human activity recognition system on a planning sheet and define the target population of the experiment in accordance with the goal of the system. Define the type of daily activities including some non-fall activities that look like falls to improve real fall detection.
Assign an ID for each activity and describe the activities in as much detail as possible. Then set the time period for each activity to be executed. Define the type of human falls and assign an ID and describe each fall for each activity along with the time period for each fall to be executed.
Consider if the falls will be self-generated by the subjects or generated by others and write this information on the planning sheet. To collect the activity and fall data, place the recording devices onto the subject as illustrated. When the subject is ready, under the supervision of a clinical expert or a responsible researcher, start the data collection in the data acquisition system and ask the subject to perform the sequences of activities and falls outlined in the planning sheet saving the timestamps of the start and end of each activity or fall.
Verify that the data from all of the sources of information are saved on the cloud after each activity or fall. To analyze the collected activity and fall data, use the feature dataset for each machine learning method to run a K-fold cross-validation. Use a common metric of evaluation like accuracy to select the best model trained per method.
Next, open the training feature dataset in the preferred programming language software and use the pandas library to read a CSV file as indicated. Split the feature dataset in pairs of inputs/outputs as indicated. Select one machine learning method and set the parameters.
Train the machine learning model and compute the estimates values of the model using the testing feature dataset. Repeat the K-fold cross-validation of the number of times K is specified in the K-fold cross-validation for each machine learning model selected. Select the suitable placements in the multimodal approach if a combination of two or more sources of information are required for the system and select the best source of information for each modality in the system.
Create a combined feature dataset using the independent datasets of these sources of information and select the machine learning classification method. Train a model for these combined sources of information and repeat the validation using the combined feature dataset. Then prepare a new dataset with the subjects under more realistic conditions using only the sources of information selected in the previous analysis.
Graphical representation of the best performance obtained for each modality depending on the machine learning model and the best window length configuration clearly illustrates that multimodal approaches obtain the best F1 score values compared to some unimodal approaches. Although notably using wearable sensors only, a similar performance to a multimodal approach can be obtained. In terms of the benchmark of the data-driven models, random forest presents the best results in almost all of the experiments while multilayer perceptron and support vector machines are not very consistent in performance.
The best performance is obtained when a single sensor is used at the waist, neck, or tight right pocket. Ankle and left wrist wearable sensors performed the worst. In addition, the waist, neck, and tight right pocket sensors with random forests classifier in a three-second window size with 50%overlapping are the most suitable wearable sensors for fall detection.
The lateral view camera performs the best fall detection and the best camera location is in a lateral view point using random forest in a three-second window size and 50%overlapping. In addition, the random forest model classifier demonstrates the best performance in accuracy and F1 score in both multi-modalities and the combination between the waist and camera one ranks in the first position. In our experiments, we recruited young people with no impairments, but subjects should be selected that align with the goal of the system and the target population using the system.
Simple multimodal fall detection systems can be designed and implemented based on this methodology. For real world adaptation, transfer learning and deep learning approaches are recommended for developing robust systems.
