Name: design and analysis for fall detection system simplification
Uploaded: 2020-04-06T13:00:00
Duration: 8 min 5 s
Description: Watch this Scientific Journal Video about Design and Analysis for Fall Detection System Simplification at JoVE.com

This research has been funded by Universidad Panamericana through the grant &#8220;Fomento a la Investigaci&#243;n UP 2018&#8221;, under project code UP-CI-2018-ING-MX-04.

All methods described here have been approved by the Research Committee of the School of Engineering of Universidad Panamericana.
NOTE: This methodology is based on the configuration of the specific types of sensors, machine-learning methods and procedures in order to configure a simple, fast and multimodal fall detection and human activity recognition system. Due to this, the following protocol is divided in phases: (1) database creation (2) data analysis (3) system simplification and (4) evaluation.
1. Database creation
<ol>
	<li>Set up the data acquisition system. This will collect all data from subjects and store the information in a retrieval database.
	<ol>
		<li>Select the types of wearable sensors, ambient sensors and vision-based devices required as sources of information. Assign an ID for each source of information, the number of channels per source, the technical specifications and the sampling rate of each of them.</li>
		<li>Connect all sources of information (i.e., wearables and ambient sensors, and vision-based devices) to a central computer or a distributed computer system:
		<ol>
			<li>Verify that wired-based devices are connected properly to one client computer. Verify that wireless-based devices are fully charged. Consider that low battery might impact wireless connections or sensor values. Moreover, intermittent or lost connections will increase loss of data.</li>
		</ol>
		</li>
		<li>Set up each of the devices to retrieve data.</li>
		<li>Set up the data acquisition system for storing data on the cloud. Due to the large amount of data to be stored, cloud computing is considered in this protocol.</li>
		<li>Validate that the data acquisition system fulfills data synchronization and data consistency20 properties. This maintains the integrity of data storage from all the sources of information. It might require new approaches in data synchronization. For example, see Pe&#241;afort-Asturiano et al.20.
		<ol>
			<li>Start collecting some data with the sources of information and store data in a preferred system. Include timestamps in all data.</li>
			<li>Query the database and determine if all sources of information are collected at the same sample rates. If done properly, go to Step 1.1.6. Otherwise, perform up-sampling or down-sampling using criteria reported in Pe&#241;afort-Asturiano, et al.20.</li>
		</ol>
		</li>
		<li>Set up the environment (or laboratory) by considering the conditions required and the restrictions imposed by the goal of the system. Set conditions for impact force attenuation in the simulated falls as compliant flooring systems suggested in Lachance, et al.23 to ensure participants safety.
		<ol>
			<li>Use a mattress or any other compliant flooring system and place it at the center of the environment (or laboratory).</li>
			<li>Keep all objects away from the mattress to give at least one meter of safe space all around. If required, prepare personal protective equipment for participants (e.g., gloves, cap, goggles, knee support, etc.). 
			NOTE: The protocol can be paused here.</li>
		</ol>
		</li>
	</ol>
	</li>
	<li>Determine the human activities and falls that the system will detect after configuration. It is important to have in mind the purpose of the fall detection and human activity recognition system, as well as the target population.
	<ol>
		<li>Define the goal of the fall detection and human activity recognition system. Write it down in a planning sheet. For this case study, the goal is to classify the types of human falls and activities performed in an indoor daily basis of elderly people.</li>
		<li>Define the target population of the experiment in accordance with the goal of the system. Write it down in the planning sheet. In the study, consider elderly people as the target population.</li>
		<li>Determine the type of daily activities. Include some non-fall activities that look like falls in order to improve real fall detection. Assign an ID for all of them and describe them as detailed as possible. Set the time period for each activity to be executed. Write all this information down in the planning sheet.</li>
		<li>Determine the type of human falls. Assign an ID for all of them and describe them as detailed as possible. Set the time period for each fall to be executed . Consider if the falls will be self-generated by the subjects or generated by others (e.g., pushing the subject). Write all this information down in the planning sheet.</li>
		<li>In the planning sheet, write down the sequences of activities and falls that a subject will perform. Specify the period of time, the number of trials per activity/fall, the description to perform the activity/fall, and the activity/fall IDs. 
		NOTE: The protocol can be paused here.</li>
	</ol>
	</li>
	<li>Select the relevant subjects to the study that will execute the sequences of activities and falls. Falls are rare events to catch in real life and usually occur to old persons. Nevertheless, for safety reasons, do not include elderly and impaired people in fall simulation under medical advice. Stunts have been used to avoid injuries22.
	<ol>
		<li>Determine the gender, age range, weight and height of the subjects. Define any impairment conditions required. Also, define the minimum number of subjects required for the experiment.</li>
		<li>Randomly select the set of subjects required, following the conditions stated in the previous step. Use a call for volunteers to recruit them. Fulfill all ethical guidelines applicable from the institution and country, as well as any international regulation when experimenting with humans. 
		NOTE: The protocol can be paused here.</li>
	</ol>
	</li>
	<li>Retrieve and store data from subjects. This information will be useful for further experimental analysis. Complete the following steps under supervision by a clinical expert or a responsible researcher.
	<ol>
		<li>Start collecting data with the data acquisition system configured in Step 1.1.</li>
		<li>Ask each of the subjects to perform the sequences of activities and falls declared in Step 1.2. Clearly save the timestamps of the start and end of each activity/fall. Verify that data from all sources of information is saved on the cloud.</li>
		<li>If the activities were not properly done or there were issues with devices (e.g., lost connection, low battery, intermittent connection), discard the samples and repeat Step 1.4.1 until no device issues are found. Repeat Step 1.4.2 for each trial, per subject, declared in the sequence of Step 1.2. 
		NOTE: The protocol can be paused here.</li>
	</ol>
	</li>
	<li>Pre-process all data acquired. Apply up-sampling and down-sampling for each of the sources of information. See details about pre-processing data for fall detection and human activity recognition in Mart&#237;nez-Villase&#241;or et al.21. 
	NOTE: The protocol can be paused here.</li>
</ol>
2. Data Analysis
<ol>
	<li>Select the mode of data treatment. Select Raw Data if the data stored in the database will be used outright (i.e., using deep learning for automatic feature extraction) and go to Step 2.2. Select Feature Data if feature extraction will be used for further analysis and go to Step 2.3.</li>
	<li>For Raw Data, no extra steps are required so go to Step 2.5.</li>
	<li>For Feature Data, extract features from the raw data.
	<ol>
		<li>Segment raw data in time windows. Determine and fix the time window length (e.g., frames of one-second size). In addition, determine if these time windows will be overlapping or not. A good practice is to choose 50% overlapping.</li>
		<li>Extract features from each segment of data. Determine the set of temporal and frequential features to be extracted from the segments. See Mart&#237;nez-Villase&#241;or et al.21 for common feature extraction.</li>
		<li>Save the feature extraction data set on the cloud, in an independent database.</li>
		<li>If different time windows will be selected, repeat Steps 2.3.1 to 2.3.3, and save each feature data set in independent databases. 
		NOTE: The protocol can be paused here.</li>
	</ol>
	</li>
	<li>Select the most important features extracted and reduce the feature data set. Apply some commonly used feature selection methods (e.g., univariate selection, principal components analysis, recursive feature elimination, feature importance, correlation matrix, etc.).
	<ol>
		<li>Select a feature selection method. Here, we used feature importance.</li>
		<li>Use each feature to train a given model (we employed RF) and measure the accuracy (see Equation 1).</li>
		<li>Rank the features by sorting in order of the accuracy.</li>
		<li>Select the most important features. Here, we used the best ranked first ten features. 
		NOTE: The protocol can be paused here.</li>
	</ol>
	</li>
	<li>Select a machine learning classification method and train a model. There are well-known machine learning methods16,17,18,21, such as: support vector machines (SVM), random forest (RF), multilayer perceptron (MLP) and k-nearest neighbors (KNN), among many others.
	<ol>
		<li>Optionally, if a deep learning approach is selected, then consider21: convolutional neural networks (CNN), long short-term memory neural networks (LSTM), among others.</li>
		<li>Select a set of machine learning methods. Here, we used the following methods: SVM, RF, MLP and KNN.</li>
		<li>Fix the parameters of each of the machine learning methods, as suggested in literature21.</li>
		<li>Create a combined feature data set (or raw data set) using the independent feature data sets (or raw data sets), to combine types of sources of information. For example, if a combination of one wearable sensor and one camera is required, then combine the feature data sets from each of these sources.</li>
		<li>Split the feature data set (or raw data set) in training and testing sets. A good choice is to randomly divide 70% for training and 30% for testing.</li>
		<li>Run a k-fold cross-validation21 using the feature data set (or raw data set), for each machine learning method. Use a common metric of evaluation, like accuracy (see Equation 1) to select the best model trained per method. Leave-one subject-out (LOSO) experiments3 are also recommend.
		<ol>
			<li>Open the training feature data set (or raw data set) in the preferred programming language software. Python is recommended. For this step, use pandas library to read a CSV file as follows: 
			training_set = pandas.csv(&#60;filename.csv&#62;).</li>
			<li>Split the feature data set (or raw data set) in pairs of inputs-outputs. For example, use Python to declare the x-values (inputs) and the y-values (outputs): 
			training_set_X = training_set.drop(&#8216;tag&#8217;,axis=1), training_set_Y = training_set.tag 
			where tag represents the column of the feature data set that includes the target values.</li>
			<li>Select one machine learning method and set the parameters. For example, use SVM in Python with the library sklearn like the following command: 
			classifier = sklearn.SVC(kernel = &#8216;poly&#8217;) 
			in which the kernel function is selected as polynomial.</li>
			<li>Train the machine learning model. For example, use the above classifier in Python to train the SVM model: 
			classifier.fit(training_set_X,training_set_Y).</li>
			<li>Compute the estimates values of the model using the testing feature data set (or the raw data set). For example, use the estimate function in Python as follows: estimates = classifier.predict(testing_set_X) where testing_set_X represents the x-values of the testing set.</li>
			<li>Repeat Steps 2.5.6.1 to 2.5.6.5, the number of times k specified in the k-fold cross validation (or the number of times required for the LOSO approach).</li>
			<li>Repeat Steps 2.5.6.1 to 2.5.6.6 for each machine learning model selected. 
			NOTE: The protocol can be paused here.</li>
		</ol>
		</li>
		<li>Compare the machine learning methods by testing the selected models with the testing data set. Other metrics of evaluation can be used: accuracy (Equation 1), precision (Equation 2), sensitivity (Equation 3), specificity (Equation 4) or F1-score (Equation 5), where TP are the true positives, TN are the true negatives, FP are the false positives and FN are the false negatives. 
		<img alt="Equation 1" src="/files/ftp_upload/60361/60361equ01.jpg" /> 
		<img alt="Equation 2" src="/files/ftp_upload/60361/60361equ02.jpg" /> 
		<img alt="Equation 3" src="/files/ftp_upload/60361/60361equ03.jpg" /> 
		<img alt="Equation 4" src="/files/ftp_upload/60361/60361equ04.jpg" /> 
		<img alt="Equation 5" src="/files/ftp_upload/60361/60361equ05.jpg" /></li>
		<li>Use other beneficial performance metrics such as the confusion matrix9 to evaluate the classification task of the machine learning models, or a decision-independent precision-recall9 (PR) or receiver operating characteristic9 (ROC) curves. In this methodology, recall and sensitivity are considered equivalent.</li>
		<li>Use qualitative features of the machine learning models to compare among them, such as: ease of machine learning interpretation; real-time performance; limited resources of time, memory and processing computing; and ease of machine learning deployment in edge devices or embedded systems.</li>
		<li>Select the best machine learning model using the information from: The quality metrics (Equations 1&#8211;5), the performance metrics and the qualitative features of machine learning feasibility of Steps 2.5.6, 2.5.7 and 2.5.8. 
		NOTE: The protocol can be paused here.</li>
	</ol>
	</li>
</ol>
3. System simplification
<ol>
	<li>Select the suitable placements of sources of information. Sometimes, it is necessary to determine the best placement of sources of information (e.g., which location of a wearable sensor is better).
	<ol>
		<li>Determine the subset of sources of information that will be analyzed. For example, if there are five wearable sensors in the body and just one has to be selected as the best sensor placed, each of these sensors will be part of the subset.</li>
		<li>For each source of information in this subset, create a separate data set and store it separately. Keep in mind that this data set could be either the previous feature data set or the raw data set. 
		NOTE: The protocol can be paused here.</li>
	</ol>
	</li>
	<li>Select a machine learning classification method and train a model for one source of information placement. Complete Steps from 2.5.1 to 2.5.6 using each of the data sets created in Step 3.1.2. Detect the most suitable source of information placement by ranking. For this case study, we use the following methods: SVM, RF, MLP and KNN. 
	Note: The protocol can be paused here.</li>
	<li>Select the suitable placements in a multimodal approach if a combination of two or more sources of information are required for the system (e.g., combination of one wearable sensor and one camera). In this case study, use waist-wearable sensor and camera 1 (lateral view) as the modalities.
	<ol>
		<li>Select the best source of information of each modality in the system and create a combined feature data set (or raw data set) using the independent data sets of these sources of information.</li>
		<li>Select a machine learning classification method and train a model for these combined sources of information. Complete Steps 2.5.1 to 2.5.6 using the combined feature data set (or raw data set). In this study, use the following methods: SVM, RF, MLP and KNN. 
		NOTE: The protocol can be paused here.</li>
	</ol>
	</li>
</ol>
4. Evaluation
<ol>
	<li>Prepare a new data set with users in more realistic conditions. Use only the sources of information selected in the previous step. Preferable, implement the system in the target group (e.g., elderly people). Collect data in longer periods of time.
	<ol>
		<li>Optionally if the target group is used only, create a selection group protocol including the terms of exclusion (e.g., any physical or psychological impairment) and stop criteria prevention (e.g., detect any physical injury during the trials; suffering nausea, dizziness and/or vomiting; fainting). Consider also ethical concerns and data privacy issues.</li>
	</ol>
	</li>
	<li>Evaluate the performance of the fall detection and human activity recognition system developed so far. Use Equations 1&#8211;5 to determine the accuracy and predictive power of the system, or any other performance metrics.</li>
	<li>Discuss about the findings on the experimental results.</li>
</ol>

<ol>
	<li>United Nations. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=World+Population+Prospects:+The+2017+Revision,+Key+Findings+and+Advance+Tables.">World Population Prospects: The 2017 Revision, Key Findings and Advance Tables.</a> United Nations. Department of Economic and Social Affairs, Population Division. , (2017).</li><li>World Health Organization. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Ageing,+and+Life+Course+Unit.">Ageing, and Life Course Unit.</a> WHO Global Report on Falls Prevention in Older Age. , (2008).</li><li>Igual, R., Medrano, C., Plaza, I. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Challenges,+Issues+and+Trends+in+Fall+Detection+Systems.">Challenges, Issues and Trends in Fall Detection Systems.</a> Biomedical Engineering Online. 12 (1), 66 (2013).</li><li>Noury, N., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Fall+Detection-Principles+and+Methods.">Fall Detection-Principles and Methods.</a> 2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. , 1663-1666 (2007).</li><li>Mubashir, M., Shao, L., Seed, L. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+Survey+on+Fall+Detection:+Principles+and+Approaches.">A Survey on Fall Detection: Principles and Approaches.</a> Neurocomputing. 100, 144-152 (2002).</li><li>Perry, J. T., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Survey+and+Evaluation+of+Real-Time+Fall+Detection+Approaches.">Survey and Evaluation of Real-Time Fall Detection Approaches.</a> Proceedings of the 6th International Symposium High-Capacity Optical Networks and Enabling Technologies. , 158-164 (2009).</li><li>Xu, T., Zhou, Y., Zhu, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=New+Advances+and+Challenges+of+Fall+Detection+Systems:+A+Survey.">New Advances and Challenges of Fall Detection Systems: A Survey.</a> Applied Sciences. 8 (3), 418 (2018).</li><li>Rougier, C., Meunier, J., St-Arnaud, A., Rousseau, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Robust+Video+Surveillance+for+Fall+Detection+Based+on+Human+Shape+Deformation.">Robust Video Surveillance for Fall Detection Based on Human Shape Deformation.</a> IEEE Transactions on Circuit Systems for Video Technologies. 21, 611-622 (2011).</li><li>Bulling, A., Blanke, U., Schiele, B. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+Tutorial+on+Human+Activity+Recognition+Using+Body-Worn+Inertial+Sensors.">A Tutorial on Human Activity Recognition Using Body-Worn Inertial Sensors.</a> ACM Computing Surveys. 46 (3), 33 (2014).</li><li>Kwolek, B., Kepski, M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Human+Fall+Detection+on+Embedded+Platform+Using+Depth+Maps+and+Wireless+Accelerometer.">Human Fall Detection on Embedded Platform Using Depth Maps and Wireless Accelerometer.</a> Computational Methods and Programs in Biomedicine. 117, 489-501 (2014).</li><li>Ofli, F., Chaudhry, R., Kurillo, G., Vidal, R., Bajcsy, R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Berkeley+MHAD:+A+Comprehensive+Multimodal+Human+Action+Database.">Berkeley MHAD: A Comprehensive Multimodal Human Action Database.</a> Proceedings of the 2013 IEEE Workshop on Applications of Computer Vision. , 53-60 (2013).</li><li>Dovgan, E., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Intelligent+Elderly-Care+Prototype+for+Fall+and+Disease+Detection.">Intelligent Elderly-Care Prototype for Fall and Disease Detection.</a> Slovenian Medical Journal. 80, 824-831 (2011).</li><li>Santoyo-Ram&#243;n, J., Casilari, E., Cano-Garc&#237;a, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Analysis+of+a+Smartphone-Based+Architecture+With+Multiple+Mobility+Sensors+for+Fall+Detection+With+Supervised+Learning.">Analysis of a Smartphone-Based Architecture With Multiple Mobility Sensors for Fall Detection With Supervised Learning.</a> Sensors. 18 (4), 1155 (2018).</li><li>&#214;zdemir, A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=An+Analysis+on+Sensor+Locations+of+the+Human+Body+for+Wearable+Fall+Detection+Devices:+Principles+and+Practice.">An Analysis on Sensor Locations of the Human Body for Wearable Fall Detection Devices: Principles and Practice.</a> Sensors. 16 (8), 1161 (2016).</li><li>Ntanasis, P., Pippa, E., &#214;zdemir, A. T., Barshan, B., Megalooikonomou, V. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Investigation+of+Sensor+Placement+for+Accurate+Fall+Detection.">Investigation of Sensor Placement for Accurate Fall Detection.</a> International Conference on Wireless Mobile Communication and Healthcare. , 225-232 (2016).</li><li>Bagala, F., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Evaluation+of+Accelerometer-Based+Fall+Detection+Algorithms+on+Real-World+Falls.">Evaluation of Accelerometer-Based Fall Detection Algorithms on Real-World Falls.</a> PLoS One. 7, 37062 (2012).</li><li>Bourke, A. K., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Assessment+of+Waist-Worn+Tri-Axial+Accelerometer+Based+Fall-detection+Algorithms+Using+Continuous+Unsupervised+Activities.">Assessment of Waist-Worn Tri-Axial Accelerometer Based Fall-detection Algorithms Using Continuous Unsupervised Activities.</a> Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society. , 2782-2785 (2010).</li><li>Kerdegari, H., Samsudin, K., Ramli, A. R., Mokaram, S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Evaluation+of+Fall+Detection+Classification+Approaches.">Evaluation of Fall Detection Classification Approaches.</a> 4th International Conference on Intelligent and Advanced Systems. , 131-136 (2012).</li><li>Alazrai, R., Mowafi, Y., Hamad, E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+Fall+Prediction+Methodology+for+Elderly+Based+on+a+Depth+Camera.">A Fall Prediction Methodology for Elderly Based on a Depth Camera.</a> 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. , 4990-4993 (2015).</li><li>Pe&#241;afort-Asturiano, C. J., Santiago, N., N&#250;&#241;ez-Mart&#237;nez, J. P., Ponce, H., Mart&#237;nez-Villase&#241;or, L. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Challenges+in+Data+Acquisition+Systems:+Lessons+Learned+from+Fall+Detection+to+Nanosensors.">Challenges in Data Acquisition Systems: Lessons Learned from Fall Detection to Nanosensors.</a> 2018 Nanotechnology for Instrumentation and Measurement. , 1-8 (2018).</li><li>Mart&#237;nez-Villase&#241;or, L., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=UP-Fall+Detection+Dataset:+A+Multimodal+Approach.">UP-Fall Detection Dataset: A Multimodal Approach.</a> Sensors. 19 (9), 1988 (2019).</li><li>Rantz, M., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Falls,+Technology,+and+Stunt+Actors:+New+approaches+to+Fall+Detection+and+Fall+Risk+Assessment.">Falls, Technology, and Stunt Actors: New approaches to Fall Detection and Fall Risk Assessment.</a> Journal of Nursing Care Quality. 23 (3), 195-201 (2008).</li><li>Lachance, C., Jurkowski, M., Dymarz, A., Mackey, D. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Compliant+Flooring+to+Prevent+Fall-Related+Injuries:+A+Scoping+Review+Protocol.">Compliant Flooring to Prevent Fall-Related Injuries: A Scoping Review Protocol.</a> BMJ Open. 6 (8), 011757 (2016).</li></ol>

The authors have nothing to disclose.

It is common to encounter challenges due to synchronization, organization and data inconsistency problems20 when a dataset is created.
Synchronization 
In the acquisition of data, synchronization problems arise given that multiple sensors commonly work at different sampling rates. Sensors with higher frequencies collect more data than those with lower frequencies. Thus, data from different sources will not be paired correctly. Even if sensors run at the same sampling rates, it is possible that data will not be aligned. In this regard, the following recommendations might help to handle these synchronization problems20: (i) register timestamp, subject, activity and trial in each data sample obtained from the sensors; (ii) the most consistent and less frequent source of information has to be used as reference signal for synchronization; and (iii) use automatic or semi-automatic procedures to synchronize video recordings that manual inspection would be impractical.
Data pre-processing 
Data pre-processing must also be done, and critical decisions influence this process: (a) determine the methods for data storage and data representation of multiple and heterogeneous sources (b) decide the ways to store data in the local host or on the cloud (c) select the organization of data, including the file names and folders (d) handle missing values of data as well as redundancies found in the sensors, among others. In addition, for the data cloud, local buffering is recommended when possible to mitigate loss of data at the uploading time.
Data inconsistency 
Data inconsistency is common between trials finding variations in data sample sizes. These issues are related to data acquisition in wearable sensors. Brief interruptions of data acquisition and data collision from multiple sensors leads to data inconsistencies. In these cases, inconsistency detection algorithms are important to handle online failure in sensors. It is important to highlight that wireless-based devices should be monitored frequently throughout the experiment. Low battery might impact connectivity and result in loss of data.
Ethical 
Consent to participate and ethical approval are mandatory in every type of experimentation where people are involved.
Regarding the limitations of this methodology, it is important to notice that it is designed for approaches that consider different modalities for data collection. The systems can include wearable, ambient and/or vision sensors. It is suggested to consider the power consumption of devices and the lifetime of batteries in wireless-based sensors, due to the issues such as loss of data collection, diminishing connectivity and power consumption in the whole system. Moreover, this methodology is intended for systems that use machine learning methods. An analysis of the selection of these machine learning models should be done beforehand. Some of these models could be accurate, but highly time and energy consuming. A trade-off between accurate estimation and limited resource availability for computing in machine learning models must be taken into consideration. It is also important to observe that, in data collection of the system, the activities were conducted in the same order; also, trials were performed in the same sequence. For safety reasons, a protective mattress was used for subjects to fall onto. In addition, the falls were self-initiated. This is an important difference between simulated and real falls, which generally occur towards hard materials. In that sense, this dataset recorded falls with an intuitive reaction trying not to fall. Moreover, there are some differences between real falls in elderly or impaired people and the simulation falls; and these must be taken into account when designing a new fall detection system. This study was focused on young people without any impairments, but it is remarkable to say that the selection of subjects should be aligned to the goal of the system and the target population who will use it.
From the related works described above10,11,12,13,14,15,16,17,18, we can observe that there are authors that use multimodal approaches focusing in obtaining robust fall detectors or focus on placement or performance of the classifier. Hence, they only address one or two of the design issues for fall detection. Our methodology allows solving simultaneously three of the main design problems of a fall detection system.
For future work, we suggest designing and implementing a simple multimodal fall detection system based on the findings obtained following this methodology. For real-world adoption, transfer learning, hierarchical classification and deep learning approaches should be used for developing more robust systems. Our implementation did not consider qualitative metrics of the machine learning models, but real-time and limited computing resources have to be taken into account for further development of human fall and activity detection/recognition systems. Lastly, in order to improve our dataset, tripping or almost falling activities and real-time monitoring of volunteers during their daily life can be considered.

Since the world phenomenon of population aging1, fall prevalence has increased and is actually considered a major health problem2. When a fall occurs, people require immediate attention in order to reduce negative consequences. Fall detection systems can reduce the amount of time in which a person receives medical attention sending an alert when a fall occurs.
There are various categorizations of fall detection systems3. Early works4 classify fall detection systems by their method of detection, roughly analytical methods and machine learning methods. More recently, other authors3,5,6 have considered data acquisition sensors as the main feature to classify fall detectors. Igual et al.3 divides fall detection systems into context-aware systems, that include vision and ambient-sensor based approaches, and wearable device systems. Mubashir et al.5 classifies fall detectors into three groups based on the devices used for data acquisition: wearable devices, ambience sensors, and vision-based devices. Perry et al.6 considers methods for measuring acceleration, methods for measuring acceleration combined with other methods, and methods not measuring acceleration. From these surveys, we can determine that sensors and methods are the main elements to classify the general research strategy.
Each of the sensors has weaknesses and strengths discussed in Xu et al.7. Vision-based approaches mainly use normal cameras, depth sensor cameras, and/or motion capture systems. Normal web cameras are low cost and easy to use, but they are sensitive to environmental conditions (light variation, occlusion, etc.), can only be used in a reduced space, and have privacy issues. Depth cameras, such as the Kinect, provide full-body 3D motion7 and are less affected by lighting conditions than normal cameras. However, approaches based on the Kinect are not as robust and reliable. Motion capture systems are more expensive and difficult to use.
Approaches based on accelerometer devices and smart phones/watches with built-in accelerometers are very commonly used for fall detection. The main drawback of these devices is that they have to be worn for long periods. Discomfort, obtrusiveness, body placement and orientation are design issues to be solved in these approaches. Although smartphones and smart watches are less obtrusive devices that sensors, older people often forget or do not always wear these devices. Nevertheless, the advantage of these sensors and devices is that they can be used in many rooms and/or outdoors.
Some systems use sensors placed around the environment to recognize falls/activities, so people do not have to wear the sensors. However, these sensors are also limited to the places where they are deployed8 and are sometimes difficult to install. Recently, multimodal fall detection systems include different combinations of vision, wearable and ambient sensors in order to gain more precision and robustness. They can also overcome some of the single sensor limitations.
The methodology used for fall detection is closely related with human activity recognition chain (ARC) presented by Bulling et al.9, which consists of stages for data acquisition, signal preprocessing and segmentation, feature extraction and selection, training and classification. Design issues must be solved for each of these stages. Different methods are used in each stage.
We present a methodology based on multimodal sensors to configure a simple, comfortable and fast human fall and human activity detection/recognition system. The goal is to build a system for accurate fall detection that can be easily implemented and adopted. The proposed novel methodology is based on ARC, but it adds some important phases to perform a deep analysis of the following issues in order to simplify the system: (a) select which sensors or combination of sensors are to be used in a simple fall detection system; (b) determine the best placement of the information sources ; and (c) select the most suitable machine learning classification method for fall detection and human activity recognition to create a simple system.
There are some related works in literature that address one or two of the above-mentioned design issues, but to our knowledge, there is no work that focuses on a methodology to overcome all of these problems.
Related works use multimodal approaches for fall detection and human activity recognition10,11,12 in order to gain robustness and increase precision. Kwolek et al.10 proposed the design and implementation of a fall detection system based on accelerometric data and depth maps. They designed an interesting methodology in which a three-axis accelerometer is implemented to detect a potential fall as well as the person&#8217;s motion. If the acceleration measure exceeds a threshold, the algorithm extracts a person differencing the depth map from the online updated depth reference map. An analysis of depth and accelerometer combinations was made using a support vector machine classifier.
Ofli et al.11 presented a Multimodal Human Action Database (MHAD) in order to provide a testbed for new human activity recognition systems. The dataset is important since the actions were gathered simultaneously using 1 optical motion capture system, 4 multi-view cameras, 1 Kinect system, 4 microphones, and 6 wireless accelerometers. The authors presented results for each modality: the Kinect, the mocap, the accelerometer, and the audio.
Dovgan et al.12 proposed a prototype for detecting anomalous behavior, including falls, in the elderly. They designed tests for three sensor systems in order to find the most appropriate equipment for fall and unusual-behavior detection. The first experiment consists of data from a smart sensor system with 12 tags attached to the hips, knees, ankles, wrists, elbows and shoulders. They also created a test dataset using one Ubisense sensor system with four tags attached to the waist, chest and both ankles, and one Xsens accelerometer. In a third experiment, four subjects only use the Ubisense system while performing 4 types of falls, 4 health problems as anomalous behavior and different activity of daily living (ADL).
Other works in literature13,14,15 address the problem of finding the best placement of sensors or devices for fall detection comparing the performance of various combinations of sensors with several classifiers. Santoyo et al.13 presented a systematic assessment evaluating the importance of the location of 5 sensors for fall detection. They compared the performance of these sensor combinations using k-nearest neighbors (KNN), support vector machines (SVM), na&#239;ve Bayes (NB) and decision tree (DT) classifiers. They conclude that the location of the sensor on the subject has an important influence on the fall detector performance independent of the classifier used.
A comparison of wearable sensor placements on the body for fall detection was presented by &#214;zdemir14. In order to determine sensor placement, the author analyzed 31 sensor combinations of the following positions: head, waist, chest, right wrist, right ankle and right thigh. Fourteen volunteers performed 20 simulated falls and 16 ADL. He found that the best performance was obtained when a single sensor is positioned on the waist from these exhaustive combination experiments. Another comparison was presented by Ntanasis15 using &#214;zdemir&#8217;s dataset. The authors compared single positions on the head, chest, waist, wrist, ankle and thigh using the following classifiers: J48, KNN, RF, random committee (RC) and SVM.
Benchmarks of the performance of different computational methods for fall detection can also be found in literature16,17,18. Bagala et al.16 presented a systematic comparison to benchmark the performance of thirteen fall detection methods tested on real falls. They only considered algorithms based on accelerometer measurements placed on the waist or trunk. Bourke et al.17 evaluated the performance of five analytical algorithms for fall detection using a dataset of ADLs and falls based on accelerometer readings. Kerdegari18 made also a comparison of the performance of different classification models for a set of recorded acceleration data. The algorithms used for fall detection were zeroR, oneR, NB, DT, multilayer perceptron and SVM.
A methodology for fall detection was proposed by Alazrai et al.18 using motion pose geometric descriptor to construct an accumulated histogram-based representation of human activity. They evaluated the framework using a dataset collected with Kinect sensors.
In summary, we found multimodal fall detection related works10,11,12 that compare the performance of different combinations of modalities. Some authors address the problem of finding the best placement of sensors13,14,15, or combinations of sensors13 with several classifiers13,15,16 with multiple sensors of the same modality and accelerometers. No work was found in literature that address placement, multimodal combinations and classifier benchmark at the same time.

<table>
	<tbody>
		<tr>
			<td>Inertial measurement wearable sensor</td>
			<td>Mbientlab</td>
			<td>MTH-MetaTracker</td>
			<td>Tri-axial accelerometer, tri-axial gyroscope and light intensity wearable sensor.</td>
		</tr>
		<tr>
			<td>Electroencephalograph brain sensor helmet MindWave</td>
			<td>NeuroSky</td>
			<td>80027-007</td>
			<td>Raw brainwave signal with one forehand sensor.</td>
		</tr>
		<tr>
			<td>LifeCam Cinema video camera</td>
			<td>Microsoft</td>
			<td>H5D-00002</td>
			<td>2D RGB camera with USB cable interface.</td>
		</tr>
		<tr>
			<td>Infrared sensor</td>
			<td>Alean</td>
			<td>ABT-60</td>
			<td>Proximity sensor with normally closed relay.</td>
		</tr>
		<tr>
			<td>Bluetooth dongle</td>
			<td>Mbientlab</td>
			<td>BLE</td>
			<td>Dongle for Bluetooth connection between the wearable sensors and a computer.</td>
		</tr>
		<tr>
			<td>Raspberry Pi</td>
			<td>Raspberry</td>
			<td>Version 3 Model B</td>
			<td>Microcontroller for infrared sensor acquisition and computer interface.</td>
		</tr>
		<tr>
			<td>Personal computer</td>
			<td>Dell</td>
			<td></td>
			<td>Intel Xeon E5-2630 v4 @2.20 GHz, RAM 32GB</td>
		</tr>
	</tbody>
</table>

design and analysis for fall detection system simplification

This paper presents a methodology based on multimodal sensors to configure a simple, comfortable and fast fall detection and human activity recognition system that can be easily implemented and adopted. The methodology is based on the configuration of specific types of sensors, machine-learning methods and procedures. The protocol is divided into four phases: (1) database creation (2) data analysis (3) system simplification and (4) evaluation. Using this methodology, we created a multimodal database for fall detection and human activity recognition, namely UP-Fall Detection. It comprises data samples from 17 subjects that perform 5 types of falls and 6 different simple activities, during 3 trials. All information was gathered using 5 wearable sensors (tri-axis accelerometer, gyroscope and light intensity), 1 electroencephalograph helmet, 6 infrared sensors as ambient sensors, and 2 cameras in lateral and front viewpoints. The proposed novel methodology adds some important stages to perform a deep analysis of the following design issues in order to simplify a fall detection system: a) select which sensors or combination of sensors are to be used in a simple fall detection system, b) determine the best placement of the sources of information, and c) select the most suitable machine learning classification method for fall and human activity detection and recognition. Even though some multimodal approaches reported in literature only focus on one or two of the above-mentioned issues, our methodology allows simultaneously solving these three design problems related to a human fall and activity detection and recognition system.

Watch this Scientific Journal Video about Design and Analysis for Fall Detection System Simplification at JoVE.com

Design and Analysis for Fall Detection System Simplification

We present a methodology based on multimodal sensors to configure a simple, comfortable and fast fall detection and human activity recognition system. The goal is to build a system for accurate fall detection that can be easily implemented and adopted.

This paper presents a methodology based on multimodal sensors to configure a simple, comfortable and fast fall detection and human activity recognition ...

Our methodology adds important stages, selection of sensor combination, placement and classification, to simplify the fall detection system with a deep analysis. There are previous works that address some fall detection design issues, but there is no work that focuses on holistic methodology for overcoming all of these problems. This methodology can also be used for human activity recognition in assisted living, sport performance evaluation, physical therapy and rehabilitation applications.When a dataset is created, challenges due to synchronization, organization, and data inconsistency can arise. A tradeoff between an accurate estimation and the model complexity should be taken into consideration. Demonstrating the procedure will be Jose Pablo Nunez Martinez, a research assistant, and Sofia Pacheco Ibanez, an undergraduate engineering student from our laboratory.Begin by setting up the data acquisition system to facilitate subject data collection and storage. Select the types of wearable sensors, ambient sensors, and vision-based devices required as sources of information and assign an ID for each source of information, the number of channels per source, the technical specifications, and the sampling rate of each of them. To connect all of the sources of information to a central computer or a distributed computer system, first verify that the wire-based devices are connected properly to a single client computer and verify that the wireless-based devices are fully charged.To set up each device to retrieve data, set the data acquisition system to allow data storage on the cloud and confirm that the data acquisition system fulfills the appropriate data synchronization and data consistency properties. Be sure to check that all of the sensors are acquiring data consistently and simultaneously and to include labels for identifying subject activity and traits. Collect sample data with the devices and store the data in a preferred system.Query the database and determine if all of the sources of information are collected at the same sample rates. After considering the conditions required in the restrictions imposed by the goal of the system, set up the testing environment placing a mattress or any other compliant flooring systems in the center of the environment to ensure participant safety. Keep any objects at least one meter away from the mattress and prepare any necessary personal protective equipment for the participants.Then set up the appropriate cameras and pair infrared sensors around the mattress as illustrated. Define the goal of the fall detection and human activity recognition system on a planning sheet and define the target population of the experiment in accordance with the goal of the system. Define the type of daily activities including some non-fall activities that look like falls to improve real fall detection.Assign an ID for each activity and describe the activities in as much detail as possible. Then set the time period for each activity to be executed. Define the type of human falls and assign an ID and describe each fall for each activity along with the time period for each fall to be executed.Consider if the falls will be self-generated by the subjects or generated by others and write this information on the planning sheet. To collect the activity and fall data, place the recording devices onto the subject as illustrated. When the subject is ready, under the supervision of a clinical expert or a responsible researcher, start the data collection in the data acquisition system and ask the subject to perform the sequences of activities and falls outlined in the planning sheet saving the timestamps of the start and end of each activity or fall.Verify that the data from all of the sources of information are saved on the cloud after each activity or fall. To analyze the collected activity and fall data, use the feature dataset for each machine learning method to run a K-fold cross-validation. Use a common metric of evaluation like accuracy to select the best model trained per method.Next, open the training feature dataset in the preferred programming language software and use the pandas library to read a CSV file as indicated. Split the feature dataset in pairs of inputs/outputs as indicated. Select one machine learning method and set the parameters.Train the machine learning model and compute the estimates values of the model using the testing feature dataset. Repeat the K-fold cross-validation of the number of times K is specified in the K-fold cross-validation for each machine learning model selected. Select the suitable placements in the multimodal approach if a combination of two or more sources of information are required for the system and select the best source of information for each modality in the system.Create a combined feature dataset using the independent datasets of these sources of information and select the machine learning classification method. Train a model for these combined sources of information and repeat the validation using the combined feature dataset. Then prepare a new dataset with the subjects under more realistic conditions using only the sources of information selected in the previous analysis.Graphical representation of the best performance obtained for each modality depending on the machine learning model and the best window length configuration clearly illustrates that multimodal approaches obtain the best F1 score values compared to some unimodal approaches. Although notably using wearable sensors only, a similar performance to a multimodal approach can be obtained. In terms of the benchmark of the data-driven models, random forest presents the best results in almost all of the experiments while multilayer perceptron and support vector machines are not very consistent in performance.The best performance is obtained when a single sensor is used at the waist, neck, or tight right pocket. Ankle and left wrist wearable sensors performed the worst. In addition, the waist, neck, and tight right pocket sensors with random forests classifier in a three-second window size with 50%overlapping are the most suitable wearable sensors for fall detection.The lateral view camera performs the best fall detection and the best camera location is in a lateral view point using random forest in a three-second window size and 50%overlapping. In addition, the random forest model classifier demonstrates the best performance in accuracy and F1 score in both multi-modalities and the combination between the waist and camera one ranks in the first position. In our experiments, we recruited young people with no impairments, but subjects should be selected that align with the goal of the system and the target population using the system.Simple multimodal fall detection systems can be designed and implemented based on this methodology. For real world adaptation, transfer learning and deep learning approaches are recommended for developing robust systems.

Research

JoVE Journal

Bioengineering

Silk Film Culture System for in vitro Analysis and Biomaterial Design

Silk Film Culture System for in vitro Analysis and Biomaterial Design

Silk films are promising protein-based biomaterials that can be fabricated with high fidelity and economically within a research laboratory environment ...

Bacterial Detection &amp; Identification Using Electrochemical Sensors

Bacterial Detection & Identification Using Electrochemical Sensors

Electrochemical  sensors are widely used for rapid and accurate measurement of blood glucose and  can be adapted for detection of a wide variety of ...

Microfluidic Picoliter Bioreactor for Microbial Single-cell Analysis: Fabrication, System Setup, and Operation

In this protocol the fabrication, experimental setup and basic operation of the recently introduced microfluidic picoliter bioreactor (PLBR) is described ...

Automated Robotic Liquid Handling Assembly of Modular DNA Devices

Recent advances in modular DNA assembly techniques have enabled synthetic biologists to test significantly more of the available &#34;design space&#34; ...

Design and Implementation of an Automated Illuminating, Culturing, and Sampling System for Microbial Optogenetic Applications

Optogenetic systems utilize genetically-encoded proteins that change conformation in response to specific wavelengths of light to alter cellular ...

Method and Instrumented Fixture for Femoral Fracture Testing in a Sideways Fall-on-the-Hip Position

Mechanical testing of femora brings valuable insights into understanding the contribution of clinically-measureable variables such as bone mineral density ...

Anatomically Realistic Neonatal Heart Model for Use in Neonatal Patient Simulators

Neonatal patient simulators (NPS) are artificial patient surrogates used in the context of medical simulation training. Neonatologists and nursing staff ...

Design and Synthesis of a Reconfigurable DNA Accordion Rack

DNA nanostructure-based mechanical systems or DNA nanomachines, which produce complex nanoscale motion in 2D and 3D in the nanometer to &#229;ngstr&#246;m ...

Biaxial Mechanical Characterizations of Atrioventricular Heart Valves

Extensive biaxial mechanical testing of the atrioventricular heart valve leaflets can be utilized to derive&#160;optimal parameters used in constitutive ...

Electric and Magnetic Field Devices for Stimulation of Biological Tissues

Electric fields (EFs) and magnetic fields (MFs) have been widely used by tissue engineering to improve cell dynamics such as proliferation, migration, ...