Assessing the Accuracy of Fitness Smartwatch Data for Cardiovascular and Physical Activity Monitoring: A Validation Study in Digital Health

Summary

Wearable technology has low cost and offers convenient monitoring of physiological data. However, the accuracy and reliability of these devices require cautious assessment to ensure their effectiveness and safety for users. This report describes the validation process of a commercial smartwatch in monitoring physiological data and physical activity.

Abstract

This study aims to validate the accuracy of low-cost fitness smartwatches by comparing their data with gold-standard measurements for cardiovascular and physical activity parameters. The study enrolled 50 subjects, 26 undergoing validation testing for heart rate, blood oxygen saturation (SpO2), and sleep data against polysomnography (PSG). Additionally, 24 subjects participated in the 3-Minute Walk Test (3MWT) and Stairs Climbing (SC), with step counts validated against manual video calculations. Results showed no significant difference between the device's measurements and gold standard values for shallow sleep, deep sleep, REM time, mean heart rate, minimum heart rate, and SpO2. However, the device significantly underestimated manually counted steps (p = 0.009 (3MWT); p = 0.012 (SC)), total sleep duration (p = 0.004), and wake time (p = 8.94 × 10^-8) while overestimating maximum heart rate (p = 0.011). These findings highlight the importance of accurate validation and interpretation of wearable device data in clinical contexts. Given these limitations, excluding the device's readings in future analyses is recommended to maintain data reliability and research integrity. This study underscores the need for ongoing validation and improvement of wearable technology to ensure its reliability and effectiveness in healthcare.

Introduction

Wearable technology has grown in popularity, becoming commonplace in various parts of daily life¹. These technologies, equipped with sensors and algorithms, have transformed how physiological parameters are monitored and interpreted, providing users with health information, tracking workouts, and allowing users to have a healthier lifestyle. The integration of artificial intelligence and pattern recognition, combined with increasingly popular features like virtual and augmented reality features, not only enhances the functionality of wearable devices but also enables advanced personalized data analysis and more engaging user experience

Protocol

The study protocol is approved by the Universiti Malaya Medical Centre (UMMC) Ethics Review Board (MREC No: 2021325-9983). The validation study of fitness smartwatch measurements is divided into three parts: (1) validation of sleep measurements against the gold standard using a PSG machine, (2) validation of step measurements by comparing manual calculations from video recordings, and (3) data analysis of the validation tests.

1. Validation of sleep measurements against polysomnography (PSG)

NOTE: The study is conducted in a controlled sleep laboratory to minimize external disruptions. Prior to the....

Discussion

Several limitations of the fitness smartwatch were identified based on the mixed results in the comparison analysis. The high variability in the smartwatch's measurements may stem from its reliance on movement and heart rate data rather than more detailed measures, such as the electroencephalogram (EEG) used in PSG. Deep sleep and REM time showed better agreement, which suggests that the fitness smartwatch may be suitable for general sleep monitoring. However, it may not be reliable for clinical diagnosis, such as detail.......

Materials

Name	Company	Catalog Number	Comments
Sleep Diagnostic System	Natus Neurology & Compumedics		Referred in manuscript as PSG machine
Xiaomi Mi Band 6	Xiaomi		Referred in manuscript as fitness smartwatch