Methodology for Establishing a Community-Wide Life Laboratory for Capturing Unobtrusive and Continuous Remote Activity and Health Data

Summary

Unobtrusive sensors and pervasive computing technology incorporated into the daily home life of older adults enables meaningful health and activity changes to be recorded continuously for months to years, providing ecologically valid, high frequency, multi-domain data for research or clinical use.

Abstract

An end-to-end suite of technologies has been established for the unobtrusive and continuous monitoring of health and activity changes occurring in the daily life of older adults over extended periods of time. The technology is aggregated into a system that incorporates the principles of being minimally obtrusive, while generating secure, privacy protected, continuous objective data in real-world (home-based) settings for months to years. The system includes passive infrared presence sensors placed throughout the home, door contact sensors installed on exterior doors, connected physiological monitoring devices (such as scales), medication boxes, and wearable actigraphs. Driving sensors are also installed in participants' cars and computer (PC, tablet or smartphone) use is tracked. Data is annotated via frequent online self-report options that provide vital information with regard to the data that is difficult to infer via sensors such as internal states (e.g., pain, mood, loneliness), as well as data referent to activity pattern interpretation (e.g., visitors, rearranged furniture). Algorithms have been developed using the data obtained to identify functional domains key to health or disease activity monitoring, including mobility (e.g., room transitions, steps, gait speed), physiologic function (e.g., weight, body mass index, pulse), sleep behaviors (e.g., sleep time, trips to the bathroom at night), medication adherence (e.g., missed doses), social engagement (e.g., time spent out of home, time couples spend together), and cognitive function (e.g., time on computer, mouse movements, characteristics of online form completion, driving ability). Change detection of these functions provides a sensitive marker for the application in health surveillance of acute illnesses (e.g., viral epidemic) to the early detection of prodromal dementia syndromes. The system is particularly suitable for monitoring the efficacy of clinical interventions in natural history studies of geriatric syndromes and in clinical trials.

Introduction

Prevailing clinical research is fraught with limitations in the reliability and validity of data captured because of inherent shortcomings of assessment methodology. Interviews are constrained by the times when the clinician and patient can coordinate schedules. Time allotted for examinations is limited by what the volunteer can reasonably be asked to do in a single session. These brief, widely spaced sessions - even if augmented by occasional telephone calls or Internet queries - severely limit the potential to detect meaningful change in function or wellbeing over time. Current test sessions are largely composed of requests for information that can be difficult to recall and verify (e.g., "do you remember to take your medications?") or performance of artificial tasks (e.g., "stand up and sit down as fast as you can"; "remember these ten words"). The assessments are often designed to restrict test-to-test variability when in fact variability in performance itself may be a key diagnostic feature. Further, these brief biopsies of time are conducted under artificial conditions rather than within the normal flow of day-to-day life. Therefore, they are of limited ecological validity. Finally, the current paradigm inherently cannot provide direct linkage of interdependent key events or outcomes (e.g., sleep, socialization, physical activity) because the data are not time-stamped other than as recalled.

An approach to overcoming these shortcomings lies in the development of systems that can be embedded in home or community that take advantage of advances in pervasive computing and sensing technology, wireless communications, and high frequency multi-domain data analytics. The technology and experience in this area are growing and a number of systems have been developed, but have been limited in deployment, features or longitudinal experience^1,2,3,4. In this manuscript, we describe a protocol developed as a means to provide real-time, continuous and longitudinal home-based assessment of health-relevant data to improve upon the limitations of the current health assessment paradigm. Oregon Center for Aging & Technology (ORCATECH) has developed a home-based system based on pervasive computing and sensing technology to provide continuous, real-time assessment of health-relevant activity and behavior. Bringing the assessment into home to enable largely unobtrusive and continuous monitoring of real-world activity substantially overcomes current limitations. First, since the core system is embedded in the life-space of the participants as part of their ambient environment, it is inherently convenient. Assessments requiring discrete responses can be collected when a person is most at ease and, in the case of passive collection methods, as frequently as necessary without burdening a participant. Second, being in the person's normal life-space affords the opportunity to collect data that are immediately ecologically relevant, not simply testing contrived measures of function, but everyday cognition. For example, prospective memory failure, a common complaint difficult to naturalistically test in the clinic, can be assessed at home by automatic daily tracking of medication taking behavior, thus tapping both everyday cognition, as well as a key performance metric known to be sensitive to cognitive change. Third, because the data are digital and time-stamped, measurement of multiple interrelated measures aligned in time is facilitated. For example, time on the telephone and time out of home (measures of social engagement or withdrawal), computer use (measuring initiation, psychomotor activity and cognitive function), and other measures that have been shown to change with functional decline (sleep behavior, weight, walking speed) can add to the sensitivity of the sensor net to distinguish subtle changes that may not be otherwise apparent. Importantly, the effect of health and life events on cognition and function (e.g., weekly reports of pain, medication change, low mood) can also be linked to this data stream as they occur. Finally, conventional testing and queries can be presented via computer or related interfaces (e.g., tablet, smartphone), providing unparalleled opportunities to simultaneously compare legacy test performance to novel digitally derived measures from the same tests, such as response or pause times, learning curves and intra-test variability. This new approach thus transforms current assessment to be more convenient, unobtrusive, continuous, multi-domain and naturalistic. Ultimately, the basic platform of in-home sensor based assessment technology and methodology, provides a system that can be tuned and scaled to address a wide range of specific research questions related to health and wellbeing with noted advantages over the current accepted practice of infrequent clinic or telephone-based assessments.

The following protocol outlines the process of deploying this platform for unobtrusive in-home behavioral and health-related data collection. In developing this platform, a key goal has been to provide a basic suite of assessment functions that can provide the data necessary to infer both general domains of health and well-being (physical, cognitive, social, emotive), as well as more specific behaviors (e.g., medication taking, walking, sleep-related activities, physiologic activity). The development of the platform has been guided by several principles including using the most passive unobtrusive sensing approaches, minimizing direct user engagement with technology, being technology 'agnostic' (i.e., employing the best devices or technical solutions rather than requiring a particular approach or product), being durable (for long-term assessment) and scalable, and minimizing hands-on maintenance.

The platform described has evolved over the past twelve years, importantly informed by a range of end-users, from "digitally naive" to early adopters. Periodic surveys and focus groups have been key to informing this development^5,6,7. Hundreds of volunteers have allowed the systems to be continuously deployed in their homes for up to eleven years with iterative modifications being introduced based on the advances in technology, new functional capacities requested by the research community, and the key constant input of individuals living in home where the technology has been deployed. Collectively, these volunteers have formed a "living" laboratory in the community which we call the "Life Laboratory" where their homes and the continuous data collected throughout the day provide a unique level of details about health, activity, and the life course.

A basic platform of sensing technology forms the backbone of the overall system for capturing continuous home-based data. The elements of this platform are described subsequently. The core platform is modified (elements may be added or removed) based on the information obtained during the process of gathering user attitudes, and beliefs and outcome measures of interest for the study using the research platform. Because data communication protocols are standardized, the system has been designed to allow any device that follows these protocols to be incorporated into the network.

The basic platform described here is based on the use case of volunteers in the Life Laboratory (LL) who consent to have the platform deployed within their homes to collect naturalistic activity and behavior data of their normal life activities for many years (longest current continuous deployment = 11 years).

The hub computer and Ethernet/WiFi connection allow data collection from system devices and transmission back to secure servers at ORCATECH without participant interference. The hub computer is configured to the specific participant and home set-up upon system installation using a laptop or tablet and a control panel that connects to a centralized digital participant management system. Additional data collection devices (such as sensors, MedTracker, and scale) can be configured by communicating with the hub computer in the same way.

The ORCATECH Console and Remote Technology Management System is a custom digital technology and data management system called "Console" that allows participant home technology configuration and system set-up, as well as ongoing remote technology management of homes including secure data collection and monitoring. In addition, to facilitate deployment of the system in the community where each home may have a unique layout, a graphing tool based on a tablet interface is used to automatically record where various sensors are located and their valid physical adjacencies to other sensors (Figure 2). This is important for reference during remote monitoring of the system at the home level.

Passive infrared (PIR) Motion Sensors are digitally assigned to a given home during system installation, communicating with the hub computer via a wireless USB Dongle. One sensor is placed per room to sense motion within the room and participant transitions from room to room. A straight "Sensor Line" of four sensors is placed on the ceiling of a hallway or other area where the participant walks regularly at a consistent pace. This sensor line allows unobtrusive gathering of walking speed many times per day. Other metrics can be derived from these activity sensors such as dwell time or number of room transitions. Door contact sensors are placed around the home at all external doors to detect participants' coming and going from the home, and on the refrigerator to determine general frequency of food access.

On-Line Weekly Health and Activity Self-Reports are required to make optimal sense of the data from the passive system of collection devices. These data are critical to the analysis of participant report of events in the home relative to the sensor collected data. The online weekly self-report survey can be completed on any computing device (e.g., laptop, tablet, smartphone) with an Internet connection to query participants about trips out of the home, visitors in the home, health changes, space changes within the home, loneliness, depression, and pain level. Weekly data collection relies on a relatively short window of recollection, which provides much higher resolution of data and likelihood of accuracy than, for instance, annual or semi-annual check-ups. Furthermore, this self-report process also allows investigators to examine passive indicators of potential cognitive impairment, such as variation in the time to complete the survey, variation in number of clicks, increased difficulties reporting accurate dates, or impairment markers in free text responses. As part of the basic platform, we install a seven day electronic pill box that records whether or not the designated day's compartment was opened and the time(s) that it was opened each day. This provides information about medication adherence as well as a potential indication of cognitive decline if consistency of medication-taking decreases.

A wireless digital bioimpedence scale that also collects pulse, body composition metrics, pulse wave velocity, environmental temperature, and ambient carbon dioxide level is installed in the bathroom, providing data on participants' daily weight. This data can then be correlated with other reported events (e.g., health status, medications), as well as other passive indicators of behavior, such as protocol adherence and frequency of use over time.

In cases where our participants drive, we install a driving sensor in their vehicles. This sensor provides information about driving habits such as frequency, timing, duration, and distance of trips, as well as frequency of hard stops or hard accelerations.

A wrist worn wearable device collects physical activity data both in and out of the home. Several brands and models of wearables have been used in Life Laboratory homes.

Depending on the project, an investigator using the ORCATECH platform may choose to supplement the basic sensor set with additional data-collection components. Examples tested in the past include a phone sensor to monitor socialization through landline phone activity, the development and implementation of a digital balance-board for balance testing, a tablet with periodic cognitive tasks for the participant to complete in their own home, and an automated texting system to evaluate the efficacy of medication reminders via phone.

To handle the diverse data generated by the ORCATECH Life Lab, a tailored information and data system is used for collecting, annotating, maintaining, and analyzing copious activity and health data. ORCATECH has developed a custom system for participant management, self-report data collection and processing, and continuous data collection from all system devices and sensors. The system relies on a distributed NoSQL Cassandra server cluster to store the sensor data and a lambda architecture using Kafka and Spark which allows our data processing capabilities to move closer to real-time processing. Using a REST API, data is transferred into standard data analysis platforms and statistical software programs for data analysis.

Access restricted. Please log in or start a trial to view this content.

Protocol

All participants provided written informed consent. Life Laboratory participants are asked to live their lives as they normally would to allow longitudinal observational study of their life activities and patterns for the rest of their lives. They may withdraw at any time if they wish. The study protocol was approved by the Oregon Health & Science University (OHSU) Institutional Review Board (Life Laboratory OHSU IRB #2765).

1. Preparation

1. Prior to deployment, add all equipment to the Console inventory system online. Assign a name to each device or sensor along with its serial number and MAC address, allowing it to be stored within the Console inventory system.
2. Place a QR code tag on each piece of equipment, enabling specific assignments of home location for the sensors and devices when deployed in home.
3. Before deployment, all sensors and devices that are to be installed will be assigned to the home by scanning the QR code on the sensor or device. This will bring up a website that allows sensor or device assignment to that specific home.
4. Install the hub computer with a SD card which contains the hub configuration management tool.
5. Pack all the now inventoried sensors and equipment, hub computer with installed SD card into an installation kit (box) for home deployment.
6. Verify that the participant's home has an Internet service provider.

2. Home Deployment

1. Set up the hub computer by inserting the wireless dongle and the primary ZigBee coordinator dongle into the hub computer. Insert the Ethernet cable into the hub computer. Lastly, plug the hub computer's power cord into a centrally located room's power socket and connect the Ethernet cable to the home's Internet connection.
   Note: The configuration management tool will ensure that it is using the most current version of the software.
2. Connect an Internet enabled device (laptop, tablet, cell phone) to the hub computer'swireless network to access the local control panel website. The control panel website will show the status of the hub computer, as well as any sensors installed in the home (Figure 3).
3. Run the software configuration tool, ensuring the appropriate software is installed. Do this by navigating to the control panel and running Update.
4. Navigate to the control panel to check that the hub computer is communicating with the main servers. Make sure that the services that allow data to be collected from each installed device and sensor are up and running.
5. Add the sensors to the home, beginning with the motion sensors. Start by opening the sensor placement website from the control panel.
   NOTE: If the home requires more than 16 motion sensors, connect a router dongle into the hub computer and add it to the home or personal area network, also called the "PAN." Once the extension dongles (if needed) are added to the PAN, remove them from the hub computer and plug them into outlets spread throughout the home, creating a network around the home that will send the motion sensor data to the hub computer.
6. In the sensor placement website, create a virtual floor plan of the home, including all the rooms and egress doors. Make sure to select the Sensor Line as one of the areas that is added to the floor plan. Add virtual representations of sensors to the floor plan. Lastly, link the virtual representations of home areas to other each other - in a way that reflects the physical layout of the home - and to the virtual representations of the sensors.
7. Add each successive sensor to the PAN - known as the Personal Area Network - by using the sensor placement tool and physically pressing a button near the sensor's battery. Then, start attaching each sensor to the room or area in the home that is represented in the virtual floor plan.
8. Continue attaching the physical sensors to the walls of the home. Place each wall sensor at head height in each room (kitchen, bedrooms, bathrooms, living rooms) ensuring that the sensor only captures the activity in that room and does not pick up the activity from another area (for example, avoid someone walking down a hallway being picked up by the sensor in a room next to the hallway).
   NOTE: The sensor placement tool allows you to identify and create pathways between rooms.
9. Install a row of four restricted field (ceiling) sensors in a straight walkway (hallways or other areas where the participants must walk past each of the four sensors without a change in pace) on the ceiling to capture walking speed.
   1. Space these walking speed sensors 61 cm (2 feet) apart.
   2. Record the exact distance between the restricted field sensors in the sensor placement website.
10. Install door sensors on each egress door, again using the floor plan on the sensor placement website to indicate their physical location.
11. Add the pillbox to the PAN, confirming that the device has been assigned to the home inventory. Then activate the device by opening one of its lids. Because the pillbox communicates with the hub computer, ensure it is close enough to the hub computer for its signal to be detected.
    Note: The pillbox is frequently kept in the kitchen or bathroom based on participant preference.
12. To setup the scale, navigate to the scale page, located within the Devices tab in the control panel.
    1. On the scale, press the side power button for 10 seconds. The scale should display a confirmation message.
    2. Once the scale shows up in the list of devices, click the Setup button on the right side of the control panel to start the setup process.
    3. Enter the participant's height and weight when prompted in the control panel.
    4. If the participant does not have a pacemaker, toggle the Pacemaker button in the control panel notifying the scale that it can collect bioimpedence data.
    5. Place the scale in a location that has a flat, solid surface easily accessible to the participant (typically in a bathroom).
    6. Have the participant weigh themselves, confirming the scale is recording their initial weight which is entered into the control panel.
13. Set up the wrist worn wearable device by opening the wearable control panel setup page and pressing the reset button located on the back side of the device ten times.
    1. After the device shows up in the list of devices in the control panel and click the Setup button on the right side of the control panel to start the setup process.
    2. Once the account is set up, calibrate the time by using the wheel tool on the setup page.
    3. Finalize the setup by synchronizing the wearable with the hub computer. Click the Sync button in the control panel to confirm that the device is connecting properly and the time is set to the same time as the hub computer.
    4. Indicate in Console on which wrist the wearable is intended to be worn by the participant.
       NOTE: Different devices may require different procedures depending on the manufacturer. Additional sensors and devices may also be deployed and integrated into the data stream such as computer use software and driving sensors. Procedures for adding these are given next.
14. Install the commercial computer use monitoring software on the participant's computer and record their e-mail address. These e-mail addresses are used to send and receive weekly online health and activity surveys.
    1. Verify that the participant's computer operating system is compatible with the commercial computer use monitoring software.
    2. Install the software on the participant's computer using the installation program hosted on a USB flash drive.
    3. Verify that the software is operational on the computer by opening the Task Manager and checking that the software is in the list of Applications.
    4. In the Console inventory system, associate the software program with the participant's profile.
       NOTE: See the Table of Materials for the specific computer use software used (other commercially available monitoring software can be substituted).
15. Setup a driving sensor for participants
    1. Verify that the participant's car was made after 1996 and that the car is supported by the driving sensor device software.
    2. Install the driving monitoring device's app on a mobile device and use the app to setup the adapter.
    3. With the car turned off, plug the adapter into the car's on-board diagnostic (ODB) port.
    4. Wait for the app to recognize and connect to the adapter. This should take 2 - 4 min.
    5. Insert the car key into the ignition. (If the car has keyless ignition, press the car's start button). Turn the key to the position where it switches on the electric power without starting the engine.
    6. Wait for the app to finish setting up the adapter.
    7. In the Console inventory system, add the participant's account information from the app in order to allow the adapter's data to be transferred to the ORCATECH servers using the commercial software's application programming interface (API).
       NOTE: See the Table of Materials for the specific driving monitoring device used.

3. System Confirmation

1. Once all devices are in their final places in the home, confirm that the hub computer is working properly by navigating to the control panel. Check that the hub computer can communicate with the main servers to transfer data and the services to collect the data for each device type are running.
2. Review if data is streaming from each device by navigating to the data collection page on the control panel.
3. Walk near the motion sensors installed in each room in the home in order to confirm each sensor is collecting data about recent movements. Check the motion sensors by viewing the live graph of the motion sensor data activated by walking through the home.
4. Check the pillbox by opening and closing each of the compartment doors of the pillbox a few times. Review the data collection page on the control panel to see if this recent activity was measured and collected.
5. Check the scale by weighing yourself or the participant. Confirm this data is properly synched and transmitted by navigating to the Synched column within the scale Devices page in the control panel.
6. Check if the wearable device is properly synched and transmitting data by navigating to the Synched column within the wearable Devices page in the control panel.

Access restricted. Please log in or start a trial to view this content.

Results

The ORCATECH suite of technology makes it possible to collect a uniquely rich data set about the life patterns of people as they go about their usual activities. The sensor system allows unobtrusive and continuous monitoring of the volunteers in their own homes. The system has been used in dozens of studies involving hundreds of volunteers in research examining key domains of health and function such as walking speed and mobility, medication-taking behavior, mood, time in or out of home, ...

Access restricted. Please log in or start a trial to view this content.

Discussion

We have described a basic system or platform enabling home- and community-based remote sensing and reporting of salient health and wellbeing measures on a continuous basis. The system is intended to be used primarily in research at this time.

Where possible, the system uses open source tools and sensors or devices taking advantage of available APIs and software development kits (SDK). The system is designed to be technology "agnostic" such that a wide variety of sensors or devices can ...

Access restricted. Please log in or start a trial to view this content.

Materials

Name	Company	Catalog Number	Comments
Raspberry Pi 3 Model B	Raspberry Pi Foundation	Raspberry Pi 3 Model B
Motion Sensor	NYCE Sensors Inc	NCZ-3041-HA
Door/Window Sensor	NYCE Sensors Inc	NCZ-3011-HA
Curtain Motion Sensor	NYCE Sensors Inc	NCZ-3045-HA
iSort	TimerCap	iSort
Home Stealth USB Phone Recorder	Fiho	Fi3001B
Automatic Pro	Automatic	AUT-350C
Body Cardio Scale	Nokia	WBS04
Activite/Steel Activity Monitor	Nokia	HWA01 STEEL
Alta 2	Fitbit	FB406
Charge 2	Fitbit	FB407
Flex 2	Fitbit	FB403
Zigbee USB Stick	Silicon Labs	ETRX3USB
WorkTime	Nestersoft	WorkTime Corporate