WheelCon: A Wheel Control-Based Gaming Platform for Studying Human Sensorimotor Control

Podsumowanie

WheelCon is a novel, free and open-source platform to design video games that noninvasively simulates mountain biking down a steep, twisting, bumpy trail. It contains components presenting in human sensorimotor control (delay, quantization, noise, disturbance, and multiple feedback loops) and allows researchers to study the layered architecture in sensorimotor control.

Streszczenie

Feedback control theory has been extensively implemented to theoretically model human sensorimotor control. However, experimental platforms capable of manipulating important components of multiple feedback loops lack development. This paper describes WheelCon, an open-source platform aimed at resolving such insufficiencies. Using only a computer, a standard display, and inexpensive gaming steering wheel equipped with a force feedback motor, WheelCon safely simulates the canonical sensorimotor task of riding a mountain bike down a steep, twisting, bumpy trail. The platform provides flexibility, as will be demonstrated in the demos provided, so that researchers may manipulate the disturbances, delay, and quantization (data rate) in the layered feedback loops, including a high-level advanced plan layer and a low-level delayed reflex layer. In this paper, we illustrate WheelCon's graphical user interface (GUI), the input and output of existing demos, and how to design new games. In addition, we present the basic feedback model and the experimental results from the demo games, which align well with the model's prediction. The WheelCon platform can be downloaded at https://github.com/Doyle-Lab/WheelCon. In short, the platform is featured to be cheap, simple to use, and flexible to program for effective sensorimotor neuroscience research and control engineering education.

Wprowadzenie

The human sensorimotor control system is extremely robust¹, although the sensing is distributed, variable, sparse, quantized, noisy and delayed^2,3,4; the computing in the central nervous system is slow^5,6,7; and the muscle actuation fatigues and saturates⁸. Many computational theoretical models have been proposed to explain the complicated human sensorimotor control process^{4,9,10,11,12,13,14}, which is a tradeoff process in human reach and response^15,16. For example, feedback control theory predicts the optimal control policy¹², Bayesian theory models sensorimotor learning^17,18,19 and information theory sensorimotor foundation^20,21. In contrast to the abundance of theoretical models, experimental platforms capable of manipulating important components of multiple feedback loops lack development. This is in part due to the fact that designing a platform to bridge and test these aspects of sensorimotor control requires a diverse range of expertise, extending from motor control theory, signal processing, and interaction, all the way to computer graphics and programming. Researchers often develop their own custom hardware/software systems to characterize human sensorimotor control performance, which can limit the ability to compare/contrast and integrate datasets across research groups. The development of an easy-to-use and validated system could broaden the quantitative characterization of sensorimotor control.

In this paper, we present the WheelCon platform, a novel, free and open-source platform to design video games for a virtual environment that noninvasively simulates a Fitts’ Law reaching game and a mountain bike task with downing a steep, twisting and bumpy trail. The Fitts’ law for reaching task quantifies the tradeoff between speed and accuracy in which the time required for reaching a target of width at distance scales as^22,23. The 'mountain-bike task' is a combination of a pursuit and compensatory tracking task, which are two classic components of research on human sensorimotor performance, especially in terms of studying feedback loops.

WheelCon contains the highly demanded basic components presented in each theory: delay, quantization, noise, disturbance, and multiple feedback loops. It is a potential tool for studying the following diverse questions in human sensorimotor control:

• How the human sensorimotor system deals with the delay and quantization in neural signaling, which is fundamentally constrained by the limited resources (such as the space and metabolic costs) in the brain^24,25;
• How neural correlation in the human cortex with sensorimotor control²⁶;
• How humans handle the unpredictable, external disturbances in sensorimotor control²⁷;
• How the hierarchical control loops layered and integrated within human sensorimotor system^16,28,29;
• The consequence of the delay and quantization in human visual feedback³⁰ and reflex feedback³¹ in sensorimotor control;
• The optimal policy and strategy for sensorimotor learning under delay and quantization^16,17,24,29.

WheelCon integrates with a steering wheel and can simulate game conditions that manipulate the variables in these questions, such as signaling delay, quantization, noise, and disturbance, while recording the dynamic control policy and system errors. It also allows researchers to study the layered architecture in sensorimotor control. In the example of riding a mountain bike, two control layers are involved in this task: the high-layer plan and the low-layer reflex. For visible disturbances (i.e., the trail), we plan before the disturbance arrives. For disturbances unknown in advance (i.e., small bumps), the control relies on delayed reflexes. Feedback control theory proposes that effective layered architectures can integrate the higher layers' goals, plans, decisions with the lower layers' sensing, reflex, and action²⁴. WheelCon provides experimental tools to induce distinctive disturbances in the plan and reflex layers separately for testing such a layered architecture (Figure 1).

We provide a cheap, easy to use and flexible to program platform, WheelCon that bridges the gap between theoretical and experimental studies on neuroscience. To be specific, it can be used for examining the effects of delay, quantization, disturbance, potentially speed-accuracy tradeoffs. The variables that can be manipulated in control loops are shown in Table 1. It can also be applied for studying decision making and multiplexing ability across different control layers in human sensorimotor control. Moreover, WheelCon is compatible with noninvasive neural recordings, such as electroencephalography (EEG), to measure the neural response during sensorimotor control^32,33,34,35, and the non-invasive brain stimulation techniques, such as Transcranial Electrical Stimulation (tES) and Transcranial Magnetic Stimulation (TMS), to manipulate the neural activity^36,37.

Protokół

The development and application of the protocol were approved by the California Institute of Technology Institutional Review Board (IRB) and the Southern University of Science and Technology IRB. The subject provided informed consent prior to any procedures being performed.

1. System preparation and setup

1. The recommended basic hardware is a 2 GHz dual-core processor and 4 GB of system memory.
2. Build the gaming platform under the Unity platform, while using C# programing language. The Logitech gaming wheel driver and Logitech Steering Wheel SDK are needed for gaming platform development.
3. The gaming platform executable files only support Windows 10 Operating System (OS). Therefore, on a PC running Windows 10, download and install the corresponding racing wheel driver. Then download the compressed WheelCon software (https://github.com/Doyle-Lab/WheelCon/archive/master.zip) and extract the files to the local hard drive.
4. Mount the racing wheel securely at the sitting level in front of a monitor, and then connect the wheel's USB cable to the PC and the power adapter to an outlet.
5. Start the driver GUI to test for correct input readout and force feedback. Importantly, keep the driver GUI running in the background during the test.
6. To start the program, double-click on WheelCon.exe in the '\WheelCon-master\Executable & Output Files\' directory.
7. On the configuration screen, choose settings for monitor and click Play! (Figure 2a). The main menu will appear. Make sure the display size and location are as specified.
   NOTE: The 'Wheel Sensitivity' value, defining cursor speed, ranges from 0 to 1, and defaults to 0.5. In case the range of motion afforded by the racing wheel does not suit specific task parameters, adjust this value. For example, decrease the sensitivity for the aging population. However, for comparing between tasks, it is necessary to keep this value constant for the battery and across groups.

2. Task implementation

1. Fitts' law reaching game
   NOTE: The Fitts’ law reaching game simulates the reaching process. The subject requires to turn the wheel to place the vertical line into the desired region (Figure 2d).
   1. Seat the subject comfortably behind the wheels. Adjust the wheel height if necessary.
   2. On the main menu, click Fitts' Law Task (Figure 2b) and type in a name for the output file indicating subject identification and task information on the textbox.
   3. Click on Select File, choose Fitt's Law.txt in the '\WheelCon-master\ Demo Input Files\' directory, and then click Begin Game.
   4. Instruct the subject to move the green vertical line with the wheel to place it within the gray zone. This task serves to familiarize the subject with maneuvering the wheel, as well as with the color convention used throughout different tasks.
2. Mountain bike tasks
   NOTE: The mountain bike task is a combination of pursuit and compensatory tracking task. It simulates riding a mountain bike down a steep, twisted and bumpy trail. The subject can see the trail and turn the wheel to track it, while a motor can torque the wheel to mimic invisible bumps in the trial (Figure 2e).
   1. Game 1: Testing the effect of the visual delay
      NOTE: In this game, the length of the look-ahead window (advanced warning vs. delay) is manipulated.
      1. On the main menu, click Mountain Bike Task (Figure 2c) and type in a name for the output file indicating subject identification and task information on the textbox.
      2. Click on Select File, choose Vision_Delay.txt in the '\WheelCon-master\ Demo Input Files\' directory, and then click on Begin Game.
      3. Instruct the subject to move the green vertical line with the wheel in order to track the part of the gray trail that intersects the purple horizontal line.
   2. Game 2: Testing the effect of action delay
      NOTE: In this game, a delay of various lengths is added between wheel movement and action output.
      1. On the main menu, click on Mountain Bike Task and type in a name for the output file indicating subject identification and task information on the textbox.
      2. Click on Select File, choose Action_Delay.txt in the '\WheelCon-master\ Demo Input Files\' directory, and then click Begin Game.
      3. Instruct the subject to move the green vertical line with the wheel in order to track the part of the gray trail that intersects the purple horizontal line.
   3. Game 3: Testing the effect of visual quantization
      NOTE: In this game, visual input is quantized to limit the data rate.
      1. On the main menu, click on Mountain Bike Task and type in a name for the output file indicating subject identification and task information on the textbox.
      2. Click on Select File, choose Vision Quantization.txt in the '\WheelCon-master\ Demo Input Files\' directory, and then click Begin Game.
      3. Instruct the subject to move the green vertical line with the wheel in order to track the part of the gray trail that intersects the purple horizontal line.
   4. Game 4: Testing the effect of action quantization
      NOTE: In this game, action output is quantized to limit the data rate.
      1. On the main menu, click on Mountain Bike Task and type in a name for the output file indicating subject identification and task information on the textbox.
      2. Click on Select File, choose Action Quantization.txt in the '\WheelCon-master\ Demo Input Files\' directory, and then click Begin Game.
      3. Instruct the subject to move the green vertical line with the wheel in order to track the part of the gray trail that intersects the purple horizontal line.
   5. Game 5: Testing the effect of bump and trail disturbance
      NOTE: This task consists of three scenarios:
      a) "Bumps", tracking a constant trail subject despite torque disturbances on the wheel that mimic hitting bumps when riding a mountain bike;
      b) "Trail", tracking a moving trail with random turns but without bumps;
      c) "Trail with Bumps", tracking a moving trail with random turns and bumps.
      1. On the main menu, click on Mountain Bike Task and type in a name for the output file indicating subject identification and task information on the textbox.
      2. Click on Select File, choose Bump & Trail.txt in the '\WheelCon-master\ Demo Input Files\' directory, and then click Begin Game.
      3. Instruct the subject to move the green vertical line with the wheel in order to track the part of the gray trail that intersects the purple horizontal line.

3. Data output

1. Locate the TXT output file in the '\WheelCon-master\Executable & Output Files\MountainBikeData\' directory, and then open with Matlab' WheelCon Data Analysis Code.m' in the '\WheelCon-master\Source Code' directory.
2. Specify in the MATLAB script the folder and file_names variables according to the output file directory, and then run the script (Ctrl + Enter), and the output variables will be saved as column vectors to the Workspace. The error and control policy will be exported for each sampling time. See Table 2 for the detailed description.

4. Input file development

1. Open 'WheelCon Mntn Bike Trail Design Code.m' in the '\WheelCon-master\Source Code\' directory.
2. Uncomment (Ctrl + T) the section for the desired game parameters and run the script (Ctrl + Enter). The input file will be saved in the '\WheelCon-master\Source Code\' directory' in .txt format. Each column in the input files is one control variable. Refer to Table 1 for the list of control variables.

Wyniki

Modelling Feedback Control

We show a simplified feedback control model shown in Figure 1. The system dynamics is given by:

where x(t) is the error at time t, r(t) is the trail disturbance w(t), is the bump disturbance, and u(...

Dyskusje

In this paper, we have presented a free, open-source gaming platform, WheelCon, for studying the effects of delay, quantization, disturbance, and layered feedback loops in human sensorimotor control. We have shown the hardware, the software, and the GUI. The settings of a single sensorimotor control loop with delay and quantization have been implemented, which allows us to measure the effects of delay, quantization, and disturbance in sensorimotor control. The experimental results are well in line with the prediction fro...

Materiały

Name	Company	Catalog Number	Comments
Gaming Wheel	Logitech
Windows 10 OS	Microsoft