Virtual Reality Tools for Assessing Unilateral Spatial Neglect: A Novel Opportunity for Data Collection

Podsumowanie

The goal was to design, build, and pilot a novel virtual reality task to detect and characterize unilateral spatial neglect, a syndrome affecting 23-46% of acute stroke survivors, expanding the role of virtual reality in the study and management of neurologic disease.

Streszczenie

Unilateral spatial neglect (USN) is a syndrome characterized by inattention to or inaction in one side of space and affects between 23-46% of acute stroke survivors. The diagnosis and characterization of these symptoms in individual patients can be challenging and often requires skilled clinical staff. Virtual reality (VR) presents an opportunity to develop novel assessment tools for patients with USN.

We aimed to design and build a VR tool to detect and characterize subtle USN symptoms, and to test the tool on subjects treated with inhibitory repetitive transcranial magnetic stimulation (TMS) of cortical regions associated with USN.

We created three experimental conditions by applying TMS to two distinct regions of cortex associated with visuospatial processing- the superior temporal gyrus (STG) and the supramarginal gyrus (SMG) - and applied sham TMS as a control. We then placed subjects in a virtual reality environment in which they were asked to identify the flowers with lateral asymmetries of flowers distributed across bushes in both hemispaces, with dynamic difficulty adjustment based on each subject's performance.

We found significant differences in average head yaw between subjects stimulated at the STG and those stimulated at the SMG and marginally significant effects in the average visual axis.

VR technology is becoming more accessible, affordable, and robust, presenting an exciting opportunity to create useful and novel game-like tools. In conjunction with TMS, these tools could be used to study specific, isolated, artificial neurological deficits in healthy subjects, informing the creation of VR-based diagnostic tools for patients with deficits due to acquired brain injury. This study is the first to our knowledge in which artificially generated USN symptoms have been evaluated with a VR task.

Wprowadzenie

Unilateral spatial neglect (USN) is a syndrome characterized by inattention to or inaction in one side of space that affects between 23-46% of acute stroke survivors, most commonly involving injury to the right cerebral hemisphere and resulting in a tendency to ignore the left side of space and/or the survivor's body^1,2. Although the majority of patients with USN experience significant recovery in the short term, subtle USN symptoms often persist³. USN can increase patient risk for falls and impede activities of daily living^2,4 It has also been shown to negatively impact both motor and global functional outcomes^5,6.

Deficits in USN can be conceptualized as existing across multiple dimensions, such as whether a person ignores one side of space with respect to their own body (egocentric) or with respect to an external stimulus (allocentric)^7,8,9, or whether a person is unable to direct their attention (attentional) or actions (intentional) toward one side of space¹⁰. Patients often exhibit a complex constellation of symptoms that can be characterized along more than one of these dimensions. This variability of USN syndromes is thought to result from varying degrees of injury to specific neuroanatomical structures and neuronal networks, which are complex¹¹. Allocentric neglect has been associated with lesions of the angular gyrus (AG) and superior temporal gyrus (STG), while the posterior parietal cortex (PPC) including the supramarginal gyrus (SMG) has been implicated in egocentric processing^12,13,14,15. Attentional neglect is thought to involve lesions in the right IPL¹⁶, while intentional neglect is thought to be secondary to damage of the right frontal lobe¹⁷ or basal ganglia¹⁸.

Clinical assessment of USN currently relies on pen-and-paper neuropsychological instruments. These conventional assessment tools may be less sensitive than more technologically sophisticated tools, resulting in misdiagnosing or under-diagnosing some patients with USN¹⁹. Better characterization of residual deficits could facilitate the delivery of therapy to patients with milder USN and potentially improve their overall recovery, but such characterization would require very sensitive diagnostic tools. USN poses similar challenges in the laboratory setting, where it can be difficult to isolate from the motor and visual impairments that commonly accompany USN among stroke patients.

Virtual reality (VR) presents a unique opportunity to develop new tools for the diagnosis and characterization of USN. VR is a multisensory 3D environment presented in the first person with real time interactions in which individuals are able to perform tasks involving ecologically valid objects²⁰. It is a promising tool for assessing USN; the ability to precisely control what the user sees and hears allows developers to present a wide variety of virtual tasks to the user. In addition, the sophisticated hardware and software packages currently available allow for real-time collection of a wealth of data about the user's actions, including eye, head, and limb movements, far exceeding the metrics offered by traditional diagnostic tests²¹. These data streams are instantaneously available, opening up the possibility for real-time adjustment of diagnostic tasks based on user performance (e.g. targeting the ideal difficulty level for a given task). This feature can facilitate task adaptation to the wide range of severity seen in USN, which is regarded as a priority in the development of new diagnostic tools for USN²². In addition, immersive VR tasks may impose an increased burden on the patients' attentional resources^23,24, resulting in increased errors which can facilitate the detection of neglect symptoms; indeed, some VR tasks have been shown to have increased sensitivity when compared to conventional paper-and-pencil measures of USN^24,25.

In this study, the goal was to create an assessment tool that requires no expertise in neurology to operate and that can reliably detect and characterize even subtle cases of USN. We built a virtual reality-based, game-like task. We then induced a USN-like syndrome in healthy subjects with transcranial magnetic stimulation (TMS), a noninvasive brain stimulation technique that utilizes electromagnetic pulses emitted from a handheld stimulation coil, which pass through the scalp and skull of the subject and induce electric currents in the subject's brain that stimulate neurons^26,27. This technique has been utilized in the study of USN by others^{13,17,28,29,30}, though to our knowledge never in conjunction with a VR-based assessment tool.

Many researchers are already working on diagnostic and therapeutic applications of VR systems. Recent reviews^31,32 explored a number of projects aimed at the assessment of USN with VR-based techniques, and a number of other studies with this aim have been published^{33,34,35,36,37,38,39,40,41}. The majority of these studies do not utilize the full complement of VR technology that is currently available to the consumer market (e.g., a head-mounted display (HMD) and eye-tracking inserts), limiting their datasets to a smaller number of easily-quantifiable metrics. In addition, all of these studies were performed on patients with acquired brain injury leading to USN, requiring screening methods to assure that patients could at least participate in the assessment tasks (e.g., excluding patients with large visual field deficits or cognitive impairment). It is possible that more subtle cognitive, motor, or visual deficits passed under the threshold of these screening methods, possibly confounding the results of these studies. It is also possible that such screening biased the samples of participants in these studies toward a particular subtype of USN.

To avoid the screening biases of prior studies, we recruited healthy subjects and artificially simulated USN symptoms with a standard TMS protocol that is well-described in a recent manuscript¹⁵, with the goal of inducing allocentric USN-like symptoms by targeting the STG and egocentric USN-like symptoms by targeting the SMG. We designed the task to actively adjust its difficulty trial to trial and to differentiate between different subtypes of USN, specifically allocentric vs. egocentric symptoms. We also used standard paper & pencil assessments of USN to formally demonstrate that the deficits we induced with rTMS are USN-like. We believe the method will be useful to other researchers who want to test novel VR tools for the assessment and rehabilitation of USN.

Protokół

This study was approved by the local Institutional Review Board and meets all criteria set forth by Good Clinical Practice Guidelines. All participants provided informed consent before any study procedures began. Study participants were expected to participate in three separate sessions (outlined in Table 1). The elements of the experiment are described in stepwise fashion below. Session order was randomized.

Session A	Pre-rTMA VR Task	Resting Motor Threshhold*	rTMR at STG or SMG	Post-rTMS VR Behavioral Task
		5/10 pulses elicit MEP o finger twitch (*First session only)	110% of RMT for 20 min at 1 Hz (1200 pulses total)
	15 min	60 min	20 min	15 min
Session B	Pre-rTMA VR Task	Resting Motor Threshhold*	rTMR at Vertex	Post-rTMS VR Behavioral Task
		5/10 pulses elicit MEP o finger twitch (*First session only)	110% of RMT for 20 min at 1 Hz (1200 pulses total)
	15 min	60 min	20 min	15 min
Session C	Pre-rTMS paper & Pencil Behavioral Task	Resting Motor Threshhold*	rTMR at STG or SMG	Post-rTMS paper & Pencil Behavioral Task
	Bell's test; Ota's circle cancellation; stay cancellation; line bisection task	5/10 pulses elicit MEP o finger twitch (*First session only)	110% of RMT for 20 min at 1 Hz (1200 pulses total)	Bell's test; Ota's circle cancellation; stay cancellation; line bisection task
	10 min	60 min	20 min	10 min

Table 1. Structure for each study session. Session order was randomized. Estimated time for each item in italics. MEP=motor evoked potential; rTMS=Repetitive Transcranial Magnetic Stimulation; P&P=Paper and Pencil Stroke Diagnostic Tests; RMT=Resting Motor Threshold

1. Paper & pencil behavioral tasks

1. Have the subject complete the line bisection task (LBT).
   1. Have the subject sit at a table directly across from the tester. Provide the subject with a writing utensil. Provide the subject with the stimulus sheet (Figure 1), ensuring it is placed directly in front of the subject.
      ​NOTE: Although not performed in this experiment, it would be ideal to present each line to be bisected individually on separate sheets of paper to avoid biasing subject with additional context (See Ricci and Chatterjee, 2001⁴²).
   2. Instruct the subject to bisect (divide into halves) each line printed on the stimulus sheet and get as close to the middle as possible.
   3. Tell the subject to keep their head and shoulders centered as best as possible, to complete the task as quickly and accurately as possible, and to notify the tester when they are finished. Monitor the subject to ensure they are not leaning or tilting their head excessively.
   4. Collect the sheet from the subject when the subjects say they are done.
2. Have the subject complete the Bell's Test.
   1. Provide the subject with the Bell's test stimuli sheet (Figure 2).
   2. Instruct the subject to circle or cross out all of the bells on the stimulus sheet, to do so as quickly and accurately as possible, to keep their head and shoulders as centered as possible, and to notify the tester when they are finished.
   3. Monitor the subject to ensure they are not leaning or tilting their head excessively. When the subject says they are finished, ask the subject if they are sure, and allow them to double check their work.
   4. Collect the sheet from the subject when the subjects say they are done a second time.
3. Have the subject complete the star cancellation task.
   1. Present the subject with the stimulus sheet (Figure 3), ensuring it is directly in front of them.
   2. Instruct the subject to circle or cross out all of the stars on the stimulus sheet, to do so as quickly and accurately as possible, to keep their head and shoulders as centered as possible, and to notify the tester when they are finished.
   3. Monitor the subject to ensure they are not leaning or tilting their head excessively.
   4. Collect the sheet from the subject when the subjects say they are done.
4. Have the subject complete the Ota's circle cancellation task.
   1. Provide the subject with the Ota's circle cancellation stimulus sheet (Figure 4), ensuring it is placed directly in front of the subject.
   2. Instruct the subject to cross out or circle all of the open/incomplete circles, to do so as quickly and accurately as possible, to keep their shoulders as centered as possible, and to notify the tester when they are finished.
   3. Monitor the subject to ensure they are not leaning or tilting their head excessively.
   4. Collect the sheet from the subject when the subjects say they are done.
   5. Repeat this task (steps 1.4.1 through 1.4.4) with another copy of the stimulus sheet, but this time the stimulus sheet should be rotated 180 degrees from the orientation it was originally presented.

2. TMS procedures

1. Create a model for neuronavigation prior to the first session.
   1. Obtain the subject's 3T T1 MRI scan in a NIFTI or dicom file type.
   2. Upload that MRI scan into the neuronavigational software to create a 3D representation of the subject's brain.
      1. Select New Empty Project within the software. Drag the subject's MRI scan onto the field labeled "File:".
      2. Go to the Reconstructions tab.
      3. Select New Skin and on the next screen, drag the green boundary lines to encompass the entire image of the brain. Select compute skin. Adjust the Skin/Air Threshold accordingly to get an optimal reconstruction.
      4. Go back to the Reconstructions tab and select New Full Brain Curvilinear and drag the green boundary lines to encompass the entire image of the brain. Set slice spacing to 1 mm and set end depth to 18 mm. Select Compute Curvilinear.
      5. Go to Landmarks tab and select Configure Landmarks. Select New to create a landmark on the reconstruction. Place landmarks on the tip of the nose, bridge of the nose, left tragus, and right tragus.
      6. Go to the Targets tab and select Configure Targets. Select the Curvilinear Brain & Targets view. Using the inspector, peel to a depth of 5-7 mm.
      7. Follow guidelines of Shah-Basak et al. (2018)¹⁴, Neggers et al. (2006)¹¹ and Oliveri and Vallar (2009)³⁹ to locate the superior temporal gyrus or the supramarginal gyrus, and place a marker at those sites.
      8. Place a marker where the two central sulci meet along the median longitudinal fissure for sham stimulation at the vertex.
2. During the first session, find the subject's Resting Motor Threshold (may be completed before or after behavioral task).
   1. Have the subject seated in front of an optical tracking camera and place a tracker on the subject using a headband or glasses.
   2. Attach three disposable electrodes on the subject's right hand and wrist.
      1. Attach one disk electrode to the subject's first dorsal interosseous. Attach a second disk electrode to the subject's second knuckle on their right pointer finger. Attach a ground electrode to the subject's right wrist.
   3. Plug these electrodes into an electrode adaptor, which inputs into an MEP tracking software.
   4. Open the subject's project within the neuronavigational software by selecting New Online Session.
   5. Select the targets to be stimulated in this session (Vertex, SMG, STG).
   6. Go to the Polaris tab and ensure the subject tracker is within view of the camera.
   7. Go to Registration tab.
   8. Using a pointer registered to the neuronavigational software, touch the subjects' face in the same locations that the landmarks were placed in step 2.1.2.5.
      1. Click Sample and go to Next Landmark when the pointer is positioned properly on the subject's head for each landmark.
   9. Go to Validation tab.
   10. Using the pointer, touch the subject in various spots on their head and ensure the crosshairs on the screen line up with the spot being pointed to on the subject.
       1. If they do not line up, redo step 2.2.8 and make sure the pointer is as precisely placed on the landmarks as possible.
   11. Go to Perform tab and ensure the Full Brain Curvilinear View is selected so the experimenter can precisely locate brain regions to target.
   12. Set driver to be the TMS coil that will be used.
   13. Plug handheld TMS coil into TMS machine.
   14. Turn on the TMS Machine and set to single pulse. Set stimulation intensity appropriately; in this experiment, 65% of machine output was used as a starting point.
   15. Place the handheld TMS coil on the left side of the subject's head and stimulate within the motor cortex using single pulses of TMS to identify the location that stimulates the FDI. It may be helpful to have an assistant to watch the subject's finger to identify when the FDI muscle twitches due to stimulation.
   16. Alter the stimulation intensity until stimulation elicits MEP of at least 50 mV exactly 5/10 times, and this will be the resting motor threshold (rMT).
3. Stimulation in between tasks
   1. Repeat steps 2.2.1 through 2.2.13, substituting an air-cooled TMS coil for the handheld coil.
   2. Set stimulation parameters to repetitive TMS at a rate of 1 Hz for 20 minutes (1200 pulses total) with an intensity of 110% of rMT in accordance with parameters set by Shah-Basak et al. (2018)¹⁵.
   3. Place an air-cooled TMS coil with a built-in cooling system on the subject's head targeting the SMG or STG for active sessions or the Vertex for sham sessions (Figure 5).
   4. Proceed with stimulation.

3. VR behavioral task

1. Install supporting software.
   1. Download and install Pupil core software from the Pupil Labs website.
   2. Download and install Unity 3D 2018.3 from the Unity website.
   3. Download and install OpenVR tool through Unity Asset Store or through Steam.
2. Set up the VR hardware (e.g., HTC Vive Pro).
   1. Place base stations on opposite sides of the room, ensuring a clear line of sight, and plug them in.
   2. Press the Channel/Mode button on the back of each sensor to cycle through channels until one of them is set to channel " b" and one is set to " c." Both status LEDs should be white.
   3. Install Pupil Labs Binocular insert into HTC Vive Pro. Connect the Link Box to the computer (Power, USB-A, and HDMI or Mini DisplayPort).
   4. Connect the headset to the Link Box. Adjust top and side straps on headset. Adjust the lens distance.
3. Launch SteamVR.
   1. Launch SteamVR by clicking on the VR icon in the top right corner of Steam.
      1. Turn on controllers with the power button.
      2. On SteamVR, click Settings | Pair New Device to pair each controller by following on-screen instructions.
      3. Click Room Setup from the SteamVR menu and follow on-screen instructions.
4. Launch Pupil Core Software.
5. Place headset on the seated subject's head and give them both controllers. Ensure the straps are tight but comfortable. Ensure both eyes are visible by visually confirming they are centered in the Pupil Core Software's camera feeds.
6. Open the VR task in the Unity Editor and hit the Play button.
7. Run the experiment.
   1. Ask the subject to look straight ahead and click the Tare Camera button on the screen.
   2. Click the Begin Tutorial button and wait for the subject to complete the tutorial. The tutorial consists of audio instruction about the operation of the VR system controller, descriptions and examples of symmetrical (decoy) and asymmetrical (target) flowers, and a 1-minute practice session with a small number of decoy and target flowers. The tutorial lasts 75-100 seconds and tutorial performance data is not collected.
   3. When subject is finished, click the Calibrate Eye Tracking button.
      1. If the calibration is successful, the subject will automatically begin the task. Otherwise, repeat step 3.7.3.
   4. Begin the first trial by clicking the Next Trial button.
      NOTE: During the VR task, subjects are placed in a virtual forest (Figure 6). Three curved box hedges formed a semi circle within reaching distance in front of the subject. Each trial consisted of a varying number of flowers, each with 16 petals, distributed among the hedges at a direct line of sight (Figure 7). Subjects were instructed to "pick" (hold their controller over a flower so that the flower would highlight, then depress the trigger button with their index finger) all asymmetrical "target" flowers and leave alone all symmetrical "decoy" flowers. Each trial would end when the subject successfully picks all of the asymmetrical target flowers, but also would end if the subject ran out of time (2-minute time limit) or if the subject inadvertently picked all of the symmetrical decoy flower. In all of these cases the remaining flowers on the bushes would be cleared, and the experimenter would be prompted to begin the next trial.
   5. Wait until the subject is no longer actively completing a trial and then repeat step 3.7.4 unless at least 12 trials have been completed.
   6. Click the Play button again to end the task.

Wyniki

Data were collected from healthy individuals using the protocol outlined above to demonstrate how the different variables that can be extracted from the virtual reality task can be analyzed to detect subtle differences between groups.

In this study, 7 individuals (2 male) with an average age of 25.6 and an average of 16.8 years of education each underwent three separate sessions of TMS. These subjects were broken into two groups...

Dyskusje

We successfully induced and measured USN symptoms with TMS and VR, respectively. While we did not have significant results when compared to sham trials, we were able to compare multiple metrics of egocentric neglect (average head angle, time spent looking at flowers in either hemispace) and allocentric neglect (performance in selecting flowers with asymmetric petals on the left vs. the right side) between the different experimental groups, and found significant differences in average head angle between subjects stimulate...

Materiały

Name	Company	Catalog Number	Comments
AirFilm Coil (AFC) Rapid Version	Magstim	N/A	Air-cooled TMS coil
Alienware 17 R4 Laptop	Dell	N/A	NVIDIA GeForce GTX 1060 (full specs at https://topics-cdn.dell.com/pdf/alienware-17-laptop_users-guide_en-us.pdf)
BrainSight 2.0 TMS Neuronavigation Software	Rogue Research Inc	N/A	TMS neural targeting software
CED 1902 Isolated pre-amplifier	Cambridge Electronic Design Limted	N/A	EMG pre-amplifier
CED Micro 401 mkII	Cambridge Electronic Design Limted	N/A	Multi-channel waveform data acquisition unit
CED Signal 5	Cambridge Electronic Design Limted	N/A	Sweep-based data acquisition and analysis software. Used to measure TMS evoked motor responses.
HTC Vive Binocular Add-on	Pupil Labs	N/A	HTC Vive, Vive Pro, or Vive Cosmos eye tracking add-on with 2 x 200Hz eye cameras.
Magstim D70 Remote Coil	Magstim	N/A	Hand-held TMS coil
Magstim Super Rapid 2 plus 1	Magstim	N/A	Transcranial Magnetic Stimulation Unit
Unity 2018	Unity	N/A	cross-platform VR game engine
Vive Pro	HTC Vive	N/A	VR hardware system with external motion sensors; 1440x1600 pixels per eye, 90 Hz refresh rate, 110° FoV