Name: development of an audio-based virtual gaming environment to assist with navigation skills in the blind
Uploaded: 2013-03-27T08:55:00
Description: Watch this Scientific Journal Video about Development of an Audio-based Virtual Gaming Environment to Assist with Navigation Skills in the Blind Video at JoVE.com

The authors would like to thank Rabih Dow, Padma Rajagopal, Molly Connors and the staff of the Carroll Center for the Blind (Newton MA, USA) for their support in carrying out this research. This work was supported by the NIH/NEI grant: RO1 EY019924.

1. Participant Demographics
This is an on-going study that recruits blind male and female participants aged between 18-45 years. All participants are legally blind of early onset (documented prior to the age of 3) and of varying ocular etiologies. None of the study participants were previously familiar with the spatial layout of the target physical building.
2. Preparation and Familiarization with AbES
<ol>
	<li> Provide the participant with a blindfold and headphon...

<ol>
	<li>Loomis, J. M., Klatzky, R. L., Golledge, R. G. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Navigating+without+vision:+basic+and+applied+research.">Navigating without vision: basic and applied research.</a> Optom. Vis. Sci. 78, 282-289 (2001).</li><li>Siegel, A. W., White, S. H. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+development+of+spatial+representations+of+large-scale+environments.">The development of spatial representations of large-scale environments.</a> Adv. Child Dev. Behav. 10, 9-55 (1975).</li><li>Strelow, E. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=What+is+needed+for+a+theory+of+mobility:+direct+perception+and+cognitive+maps--lessons+from+the+blind.">What is needed for a theory of mobility: direct perception and cognitive maps--lessons from the blind.</a> Psychol. Rev. 92, 226-248 (1985).</li><li>Giudice, N. A., Bakdash, J. Z., Legge, G. E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Wayfinding+with+words:+spatial+learning+and+navigation+using+dynamically+updated+verbal+descriptions.">Wayfinding with words: spatial learning and navigation using dynamically updated verbal descriptions.</a> Psychol. Res. 71, 347-358 (2007).</li><li>Ashmead, D. H., Hill, E. W., Talor, C. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Obstacle+perception+by+congenitally+blind+children.">Obstacle perception by congenitally blind children.</a> Percept. Psychophys. 46, 425-433 (1989).</li><li>Dede, C. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Immersive+interfaces+for+engagement+and+learning.">Immersive interfaces for engagement and learning.</a> Science. 323, 66-69 (2009).</li><li>Bavelier, D., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Brains+on+video+games.">Brains on video games.</a> Nat. Rev. Neurosci. 12, 763-768 (2011).</li><li>Bavelier, D., Green, C. S., Dye, M. W. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Children,+wired:+for+better+and+for+worse.">Children, wired: for better and for worse.</a> Neuron. 67, 692-701 (2010).</li><li>Lange, B., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Designing+informed+game-based+rehabilitation+tasks+leveraging+advances+in+virtual+reality.">Designing informed game-based rehabilitation tasks leveraging advances in virtual reality.</a> Disabil. Rehabil. , (2012).</li><li>Merabet, L., S&#225;nchez, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Audio-based+Navigation+Using+Virtual+Environments:+Combining+Technology+and+Neuroscience.">Audio-based Navigation Using Virtual Environments: Combining Technology and Neuroscience.</a> AER Journal: Research and Practice in Visual Impairment and Blindness. 2, 128-137 (2009).</li><li>Kalia, A. A., Legge, G. E., Roy, R., Ogale, A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Assessment+of+Indoor+Route-finding+Technology+for+People+with+Visual+Impairment.">Assessment of Indoor Route-finding Technology for People with Visual Impairment.</a> J. Vis. Impair. Blind. 104, 135-147 (2010).</li><li>Lahav, O., Schloerb, D. W., Srinivasan, M. A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Newly+blind+persons+using+virtual+environment+system+in+a+traditional+orientation+and+mobility+rehabilitation+program:+a+case+study.">Newly blind persons using virtual environment system in a traditional orientation and mobility rehabilitation program: a case study.</a> Disabil. Rehabil. Assist Technol. , (2011).</li></ol>

We describe an interactive audio-based virtual environment simulator designed to improve general spatial awareness and navigation skills in the blind. We demonstrate that interacting with AbES provides accurate cues that describe the spatial relationships between objects and the overall layout of the target environment. Blind users can generate accurate spatial cognitive maps based on this auditory information and by interacting with the immersive virtual environment. Furthermore, interacting with AbES within the context...

Finding one's way in an unfamiliar environment presents as a significant challenge for the blind. Navigating successfully requires an understanding of the spatial relationships that exist between one's self and objects in the environment1,2. The mental representation that describes surrounding space is referred to as a spatial cognitive map3. Blind individuals can gather relevant spatial information regarding their surrounding environment through other sensory channels (such as hearing) allowing for the generation of an accurate spatial cognitive map for the purposes of real world navigation tasks4,5. 
Considerable interest has arisen regarding the educative potential of virtual environments and action video games as a means to learn and master skills6-9. Indeed, many strategies and approaches have been developed for the blind for this purpose (see 4,10-12). We have developed Audio-based Environment Simulator (AbES); a user-centered audio-based virtual environment that allows for simulated navigation and exploration of an existing physical building. Drawing from original architectural floor plans, a virtual rendering of a modern two-story building (located at the Carroll Center for the Blind; Newton, MA) was generated with the AbES software (Figures 1A and B). AbES incorporates an action game metaphor with a premise designed to promote full exploration of the building space. Using simple key strokes and spatialized sound cues, users navigate and explore the entire building to collect a maximum number of jewels hidden in various rooms. Users must avoid roving monsters that can take them away and hide them elsewhere in the building (Figure 1C).
We demonstrate that interacting with AbES allows a blind user to generate an accurate spatial cognitive map of a target building based on auditory information acquired within the context of an action game metaphor. This is confirmed by a series of post-training behavioral performance tests designed to assess the transfer of acquired spatial information from a virtual environment to a real-world and large scale indoor navigation task (see Figure 2 for overall study design). Our results show that blind users are able to successfully navigate throughout a building for which they were previously unfamiliar, despite the fact that at no time were they informed of the overall purpose of the study, nor were they instructed to recall the spatial layout of the building while playing the game.

<table border="1">
<tbody>
 <tr>
 <td>Name of Equipment</td>
 <th >Comments</td>
 </tr>
 <tr>
 <td>Laptop computer</td>
 <td >Laptop used exclusively for training participants and collecting data</td>
 </tr>
 <tr>
 <td>Stereo Head phones (fully enclosed circumaural design)</td>
 <td >Worn by all participants during training</td>
 </tr>
 <tr>
 <td>Blindfold</td>
 <td >Worn by all participants during training and testing</td>
 </tr>
 </tbody>
</table>

development of an audio-based virtual gaming environment to assist with navigation skills in the blind

Audio-based Environment Simulator (AbES) is virtual environment software designed to improve real world navigation skills in the blind. Using only audio based cues and set within the context of a video game metaphor, users gather relevant spatial information regarding a building's layout. This allows the user to develop an accurate spatial cognitive map of a large-scale three-dimensional space that can be manipulated for the purposes of a real indoor navigation task. After game play, participants are then assessed on their ability to navigate within the target physical building represented in the game. Preliminary results suggest that early blind users were able to acquire relevant information regarding the spatial layout of a previously unfamiliar building as indexed by their performance on a series of navigation tasks. These tasks included path finding through the virtual and physical building, as well as a series of drop off tasks. We find that the immersive and highly interactive nature of the AbES software appears to greatly engage the blind user to actively explore the virtual environment. Applications of this approach may extend to larger populations of visually impaired individuals.

Results from three early blind participants (aged between 19 and 22 years) are shown (see Table 1 for participant characteristics). In summary, all three participants showed a high level of success on all three navigation tasks following game play with the AbES software. This was confirmed by the performance scores (group mean and individual) on all three behavioral tasks (see Figure 6). The percentage correct performance for the virtual (mean: 90%) followed by the physical (mean: 88.7%)...

Watch this Scientific Journal Video about Development of an Audio-based Virtual Gaming Environment to Assist with Navigation Skills in the Blind Video at JoVE.com

Development of an Audio-based Virtual Gaming Environment to Assist with Navigation Skills in the Blind Video (Video) | JoVE

Audio-based Environment Simulator (AbES) is virtual environment software designed to improve real world navigation skills in the blind.

Development of an Audio-based Virtual Gaming Environment to Assist with Navigation Skills in the Blind

Audio-based Environment Simulator (AbES) is virtual environment software designed to improve real world navigation skills in the blind. Using only audio ...

Research

JoVE Journal

Medicine

The Measurement and Treatment of Suppression in Amblyopia

The Measurement and Treatment of Suppression in Amblyopia Video (Video) | JoVE

Amblyopia, a developmental disorder of the visual cortex, is one of the leading causes of visual dysfunction in the working age population. Current ...

Using Quantitative Real-time PCR to Determine Donor Cell Engraftment in a Competitive Murine Bone Marrow Transplantation Model

Using Quantitative Real-time PCR to Determine Donor Cell Engraftment in a Competitive Murine Bone Marrow Transplantation Model Video (Video) | JoVE

Murine bone marrow transplantation models provide an important tool in measuring hematopoietic stem cell (HSC) functions and determining genes/molecules ...

Development of an Algorithm to Perform a Comprehensive Study of Autonomic Dysreflexia in Animals with High Spinal Cord Injury Using a Telemetry Device

Development of an Algorithm to Perform a Comprehensive Study of Autonomic Dysreflexia in Animals with High Spinal Cord Injury Using a Telemetry Device Video (Video) | JoVE

Spinal cord injury (SCI) is a debilitating neurological condition characterized by somatic and autonomic dysfunctions. In particular, SCI above the ...

An Improved Method for Rapid Intubation of the Trachea in Mice

An Improved Method for Rapid Intubation of the Trachea in Mice Video (Video) | JoVE

Despite some anatomical and physiological differences, mouse models continue to be an essential tool for studying human lung disease. Bleomycin toxicity ...

Determination of the Relative Potency of an Anti-TNF Monoclonal Antibody (mAb) by Neutralizing TNF Using an In Vitro Bioanalytical Method

Determination of the Relative Potency of an Anti-TNF Monoclonal Antibody MAb by Neutralizing TNF Using an In Vitro Bioanalytical Method Video (Video) | JoVE

This protocol shows the measurement of the apoptotic activity neutralization of TNF&#945; in a mouse fibroblast cell model (WEHI 164) using an ...

Re-Arterialized Rat Partial Liver Transplantation with an in vivo Vessel-Oriented 70% Hepatectomy

Re-Arterialized Rat Partial Liver Transplantation with an in vivo Vessel-Oriented 70% Hepatectomy Video (Video) | JoVE

Split liver transplantation and living liver donor liver transplantation were developed in the clinic to utilize liver organs in a more efficient manner. ...

Assessment of the Efficacy of An Osteopathic Treatment in Infants with Biomechanical Impairments to Suckling

Assessment of the Efficacy of An Osteopathic Treatment in Infants with Biomechanical Impairments to Suckling Video (Video) | JoVE

Breastfeeding can be challenging for mother-infant dyads experiencing biomechanical suckling difficulties. Although lactation consultants (LCs) all over ...

A Computerized Functional Skills Assessment and Training Program Targeting Technology Based Everyday Functional Skills

A Computerized Functional Skills Assessment and Training Program Targeting Technology Based Everyday Functional Skills Video (Video) | JoVE

Today, many functional skills are technology-based, so development of a technology-based training program has broad importance. Here we present a ...

Blind Endotracheal Intubation in Neonatal Rabbits

Blind Endotracheal Intubation in Neonatal Rabbits Video (Video) | JoVE

The newborn rabbit is a useful animal model for various pathologies and procedures. Airway management of the rabbit is complex due to its anatomical ...

Real-Time Dynamic Navigation System for the Precise Quad-Zygomatic Implant Placement in a Patient with a Severely Atrophic Maxilla

Real-Time Dynamic Navigation System for the Precise Quad-Zygomatic Implant Placement in a Patient with a Severely Atrophic Maxilla Video (Video) | JoVE

Zygomatic implants (ZIs) are an ideal way to address cases of a severely atrophic edentulous maxilla and maxilla defects because they replace extensive ...