The authors would like to thank Rabih Dow, Padma Rajagopal, Molly Connors and the staff of the Carroll Center for the Blind (Newton MA, USA) for their support in carrying out this research. This work was supported by the NIH/NEI grant: RO1 EY019924.

1. Participant Demographics
This is an on-going study that recruits blind male and female participants aged between 18-45 years. All participants are legally blind of early onset (documented prior to the age of 3) and of varying ocular etiologies. None of the study participants were previously familiar with the spatial layout of the target physical building.
2. Preparation and Familiarization with AbES
<ol>
	<li> Provide the participant with a blindfold and headphon...

<ol>
	<li>Loomis, J. M., Klatzky, R. L., Golledge, R. G. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Navigating+without+vision:+basic+and+applied+research.">Navigating without vision: basic and applied research.</a> Optom. Vis. Sci. 78, 282-289 (2001).</li><li>Siegel, A. W., White, S. H. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+development+of+spatial+representations+of+large-scale+environments.">The development of spatial representations of large-scale environments.</a> Adv. Child Dev. Behav. 10, 9-55 (1975).</li><li>Strelow, E. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=What+is+needed+for+a+theory+of+mobility:+direct+perception+and+cognitive+maps--lessons+from+the+blind.">What is needed for a theory of mobility: direct perception and cognitive maps--lessons from the blind.</a> Psychol. Rev. 92, 226-248 (1985).</li><li>Giudice, N. A., Bakdash, J. Z., Legge, G. E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Wayfinding+with+words:+spatial+learning+and+navigation+using+dynamically+updated+verbal+descriptions.">Wayfinding with words: spatial learning and navigation using dynamically updated verbal descriptions.</a> Psychol. Res. 71, 347-358 (2007).</li><li>Ashmead, D. H., Hill, E. W., Talor, C. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Obstacle+perception+by+congenitally+blind+children.">Obstacle perception by congenitally blind children.</a> Percept. Psychophys. 46, 425-433 (1989).</li><li>Dede, C. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Immersive+interfaces+for+engagement+and+learning.">Immersive interfaces for engagement and learning.</a> Science. 323, 66-69 (2009).</li><li>Bavelier, D., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Brains+on+video+games.">Brains on video games.</a> Nat. Rev. Neurosci. 12, 763-768 (2011).</li><li>Bavelier, D., Green, C. S., Dye, M. W. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Children,+wired:+for+better+and+for+worse.">Children, wired: for better and for worse.</a> Neuron. 67, 692-701 (2010).</li><li>Lange, B., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Designing+informed+game-based+rehabilitation+tasks+leveraging+advances+in+virtual+reality.">Designing informed game-based rehabilitation tasks leveraging advances in virtual reality.</a> Disabil. Rehabil. , (2012).</li><li>Merabet, L., S&#225;nchez, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Audio-based+Navigation+Using+Virtual+Environments:+Combining+Technology+and+Neuroscience.">Audio-based Navigation Using Virtual Environments: Combining Technology and Neuroscience.</a> AER Journal: Research and Practice in Visual Impairment and Blindness. 2, 128-137 (2009).</li><li>Kalia, A. A., Legge, G. E., Roy, R., Ogale, A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Assessment+of+Indoor+Route-finding+Technology+for+People+with+Visual+Impairment.">Assessment of Indoor Route-finding Technology for People with Visual Impairment.</a> J. Vis. Impair. Blind. 104, 135-147 (2010).</li><li>Lahav, O., Schloerb, D. W., Srinivasan, M. A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Newly+blind+persons+using+virtual+environment+system+in+a+traditional+orientation+and+mobility+rehabilitation+program:+a+case+study.">Newly blind persons using virtual environment system in a traditional orientation and mobility rehabilitation program: a case study.</a> Disabil. Rehabil. Assist Technol. , (2011).</li></ol>

We describe an interactive audio-based virtual environment simulator designed to improve general spatial awareness and navigation skills in the blind. We demonstrate that interacting with AbES provides accurate cues that describe the spatial relationships between objects and the overall layout of the target environment. Blind users can generate accurate spatial cognitive maps based on this auditory information and by interacting with the immersive virtual environment. Furthermore, interacting with AbES within the context...

Finding one's way in an unfamiliar environment presents as a significant challenge for the blind. Navigating successfully requires an understanding of the spatial relationships that exist between one's self and objects in the environment1,2. The mental representation that describes surrounding space is referred to as a spatial cognitive map3. Blind individuals can gather relevant spatial information regarding their surrounding environment through other sensory channels (such as hearing) allowing for the generation of an accurate spatial cognitive map for the purposes of real world navigation tasks4,5. 
Considerable interest has arisen regarding the educative potential of virtual environments and action video games as a means to learn and master skills6-9. Indeed, many strategies and approaches have been developed for the blind for this purpose (see 4,10-12). We have developed Audio-based Environment Simulator (AbES); a user-centered audio-based virtual environment that allows for simulated navigation and exploration of an existing physical building. Drawing from original architectural floor plans, a virtual rendering of a modern two-story building (located at the Carroll Center for the Blind; Newton, MA) was generated with the AbES software (Figures 1A and B). AbES incorporates an action game metaphor with a premise designed to promote full exploration of the building space. Using simple key strokes and spatialized sound cues, users navigate and explore the entire building to collect a maximum number of jewels hidden in various rooms. Users must avoid roving monsters that can take them away and hide them elsewhere in the building (Figure 1C).
We demonstrate that interacting with AbES allows a blind user to generate an accurate spatial cognitive map of a target building based on auditory information acquired within the context of an action game metaphor. This is confirmed by a series of post-training behavioral performance tests designed to assess the transfer of acquired spatial information from a virtual environment to a real-world and large scale indoor navigation task (see Figure 2 for overall study design). Our results show that blind users are able to successfully navigate throughout a building for which they were previously unfamiliar, despite the fact that at no time were they informed of the overall purpose of the study, nor were they instructed to recall the spatial layout of the building while playing the game.

<table border="1">
<tbody>
 <tr>
 <td>Name of Equipment</td>
 <th >Comments</td>
 </tr>
 <tr>
 <td>Laptop computer</td>
 <td >Laptop used exclusively for training participants and collecting data</td>
 </tr>
 <tr>
 <td>Stereo Head phones (fully enclosed circumaural design)</td>
 <td >Worn by all participants during training</td>
 </tr>
 <tr>
 <td>Blindfold</td>
 <td >Worn by all participants during training and testing</td>
 </tr>
 </tbody>
</table>

development of an audio-based virtual gaming environment to assist with navigation skills in the blind

Audio-based Environment Simulator (AbES) is virtual environment software designed to improve real world navigation skills in the blind. Using only audio based cues and set within the context of a video game metaphor, users gather relevant spatial information regarding a building's layout. This allows the user to develop an accurate spatial cognitive map of a large-scale three-dimensional space that can be manipulated for the purposes of a real indoor navigation task. After game play, participants are then assessed on their ability to navigate within the target physical building represented in the game. Preliminary results suggest that early blind users were able to acquire relevant information regarding the spatial layout of a previously unfamiliar building as indexed by their performance on a series of navigation tasks. These tasks included path finding through the virtual and physical building, as well as a series of drop off tasks. We find that the immersive and highly interactive nature of the AbES software appears to greatly engage the blind user to actively explore the virtual environment. Applications of this approach may extend to larger populations of visually impaired individuals.

Results from three early blind participants (aged between 19 and 22 years) are shown (see Table 1 for participant characteristics). In summary, all three participants showed a high level of success on all three navigation tasks following game play with the AbES software. This was confirmed by the performance scores (group mean and individual) on all three behavioral tasks (see Figure 6). The percentage correct performance for the virtual (mean: 90%) followed by the physical (mean: 88.7%)...

Watch this Scientific Journal Video about Development of an Audio-based Virtual Gaming Environment to Assist with Navigation Skills in the Blind at JoVE.com

Development of an Audio-based Virtual Gaming Environment to Assist with Navigation Skills in the Blind

Audio-based Environment Simulator (AbES) is virtual environment software designed to improve real world navigation skills in the blind.

Audio-based Environment Simulator (AbES) is virtual environment software designed to improve real world navigation skills in the blind. Using only audio ...

development-an-audio-based-virtual-gaming-environment-to-assist-with

Research

JoVE Journal

Medicine

14.2K 조회수.  Massachusetts Eye and Ear Infirmary, Harvard Medical School. 오디오 기반의 환경 시뮬레이터 (AbES)는 맹인에서 현실 세계 탐색 능력을 향상하기위한 가상 환경 소프트웨어입니다.