Usability Evaluation of Augmented Reality: A Neuro-Information-Systems Study

Summary

This study presents an experimental paradigm for a usability test combining subjective and objective evaluations. The objective evaluation adopted Neuro-Information-Systems (NeuroIS) methods, and the subjective evaluation adopted a usability questionnaire and a NASA-Task Load Index (NASA-TLX) scale.

Abstract

This study introduces an experimental paradigm for a usability test of emerging technologies in a management information system (MIS). The usability test included both subjective and objective evaluations. For the subjective evaluation, a usability questionnaire and a NASA-TLX scale were adopted. For the objective evaluation, methods of Neuro-Information-Systems (NeuroIS) were used. From a NeuroIS perspective, this study used mobile fNIRS and eye tracking glasses for multimodal measurements, which solved the problem of ecological validity of cognitive neuroscience tools used in real-world behavior experiments. This study used Augmented Reality (AR) integrated into the Internet of Things (IoT) as an experimental object. Comparing the differences in the neuroimaging data, the physiological data, the usability questionnaire, and the NASA-TLX scale data between the two information-search modes (AR versus a website), information search with AR had a higher efficiency and a lower cognitive load compared with information search with a website during the process of consumption decision-making. The usability experiment results demonstrate that AR, as an emerging technology in retail, can effectively enhance consumer experiences and increase their purchase intention. The experimental paradigm, combining both subjective and objective evaluations in this study, could be applied to a usability test for emerging technologies, such as augmented reality, virtual reality, artificial intelligence, wearable technology, robotics, and big data. It provides a practical experimental solution for the user experience in human-computer-interactions with the adoption of emerging technologies.

Introduction

Six frontier technologies that interact with consumers, typically represented by augmented reality, virtual reality, artificial intelligence, wearable technology, robotics, and big data, are reshaping many theoretical models of consumer behavior¹. Augmented Reality (AR) is a new technology that could enhance consumer experience and improve consumer satisfaction. It superimposes textual information, images, videos, and other virtual items onto real scenarios to fuse virtuality and reality, thus enhancing information in the real world through explanation, guidance, evaluation, and prediction². AR provides a new kind of hum....

Protocol

This study was conducted in accordance with the principles of the Helsinki Declaration. All participants were informed of the purpose and safety of the experiment and signed the informed consent form before participation. This study was approved by the institutional review board of Jiangsu University of Science and Technology.

1. Experiment procedure

1. Preparation for the experiment
   1. Explain informed consent to the participants and ask them to sign the con.......

Discussion

Critical steps within the protocol
During the experiment, several steps were considered to ensure reliability of the results. First, participants who are familiar with the brands of mineral water used in the experiment were excluded, because these participants would have performed the task based on their knowledge of the brand. Second, the participants completed a pre-experiment using other brands of mineral water, which was employed to ensure that the participants were familiar with AR and website.......

Materials

Name	Company	Catalog Number	Comments
AR Engine	Unity Technologies	2020.3.1	AR development platform
AR SDK	PTC	Vuforia Engine 9.8.5	AR development kit
Eye Tracker (eye tracking glasses)	SMI, Germany	SMI ETG	Head-mounted eye tracking system
Eye Tracker Recording software	SMI, Germany	iViewETG Software	Eye Tracker Recording software
fNIRS probes	Artinis Medical Systems BV, Netherlands	Artinis PortaLite	Light source: Light emitting diodes Wavelengths: Standard nominal 760 and 850 nm
fNIRS software	Artinis Medical Systems BV, Netherlands	OxySoft 3.2.70	fNIRS data recording and analysis software
Mineral Water	Groupe Danone	Badoit	Experimental material in the AR condition   Capacity: 330ml Price: 6
Mineral Water	Nestlé	Acqua Panna	Experimental material in the website condition Capacity: 250ml Price: 5.4
Skin Preparation Gel	Weaver and Company	Nuprep	Clean the forehead skin of the participants
Smartphone	Xiaomi	Redmi K30 Ultra	Smartphone-based AR application and website