This study introduces an experimental paradigm for a usability test of emerging technologies in management information systems. The usability test included both subjective and objective evaluations. Mobile fNIRS and eye tracking glasses were used for multimodal measurements solving the ecological validity problem of cognitive neuroscience tools used in real world behavior experiments.
Demonstrating the procedure will be Di Zhang, a master student from my laboratory. Yi Wang, a master student, will act as a participant. Begin by explaining the informed consent to the participants, and ask them to sign the consent form.
Next, perform a color vision test on the participants to confirm that they have normal color discrimination. Then present 30 mineral water brands to participants and instruct them to pick their familiar brands to ensure that they were unfamiliar with the brands of mineral water used in the experiment. Next, introduce the experimental procedure to the participants.
Conduct a pre experiment on the participants using mineral water brands unused in the study to make them familiar with augmented reality or AR and website operations. To wear the fNIRS probes, clean the forehead skin of the participants with skin preparation gel. Wrap the probes in plastic wrap to avoid sweaty circumstances, and fix them on the FP one and FP two position according to the international 10 to 20 system with a black headband.
Then cover the probes with a black bandana. Use eye tracking glasses to capture eye movement in real world environments. Fit prescription lenses on the head unit of the eye tracking glasses magnetically if required and ensure that the participants can walk around freely with a corrected sight.
Once all the participants are familiar with the experiment direct each participant from group A to perform the AR experiment first and then the website experiment. Similarly, ask the participants from group B to perform the website experiment first and then the AR experiment. Ensure the participants sit comfortably on a chair and rest for two minutes to collect baseline data before the experiment.
To measure the fNIRS data, open the recording software and connect the probes to a laptop via Bluetooth adapter. Then record the concentration changes of oxygenated hemoglobin in the participant's prefrontal cortex with a sampling frequency of 10 hertz. Check the received light intensity and tissue saturation index quality to control data quality.
Ensure that the received signal is located between one and 95%To measure the eye tracking, set up the hardware for the eye tracking glasses. Connect the plug of the eye tracking glasses to a USB port on the laptop. Then open the recording software and set the sampling frequency to 120 hertz.
Perform a one point calibration by asking the participant to focus on an identifiable object in their field of view At 0.6 meters. Move the cross hair cursor to the focused object in the scene video and click on the object. Then click the record button on the software interface to start recording.
Present the usability questionnaire and NASA TLX scale to the participants after they complete the tasks of the AR and website. After completing the experiment clean the transmitter and receiver of the probes using a 70%isopropyl alcohol pad. The median difference scores of the usability questionnaire showed that the participants had a better user experience in the AR condition than in the website condition.
The eye tracking data revealed that the participants had higher information search efficiency when using AR than the website. The median difference scores of the NASA TLX scale indicated that the AR technique led to a lower cognitive load than the website technique. The fNIRS data analyzed for the differences in the mean oxygenated, hemoglobin and website conditions showed dynamic cognitive load changes.
Participants familiar with the brands of mineral water used in the experiment were excluded because they would have performed the task based on their knowledge of the brand. This study proposes a experimental paradigm combining subjective and objective evaluations in the mix which can effectively evaluate the usability of emerging technologies.
