A Cross-Disciplinary and Multi-Modal Experimental Design for Studying Near-Real-Time Authentic Examination Experiences

Summary

An experimental design was developed to investigate the real-time influences of an examination experience to assess the emotional realities students experience in higher education settings and tasks. This design is the result of a cross-disciplinary (e.g., educational psychology, biology, physiology, engineering) and multi-modal (e.g., salivary markers, surveys, electrodermal sensor) approach.

Abstract

Over the past ten years, research into students' emotions in educational environments has increased. Although researchers have called for more studies that rely on objective measures of emotional experience, limitations on utilizing multi-modal data sources exist. Studies of emotion and emotional regulation in classrooms traditionally rely on survey instruments, experience-sampling, artifacts, interviews, or observational procedures. These methods, while valuable, are mainly dependent on participant or observer subjectivity and is limited in its authentic measurement of students' real-time performance to a classroom activity or task. The latter, in particular, poses a stumbling block to many scholars seeking to objectively measure emotions and other related measures in the classroom, in real-time.

The purpose of this work is to present a protocol to experimentally study students' real-time responses to exam experiences during an authentic assessment situation. For this, a team of educational psychologists, engineers, and engineering education researchers designed an experimental protocol that retained the limits required for accurate physiological sensor measurement, best-practices of salivary collection, and an authentic testing environment. In particular, existing studies that rely on physiological sensors are conducted in experimental environments that are disconnected from educational settings (e.g., Trier Stress Test), detached in time (e.g., before or after a task), or introduce analysis error (e.g., use of sensors in environments where students are likely to move). This limits our understanding of students' real-time responses to classroom activities and tasks. Furthermore, recent research has called for more considerations to be covered around issues of recruitment, replicability, validity, setups, data cleaning, preliminary analysis, and particular circumstances (e.g., adding a variable in the experimental design) in academic emotions research that relies on multi-modal approaches.

Introduction

Psychologists have long understood the importance of humans' emotions in elucidating their behaviors¹. Within the study of education, Academic Achievement Emotions (AEE) has become the focus of emotion research². Researchers that use AAE argue that the situational contexts students find themselves in are important to consider when examining students' emotions. Students may experience test-related, class-related, or learning-related emotions that involve multi-component processes, including affective, physiological, motivational, and cognitive components. AEE is expressed in two forms: valence (positive/negative) ....

Protocol

Procedures were approved by the Institutional Review Board (IRB) under a general review at Utah State University for studies on human subjects and use of these constructs. The typical results include two semesters of an engineering statics course, each with a slightly different experimental setup, at a western institution of higher education in the United States. Practice exams, whose content paralleled the actual exams, were developed by the course instructor and were used for our study. Please note that the protoc.......

Discussion

Although physiological measures have been used in many authentic learning contexts, it is critical to design a study environment that is mindful of the limits of the current technology. Our design balances the need for an authentic testing environment and accommodates the technology. Comfortably limiting participant movement, reducing unintended interruptions, and timestamping participants' testing responses are all critical steps within the protocol.

The space and expense of the electrode.......

Materials

Name	Company	Catalog Number	Comments
1.1 cu ft medical freezer	Compact Compliance	# bci2801863	They can use any freezer as long as it can go below -20 degrees Celsius; these can be used to store salivary samples for longer periods of time (~4 months) before running salivary assays.
Camping Cooler	Amazon	(any size/type)	Can be used to store salivary samples during data collection
E4 sensor	Empatica Inc	E4 Wristband Rev2	You can use any EDA sensor or company as long as it records EDA and accelerometry
EDA Explorer	https://eda-explorer.media.mit.edu/	(open-source)	Can be used to identify potential sources of noise that are not necessarily due to movement
Laptops	Dell	Latitude 3480	They can use any desktop or laptop
Ledalab	http://www.ledalab.de/	(open-source)	Can be used to separate tonic and phasic EDA signals after following filtration steps
MATLAB	https://www.mathworks.com/products/matlab.html	(version varies according to updates)	To be used for Ledalab, EDA Explorer, and to create customized time-stamping programs.
Salivary Alpha Amylase Enzymatic Kit	Salimetrics	‎# 1-1902	For the salivary kits, you should plan to either order the company to analyze your samples and/or go to a molecular biology lab for processing
Salivary Cortisol ELISA Kit	Salimetrics	# ‎1-3002	For the salivary kits, you should plan to either order the company to analyze your samples and/or go to a molecular biology lab for processing
Testing Divider (Privacy Shields)	Amazon	#60005	They can use any brand of testing shield as long as they cover the workspace
Web Camera	Amazon	Logitech c920	They can use any web camera as long as it is HD and 1080p or greater