Name: a cross-disciplinary and multi-modal experimental design for studying near-real-time authentic examination experiences
Uploaded: 2019-09-04T14:00:00
Description: Watch this Scientific Journal Video about A Cross-Disciplinary and Multi-Modal Experimental Design for Studying Near-Real-Time Authentic Examination Experiences at JoVE.com

This material is based upon work supported in part by the National Science Foundation (NSF) No. EED-1661100 as well as an NSF GRFP grant given to Darcie Christensen (No. 120214). Any opinions, findings, and conclusions or recommendations expressed in this material do not necessarily reflect those of NSF or USU. We want to thank Sheree Benson for her kind discussions and recommendations for our statistical analysis.
Author contributions in this paper are as follows: Villanueva (research design, data collection and analysis, writing, editing); Husman (research design, data collection, writing, editing); Christensen (data collection and analysis, writing, editing); Youmans (data collection and analysis, writing, and editing); Khan (data collection and analysis, writing, editing); Vicioso (data collection and analysis, editing); Lampkins (data collection and editing); Graham (data collection and editing)

Procedures were approved by the Institutional Review Board (IRB) under a general review at Utah State University for studies on human subjects and use of these constructs. The typical&#160;results include two semesters of an engineering statics course, each with a slightly different experimental setup, at a western institution of higher education in the United States. Practice exams, whose content paralleled the actual exams, were developed by the course instructor and were used for our study. Please note that the protoc...

Although physiological measures have been used in many authentic learning contexts, it is critical to design a study environment that is mindful of the limits of the current technology. Our design balances the need for an authentic testing environment and accommodates the technology. Comfortably limiting participant movement, reducing unintended interruptions, and timestamping participants&#39; testing responses are all critical steps within the protocol.
The space and expense of the electrode...

Psychologists have long understood the importance of humans&#39; emotions in elucidating their behaviors1. Within the study of education, Academic Achievement Emotions (AEE) has become the focus of emotion research2. Researchers that use AAE argue that the situational contexts students find themselves in are important to consider when examining students&#39; emotions. Students may experience test-related, class-related, or learning-related emotions that involve multi-component processes, including affective, physiological, motivational, and cognitive components. AEE is expressed in two forms: valence (positive/negative) and activation (focused/unfocused energy). Positive activating emotions, such as enjoyment, may increase reflective processes like metacognition, whereas positive deactivating emotions such as pride may result in low levels of cognitive processing. Negative activating emotions such as anger and anxiety may spark engagement, whereas negative deactivating emotions such as hopelessness may dampen motivation3,4,5. Academic emotions contribute to how we learn, perceive, decide, respond, and problem-solve2. To regulate academic emotions, an individual must possess self-efficacy (SE)6,7,8, which is their confidence in their ability to employ control over their motivation, behavior, and social environment6. Self-efficacy and academic emotions are interrelated, where lower self-efficacy is tied to negative deactivating emotions (e.g., anxiety, anger, boredom) and higher self-efficacy is tied to positive activating emotions (e.g., happiness, hope, excitement)6,7,8. SE is also believed to be strongly tied to performance6,7,8.
Research that has examined classroom emotions have relied on self-reports, observations, interviews, and artifacts (e.g., exams, projects)9,10. Although these methods provide rich contextual information about students&#39; classroom experiences, they have significant limitations. For example, interviews, observations, and self-reports rely on individuals&#39; introspections10. Other methods have sought to examine academic emotions more proximally than prior researchers, such as those based on experience-sampling approaches where researchers ask students to report on their emotions during the school day11. Although this research allows us to report students&#39; emotions more accurately, this work relies on self-report methods and does not allow for real-time reporting as students have to pause their work on the exam to address the experience survey.
Recently, researchers have begun to address concerns about self-report measures through the use of biological or physiological measures of emotion9, that combined with other instruments or techniques such as surveys, observations, or interviews, consists of a multi-modal form of data collection for educational and psychological research12. For example, biological techniques, including salivary biomarkers, are being used&#160;to understand the role biological processes have on cognition, emotion, learning, and performance13,14,15. For cognitive processes, androgens (e.g., testosterone) have been linked to different spatial recognition patterns in adults and children16,17 whereas hypothalamic-pituitary-adrenocortical hormones (e.g., cortisol) and adrenergic hormones (e.g., salivary &#945;-amylase or sAA) are linked to stress responsiveness amongst individuals18,19,20.
Electrodermal activity (EDA) represents a physiological measure of the activation of the autonomic nervous system (ANS) and is linked to increased activation of the system, cognitive load, or intense&#160;emotional responses21,22,23. In examination activities, EDA is affected by physical mobility21,22, bodily and ambient temperatures24,25,26,27, and verbalization of thoughts28, as well as sensitivity and degree of connectivity of the analog-digital electrodes to the skin29.
Although these can be limitations to using EDA, this technique can still provide valuable insight into what happens during near-real-time examinations and can serve as a promising tool to explore AEE and by extent, self-efficacy. As a result, an accurate picture of students&#39; AEE can be obtained through a combination of survey methods, to determine the valence of emotion, and physiological and biological data, to measure the activation of that emotion. This paper builds upon a previous&#160;publication on examination activities30 and expands the scope of that work to include multi-modal approaches (using experience-sampling surveys, EDA sensors, and salivary biomarkers) in an examination scenario. It is essential&#160;to mention that the protocol described below allows for multiple participant data to be collected at the same time within a single experimental setting.

<table>
	<tbody>
		<tr>
			<td>1.1 cu ft medical freezer</td>
			<td>Compact Compliance</td>
			<td># bci2801863</td>
			<td>They can use any freezer as long as it can go below -20 degrees Celsius; these can be used to store salivary samples for longer periods of time (~4 months) before running salivary assays.</td>
		</tr>
		<tr>
			<td>Camping Cooler</td>
			<td>Amazon</td>
			<td>(any size/type)</td>
			<td>Can be used to store salivary samples during data collection</td>
		</tr>
		<tr>
			<td>E4 sensor</td>
			<td>Empatica Inc</td>
			<td>E4 Wristband Rev2</td>
			<td>You can use any EDA sensor or company as long as it records EDA and accelerometry</td>
		</tr>
		<tr>
			<td>EDA Explorer</td>
			<td>https://eda-explorer.media.mit.edu/</td>
			<td>(open-source)</td>
			<td>Can be used to identify potential sources of noise that are not necessarily due to movement</td>
		</tr>
		<tr>
			<td>Laptops</td>
			<td>Dell</td>
			<td>Latitude 3480</td>
			<td>They can use any desktop or laptop</td>
		</tr>
		<tr>
			<td>Ledalab</td>
			<td>http://www.ledalab.de/</td>
			<td>(open-source)</td>
			<td>Can be used to separate tonic and phasic EDA signals after following filtration steps</td>
		</tr>
		<tr>
			<td>MATLAB</td>
			<td>https://www.mathworks.com/products/matlab.html</td>
			<td>(version varies according to updates)</td>
			<td>To be used for Ledalab, EDA Explorer, and to create customized time-stamping programs.</td>
		</tr>
		<tr>
			<td>Salivary Alpha Amylase Enzymatic Kit</td>
			<td>Salimetrics</td>
			<td>&#8206;# 1-1902</td>
			<td>For the salivary kits, you should plan to either order the company to analyze your samples and/or go to a molecular biology lab for processing</td>
		</tr>
		<tr>
			<td>Salivary Cortisol ELISA Kit</td>
			<td>Salimetrics</td>
			<td># &#8206;1-3002</td>
			<td>For the salivary kits, you should plan to either order the company to analyze your samples and/or go to a molecular biology lab for processing</td>
		</tr>
		<tr>
			<td>Testing Divider (Privacy Shields)</td>
			<td>Amazon</td>
			<td>#60005</td>
			<td>They can use any brand of testing shield as long as they cover the workspace</td>
		</tr>
		<tr>
			<td>Web Camera</td>
			<td>Amazon</td>
			<td>Logitech c920</td>
			<td>They can use any web camera as long as it is HD and 1080p or greater</td>
		</tr>
	</tbody>
</table>

a cross-disciplinary and multi-modal experimental design for studying near-real-time authentic examination experiences

Over the past ten years, research into students&#39; emotions in educational environments has increased. Although researchers have called for more studies that rely on objective measures of emotional experience, limitations on utilizing multi-modal data sources exist. Studies of emotion and emotional regulation in classrooms traditionally rely on survey instruments, experience-sampling, artifacts, interviews, or observational procedures. These methods, while valuable, are mainly&#160;dependent on participant or observer subjectivity and is limited in its authentic measurement of students&#39; real-time performance to a classroom activity or task. The latter, in particular, poses a stumbling block to many scholars seeking to objectively measure emotions and other related measures in the classroom, in real-time.
The purpose of this work is to present a protocol to experimentally study students&#39; real-time responses to exam experiences during an authentic assessment situation. For this, a team of educational psychologists, engineers, and engineering education researchers designed an experimental protocol that retained the limits required for accurate physiological sensor measurement, best-practices of salivary collection, and an authentic testing environment. In particular, existing studies that rely on physiological sensors are conducted in experimental environments that are disconnected from educational settings (e.g., Trier Stress Test), detached in time (e.g., before or after a task), or introduce analysis error (e.g., use of sensors in environments where students are likely to move). This limits our understanding of students&#39; real-time responses to classroom activities and tasks. Furthermore, recent research has called for more considerations to be covered around issues of recruitment, replicability, validity, setups, data cleaning, preliminary analysis, and particular&#160;circumstances (e.g., adding a variable in the experimental design) in academic emotions research that relies on multi-modal approaches.

In this study, we were interested in studying the influences of self-efficacy, performance, and physiological (EDA sensors) and biological (sAA and cortisol) responses of undergraduate engineering students as they took a practice exam. The data shown is a representative subset of samples: (a) one that considered surveys and electrodermal sensors (experiment design A) and (b) one that included the same exam along with the salivary biomarker data (experiment design B). While we collected em...

Watch this Scientific Journal Video about A Cross-Disciplinary and Multi-Modal Experimental Design for Studying Near-Real-Time Authentic Examination Experiences at JoVE.com

A Cross-Disciplinary and Multi-Modal Experimental Design for Studying Near-Real-Time Authentic Examination Experiences

An experimental design was developed to investigate the real-time influences of an examination experience to assess the emotional realities students experience in higher education settings and tasks. This design is the result of a cross-disciplinary (e.g., educational psychology, biology, physiology, engineering) and multi-modal (e.g., salivary markers, surveys, electrodermal sensor) approach.

Over the past ten years, research into students&#39; emotions in educational environments has increased. Although researchers have called for more studies ...

Research

JoVE Journal

Behavior

Simultaneous Scalp Electroencephalography (EEG), Electromyography (EMG), and Whole-body Segmental Inertial Recording for Multi-modal Neural Decoding

Simultaneous Scalp Electroencephalography EEG, Electromyography EMG, and Whole-body Segmental Inertial Recording for Multi-modal Neural Decoding

Recent  studies support the involvement of supraspinal networks in control of bipedal  human walking. Part of this evidence  encompasses studies, ...

Examination of Rapid Dopamine Dynamics with Fast Scan Cyclic Voltammetry During Intra-oral Tastant Administration in Awake Rats

Rapid, phasic dopamine (DA) release in the mammalian brain plays a critical role in reward processing, reinforcement learning, and motivational control. ...

Experimental Assessment of Mouse Sociability Using an Automated Image Processing Approach

Mouse is the preferred model organism for testing drugs designed to increase sociability. We present a method to quantify mouse sociability in which the ...

Asymmetric Walkway: A Novel Behavioral Assay for Studying Asymmetric Locomotion

Behavioral assays are commonly used for the assessment of sensorimotor impairment in the central nervous system (CNS). The most sophisticated methods for ...

The Social Dimension of Stress: Experimental Manipulations of Social Support and Social Identity in the Trier Social Stress Test

In many situations humans are influenced by the behavior of other people and their relationships with them. For example, in stressful situations ...

An Experimental Analysis of Children's Ability to Provide a False Report about a Crime

A considerable amount of research has evaluated children's lie-telling behaviors and skills1-2; however, limitations with the tasks used for eliciting ...

Determining Pain Detection and Tolerance Thresholds Using an Integrated, Multi-Modal Pain Task Battery

Human pain models are useful in the assessing the analgesic effect of drugs, providing information about a drug&#39;s pharmacology and identify ...

A Wireless, Bidirectional Interface for In Vivo Recording and Stimulation of Neural Activity in Freely Behaving Rats

A Wireless, Bidirectional Interface for In Vivo Recording and Stimulation of Neural Activity in Freely Behaving Rats

In vivo electrophysiology is a powerful technique to investigate the relationship between brain activity and behavior at a millisecond and micrometer ...

A Visual Guide for Studying Behavioral Defenses to Pathogen Attacks in Leaf-Cutting Ants

The complex lifestyle, evolutionary history of advanced cooperation, and disease defenses of leaf-cutting ants are well studied. Although numerous studies ...

Multi-Modal Signals for Analyzing Pain Responses to Thermal and Electrical Stimuli

The assessment of pain relies mostly on methods that require a person to communicate. However, for people with cognitive and verbal impairments, existing ...