hardware, software, and class setup to detect attention levels in students

교실 참여 향상: 학생 주의 수준에 대한 AI 기반 분석

In modern educational settings, accurately assessing and maintaining students&#39; attention is crucial for effective teaching and learning. However, traditional methods of gauging engagement, such as self-reporting or subjective teacher observations, are time-consuming and prone to biases. To address this challenge, Artificial Intelligence (AI) techniques have emerged as promising solutions for automated attention detection. One significant aspect of understanding students&#39; engagement levels is emotion recognition1. AI systems can analyze facial expressions to identify emotions, such as neutral, disgust, surprise, sadness, fear, happiness, and anger2.
Gaze direction and body posture are also crucial indicators of students&#39; attention3. By utilizing cameras and advanced machine learning algorithms, AI systems can accurately track where students are looking and analyze their body posture to detect signs of disinterest or fatigue4. Furthermore, incorporating biometric data enhances the accuracy and reliability of attention detection5. By collecting measurements, such as heart rate and blood oxygen saturation levels, through smartwatches worn by students, objective indicators of attention can be obtained, complementing other sources of information.
This paper proposes a system that evaluates an individual&#39;s level of attention using color cameras and other different sensors. It combines emotion recognition, gaze direction analysis, body posture assessment, and biometric data to provide educators with a comprehensive set of tools for optimizing the teaching-learning process and improving student engagement. By employing these tools, educators can gain a comprehensive understanding of the teaching-learning process and enhance student engagement, thereby optimizing the overall educational experience. By applying AI techniques, it is even possible to automatically evaluate this data.
The main goal of this work is to describe the system that allows us to capture all the information and, once captured, to train an AI model that allows us to obtain the attention of the whole class in real-time. Although other works have already proposed capturing attention using visual or emotional information6, this work proposes the combined use of these techniques, which provides a holistic approach to allow the use of more complex and effective AI techniques. Moreover, the datasets hitherto available are limited to either a set of videos or one of biometric data. The literature includes no datasets that provide complete data with images of the student&#39;s face or their body, biometric data, data on the teacher&#39;s position, etc. With the system presented here, it is possible to capture this type of dataset.
The system associates a level of attention with each student at each point of time. This value is a probability value of attention between 0% and 100%, which can be interpreted as low level of attention (0%-40%), medium level of attention (40%-75%), and high level of attention (75%-100%). Throughout the text, this probability of attention is referred to as the level of attention, student attention, or whether students are distracted or not, but these are all related to the same output value of our system.
Over the years, the field of automatic engagement detection has grown significantly due to its potential to revolutionize education. Researchers have proposed various approaches for this area of study.
Ma et al.7 introduced a novel method based on a Neural Turing Machine for automatic engagement recognition. They extracted certain features, such as eye gaze, facial action units, head pose, and body pose, to create a comprehensive representation of engagement recognition.
EyeTab8, another innovative system, used models to estimate where someone is looking with both their eyes. It was specially made to work smoothly on a standard tablet with no modifications. This system harnesses well-known algorithms for processing images and analyzing computer vision. Their gaze estimation pipeline includes a Haar-like feature-based eye detector, as well as a RANSAC-based limbus ellipse fitting approach.
Sanghvi et al.9 propose an approach that relies on vision-based techniques to automatically extract expressive postural features from videos recorded from a lateral view, capturing the behavior of the children. An initial evaluation is conducted, involving the training of multiple recognition models using contextualized affective postural expressions. The results obtained demonstrate that patterns of postural behavior can effectively predict the engagement of the children with the robot.
In other works, such as&#160;Gupta et al.10, a deep learning-based method is employed to detect the real-time engagement of online learners by analyzing their facial expressions and classifying their emotions. The approach utilizes facial emotion recognition to calculate an engagement index (EI) that predicts two engagement states: engaged and disengaged. Various deep learning models, including Inception-V3, VGG19, and ResNet-50, are evaluated and compared to identify the most effective predictive classification model for real-time engagement detection.
In&#160;Altuwairqi et al.11, the researchers present a novel automatic multimodal approach for assessing student engagement levels in real-time. To ensure accurate and dependable measurements, the team integrated and analyzed three distinct modalities that capture students&#39; behaviors: facial expressions for emotions, keyboard keystrokes, and mouse movements.
Guill&#233;n et al.12 propose the development of a monitoring system that uses electrocardiography (ECG) as a primary physiological signal to analyze and predict the presence or absence of cognitive attention in individuals while performing a task.
Alban et al.13 utilize a neural network (NN) to detect emotions by analyzing the heart rate (HR) and electrodermal activity (EDA) values of various participants in both time and frequency domains. They find that an increase in the root-mean-square of successive differences (RMSDD) and the standard deviation normal-to-normal (SDNN) intervals, coupled with a decrease in the average HR, indicate heightened activity in the sympathetic nervous system, which is associated with fear.
Kajiwara et al.14 propose an innovative system that employs wearable sensors and deep neural networks to forecast the level of emotion and engagement in workers. The system follows a three-step process. Initially, wearable sensors capture and collect data on behaviors and pulse waves. Subsequently, time series features are computed based on the behavioral and physiological data acquired. Finally, deep neural networks are used to input the time series features and make predictions on the individual&#39;s emotions and engagement levels.
In other research, such as&#160;Costante et al.15, an approach based on a novel transfer metric learning algorithm is proposed, which utilizes prior knowledge of a predefined set of gestures to enhance the recognition of user-defined gestures. This improvement is achieved with minimal reliance on additional training samples. Similarly, a sensor-based human activity recognition framework16 is presented to address the goal of the impersonal recognition of complex human activities. Signal data collected from wrist-worn sensors is utilized in the human activity recognition framework developed, employing four RNN-based DL models (Long-Short Term Memories, Bidirectional Long-Short Term Memories, Gated Recurrent Units, and Bidirectional Gated Recurrent Units) to investigate the activities performed by the user of the wearable device.

저자 스포트라이트: 교실 주의력 측정의 기술적 및 주관적 문제 해결

학생의 주의력 수준을 감지하기 위한 인공 지능 기반 시스템

Our research focuses on exploring the feasibility of using artificial intelligence to measure and detect the student's level of attention in a classroom setting. We are investigating whether custom-developed algorithms and methodologies can effectively assess student engagement during classes. The most recent approaches are based on artificial intelligence.Specifically, the learning algorithms are state of art. This kind of algorithm uses vast amounts of data to train their inner models, and then they can be used to classify images, estimate the pose of a person, or perform a range of other interesting tasks. We have two main critical challenges in our study.First, there are technical matters that must be solved. For instance, how to set up the devices, how to estimate the motion from the picture, the pose, and the rest of other parameters we consider in our approach. And then there is a subjective branch as it is unclear whether all individuals express the level of attention the same way.The strongest point of our system is that we use a multimodal approach. We analyze different inputs, facial emotion, body pose, head pose, accelerator data, and heart rate to automatically classify the level of attention. In addition, our methods heavily relies on artificial intelligence, which has reportedly been used with high success in other related research areas.

하드웨어, 소프트웨어 및 교실 설정을 통해 학생들의 주의 수준을 감지합니다.

시작하려면 원하는 위치에 Wi-Fi 기능이 있는 라우터를 설정하여 방 전체를 커버하도록 하고, 각 학생의 위치에 대해 하나의 스마트워치와 하나의 카메라를 설정합니다. 학생 2명당 1개의 내장 장치를 설정합니다. 두 개의 삼각대에 두 대의 카메라를 고정하고 다른 내장 장치에 연결합니다.이제 USB 링크를 사용하여 카메라를 해당 임베디드 장치에 연결하고 켭니다. 스마트워치도 켭니다. 그런 다음 모든 임베디드 장치와 스마트워치를 라우터의 Wi-Fi 네트워크에 연결합니다.카메라를 각 학생의 책상 모서리에 놓고 앞을 가리키고 약간 위쪽으로 기울여 학생의 얼굴이 명확하게 보이도록 합니다. 그런 다음 카메라가 있는 두 개의 삼각대를 교실 첫 번째 줄의 통로에서 가장 가까운 책상 앞에 놓습니다. 대부분의 학생을 명확하게 볼 수 있도록 삼각대 라이저를 가장 높은 위치로 조정하십시오.스마트워치 및 임베디드 장치에서 캡처 소프트웨어를 실행하여 이미지와 가속도계, 자이로스코프, 심박수 및 조명 데이터를 보냅니다. 또한 서버 소프트웨어를 실행하여 이 데이터를 수집하고 저장합니다. 다음으로, 천정 카메라를 배치하여 장면을 지배하여 학생들의 신체를 명확하게 볼 수 있도록 합니다.또한 각 학생 앞에 추가 카메라를 배치하십시오. 학생들에게 자리에 앉게 하고, 실험의 목표에 대해 알려주고, 주로 사용하는 손에 스마트워치를 착용하도록 지시합니다. 서버에서 데이터 수집 및 수집을 시작하고 일반 수업 수업을 재개합니다.

hardware-software-class-setup-to-detect-attention-levels

Research

JoVE Journal

Engineering

Hardware, Software, and Class Setup to Detect Attention Levels in Students

Artificial Intelligence-Based System for Detecting Attention Levels in Students

The attention level of students in a classroom can be improved through the use of Artificial Intelligence (AI) techniques. By automatically identifying ...