Intelligent Interaction Laboratory

Objective: Confusion is the primary epistemic emotion in the learning process, influencing students’ engagement and whether they become frustrated or bored. However, research on confusion in learning is still in its early stages, and there is a need to better understand how to recognize it and what EEG signals indicate its occurrence. The present work investigates confusion during reasoning learning using EEG, and aims to fill this gap with a multidisciplinary approach combining educational psychology, neuroscience and computer science. Approach: First, we design an experiment to actively and accurately induce confusion in reasoning. Second, we propose a subjective and objective joint labeling technique to address the label noise issue. Third, to confirm that the confused state can be distinguished from the non-confused state, we compare and analyze the mean band power of confused and unconfused states across five typical bands. Finally, we present an EEG database for confusion analysis, together with benchmark results from conventional (Naive Bayes, SVM, Random Forest, and ANN) and end-to-end (LSTM, ResNet, and EEGNet) machine learning methods. Main results: Findings revealed: 1. Significant differences in the power of delta, theta, alpha, beta and lower gamma between confused and non-confused conditions; 2. A higher attentional and cognitive load when participants were confused; and 3. The Random Forest algorithm with time-domain features achieved a high accuracy/F1 score (88.06%/0.88 for the subject-dependent approach and 84.43%/0.84 for the subject-independent approach) in the binary classification of the confused and non-confused states. Significance: The study advances our understanding of confusion and provides practical insights for recognizing and analyzing it in the learning process. It extends existing theories on the differences between confused and non-confused states during learning and contributes to the cognitive-affective model. The research enables researchers, educators, and practitioners to monitor confusion, develop adaptive systems, and test recognition approaches.

One major challenge in appearance-based gaze estimation is the lack of high-quality labeled data. Establishing databases or datasets is a way to obtain accurate gaze data and test methods or tools. However, the methods of collecting data in existing databases are designed on artificial chasing target tasks or unintentional free-looking tasks, which are not natural and real eye interactions and cannot reflect the inner cognitive processes of humans. To fill this gap, we propose the first gaze estimation dataset collected from an actual psychological experiment by the eye tracker, called the RavenGaze dataset. We design an experiment employing Raven’s Matrices as visual stimuli and collecting gaze data, facial videos as well as screen content videos simultaneously. Thirty-four participants were recruited. The results show that the existing algorithms perform well on our RavenGaze dataset in the 3D and 2D gaze estimation task, and demonstrate good generalization ability according to cross-dataset evaluation task. RavenGaze and the establishment of the benchmark lay the foundation for other researchers to do further in-depth research and test their methods or tools. Our dataset is available at https://intelligentinteractivelab.github.io/datasets/RavenGaze/index.html.

Falling, as one of the main harm threats to the elderly, has drawn researchers’ attentions and has always been one of the most valuable research topics in the daily health-care for the elderly in last two decades. Before 2014, several researchers reviewed the development of fall detection, presented issues and challenges, and navigated the direction for the study in the future. With smart sensors and Internet of Things (IoT) developing rapidly, this field has made great progress. However, there is a lack of a review and discussion on novel sensors, technologies and algorithms introduced and employed from 2014, as well as the emerging challenges and new issues. To bridge this gap, we present an overview of fall detection research and discuss the core research questions on this topic. A total of 6830 related documents were collected and analyzed based on the key words. Among these documents, the twenty most influential and highly cited articles are selected and discussed profoundly from three perspectives: sensors, algorithms and performance. The findings would assist researchers in understanding current developments and barriers in the systems of fall detection. Although researchers achieve fruitful work and progress, this research domain still confronts challenges on theories and practice. In the near future, the new solutions based on advanced IoT will sustainably urge the development to prevent falling injuries.