Objective: Confusion is the primary epistemic emotion in the learning process, influencing students’ engagement and whether they become frustrated or bored. However, research on confusion in learning is still in its early stages, and there is a need to better understand how to recognize it and what EEG signals indicate its occurrence. The present work investigates confusion during reasoning learning using EEG, and aims to fill this gap with a multidisciplinary approach combining educational psychology, neuroscience and computer science. Approach: First, we design an experiment to actively and accurately induce confusion in reasoning. Second, we propose a subjective and objective joint labeling technique to address the label noise issue. Third, to confirm that the confused state can be distinguished from the non-confused state, we compare and analyze the mean band power of confused and unconfused states across five typical bands. Finally, we present an EEG database for confusion analysis, together with benchmark results from conventional (Naive Bayes, SVM, Random Forest, and ANN) and end-to-end (LSTM, ResNet, and EEGNet) machine learning methods. Main results: Findings revealed: 1. Significant differences in the power of delta, theta, alpha, beta and lower gamma between confused and non-confused conditions; 2. A higher attentional and cognitive load when participants were confused; and 3. The Random Forest algorithm with time-domain features achieved a high accuracy/F1 score (88.06%/0.88 for the subject-dependent approach and 84.43%/0.84 for the subject-independent approach) in the binary classification of the confused and non-confused states. Significance: The study advances our understanding of confusion and provides practical insights for recognizing and analyzing it in the learning process. It extends existing theories on the differences between confused and non-confused states during learning and contributes to the cognitive-affective model. The research enables researchers, educators, and practitioners to monitor confusion, develop adaptive systems, and test recognition approaches.
@article{10.1088/1741-2552/acbfe0,author={Xu, Tao and Wang, Jiabao and Zhang, Gaotian and Zhang, Ling and Zhou, Yun},title={Confused or not: decoding brain activity and recognizing confusion in reasoning learning using EEG},journal={Journal of Neural Engineering},url={http://iopscience.iop.org/article/10.1088/1741-2552/acbfe0},doi={10.1088/1741-2552/acbfe0},year={2023},}
RavenGaze: A Dataset for Gaze Estimation Leveraging Psychological Experiment Through Eye Tracker
Tao Xu, Bo Wu, Yuqiong Bai, and 1 more author
In 2023 IEEE 17th International Conference on Automatic Face and Gesture Recognition (FG), 2023
One major challenge in appearance-based gaze estimation is the lack of high-quality labeled data. Establishing databases or datasets is a way to obtain accurate gaze data and test methods or tools. However, the methods of collecting data in existing databases are designed on artificial chasing target tasks or unintentional free-looking tasks, which are not natural and real eye interactions and cannot reflect the inner cognitive processes of humans. To fill this gap, we propose the first gaze estimation dataset collected from an actual psychological experiment by the eye tracker, called the RavenGaze dataset. We design an experiment employing Raven’s Matrices as visual stimuli and collecting gaze data, facial videos as well as screen content videos simultaneously. Thirty-four participants were recruited. The results show that the existing algorithms perform well on our RavenGaze dataset in the 3D and 2D gaze estimation task, and demonstrate good generalization ability according to cross-dataset evaluation task. RavenGaze and the establishment of the benchmark lay the foundation for other researchers to do further in-depth research and test their methods or tools. Our dataset is available at https://intelligentinteractivelab.github.io/datasets/RavenGaze/index.html.
@inproceedings{10042793,author={Xu, Tao and Wu, Bo and Bai, Yuqiong and Zhou, Yun},booktitle={2023 IEEE 17th International Conference on Automatic Face and Gesture Recognition (FG)},title={RavenGaze: A Dataset for Gaze Estimation Leveraging Psychological Experiment Through Eye Tracker},year={2023},volume={},number={},pages={1-6},doi={10.1109/FG57933.2023.10042793},}
@article{xu2022dagam,title={DAGAM: A domain adversarial graph attention model for subject independent EEG-based emotion recognition},author={Xu, Tao and Dang, Wang and Wang, Jiabao and Zhou, Yun},journal={Journal of Neural Engineering},year={2022},doi={10.1088/1741-2552/acae06},url={https://iopscience.iop.org/article/10.1088/1741-2552/acae06#back-to-top-target},dimensions={true},}
@article{zhou2019beyond,title={Beyond engagement: an EEG-based methodology for assessing user’s confusion in an educational game},author={Zhou, Yun and Xu, Tao and Li, Shaoqi and Shi, Ruifeng},journal={Universal Access in the Information Society},volume={18},pages={551--563},year={2019},publisher={Springer}}
Guess or Not? A Brain-Computer Interface Using EEG Signals for Revealing the Secret behind Scores
Tao Xu, Yun Zhou, Yuhan Wang, and 2 more authors
In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, 2019
@inproceedings{xu2019guess,title={Guess or Not? A Brain-Computer Interface Using EEG Signals for Revealing the Secret behind Scores},author={Xu, Tao and Zhou, Yun and Wang, Yuhan and Zhao, Zichen and Li, Shiqian},booktitle={Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems},pages={1--6},year={2019},}
User attitudes and behaviors toward personalized control of privacy settings on smartphones
Yun Zhou, Lianyong Qi, Alexander Raake, and 3 more authors
Concurrency and Computation: Practice and Experience, 2019
2018
Confusion state induction and EEG-based detection in learning
Yun Zhou, Tao Xu, Shiqian Li, and 1 more author
In 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 2018
@inproceedings{zhou2018confusion,title={Confusion state induction and EEG-based detection in learning},author={Zhou, Yun and Xu, Tao and Li, Shiqian and Li, Shaoqi},booktitle={2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)},pages={3290--3293},year={2018},organization={IEEE},}
Analyzing Impact Factors for Smartphone Sharing Decisions Using Decision Tree
Tao Xu, Yun Zhou, Alexander Raake, and 1 more author
In Human-Computer Interaction. Interaction in Context: 20th International Conference, HCI International 2018, Las Vegas, NV, USA, July 15–20, 2018, Proceedings, Part II 20, 2018
@inproceedings{xu2018analyzing,title={Analyzing Impact Factors for Smartphone Sharing Decisions Using Decision Tree},author={Xu, Tao and Zhou, Yun and Raake, Alexander and Zhang, Xuyun},booktitle={Human-Computer Interaction. Interaction in Context: 20th International Conference, HCI International 2018, Las Vegas, NV, USA, July 15--20, 2018, Proceedings, Part II 20},pages={628--637},year={2018},organization={Springer}}
Learning emotions EEG-based recognition and brain activity: A survey study on BCI for intelligent tutoring system
@article{xu2018learning,title={Learning emotions EEG-based recognition and brain activity: A survey study on BCI for intelligent tutoring system},author={Xu, Tao and Zhou, Yun and Wang, Zi and Peng, Yixin},journal={Procedia computer science},volume={130},pages={376--382},year={2018},url={https://www.sciencedirect.com/science/article/pii/S1877050918304095},publisher={Elsevier},}
Learning in doing: a model of design and assessment for using new interaction in educational game
Yun Zhou, Tao Xu, Zhixin Zhu, and 1 more author
In Learning and Collaboration Technologies. Learning and Teaching: 5th International Conference, LCT 2018, Held as Part of HCI International 2018, Las Vegas, NV, USA, July 15-20, 2018, Proceedings, Part II 5, 2018
@inproceedings{zhou2018learning,title={Learning in doing: a model of design and assessment for using new interaction in educational game},author={Zhou, Yun and Xu, Tao and Zhu, Zhixin and Wang, Zi},booktitle={Learning and Collaboration Technologies. Learning and Teaching: 5th International Conference, LCT 2018, Held as Part of HCI International 2018, Las Vegas, NV, USA, July 15-20, 2018, Proceedings, Part II 5},pages={225--236},year={2018},organization={Springer}}
@article{lyu2018synergized,title={A synergized pulsing-imaging network (SPIN)},author={Lyu, Qing and Xu, Tao and Shan, Hongming and Wang, Ge},journal={arXiv preprint arXiv:1805.12006},year={2018},doi={10.1016/j.procs.2018.04.056},url={https://doi.org/10.1016/j.procs.2018.04.056},dimensions={true}}
Falling, as one of the main harm threats to the elderly, has drawn researchers’ attentions and has always been one of the most valuable research topics in the daily health-care for the elderly in last two decades. Before 2014, several researchers reviewed the development of fall detection, presented issues and challenges, and navigated the direction for the study in the future. With smart sensors and Internet of Things (IoT) developing rapidly, this field has made great progress. However, there is a lack of a review and discussion on novel sensors, technologies and algorithms introduced and employed from 2014, as well as the emerging challenges and new issues. To bridge this gap, we present an overview of fall detection research and discuss the core research questions on this topic. A total of 6830 related documents were collected and analyzed based on the key words. Among these documents, the twenty most influential and highly cited articles are selected and discussed profoundly from three perspectives: sensors, algorithms and performance. The findings would assist researchers in understanding current developments and barriers in the systems of fall detection. Although researchers achieve fruitful work and progress, this research domain still confronts challenges on theories and practice. In the near future, the new solutions based on advanced IoT will sustainably urge the development to prevent falling injuries.
@article{xu2018new,title={New advances and challenges of fall detection systems: A survey},author={Xu, Tao and Zhou, Yun and Zhu, Jing},journal={Applied Sciences},volume={8},number={3},pages={418},year={2018},publisher={MDPI},doi={10.3390/app8030418},url={https://doi.org/10.3390/app8030418},dimensions={true},}
Elders’ fall detection based on biomechanical features using depth camera
Tao Xu, and Yun Zhou
International journal of wavelets, multiresolution and information processing, 2018
@article{xu2018elders,title={Elders’ fall detection based on biomechanical features using depth camera},author={Xu, Tao and Zhou, Yun},journal={International journal of wavelets, multiresolution and information processing},volume={16},number={02},pages={1840005},year={2018},publisher={World Scientific},doi={10.1142/S0219691318400052},url={https://doi.org/10.1142/S0219691318400052},dimensions={true}}
Promoting knowledge construction: a model for using virtual reality interaction to enhance learning
@article{zhou2018promoting,title={Promoting knowledge construction: a model for using virtual reality interaction to enhance learning},author={Zhou, Yun and Ji, Shangpeng and Xu, Tao and Wang, Zi},journal={Procedia computer science},volume={130},pages={239--246},year={2018},publisher={Elsevier},doi={10.1016/j.procs.2018.04.035},url={https://doi.org/10.1016/j.procs.2018.04.035},dimensions={true},}
Guess or Not? A Brain-Computer Interface Using EEG Signals for Revealing the Secret behind ScoresIn Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, 2019
Confusion state induction and EEG-based detection in learningIn 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 2018
