[ Article ]

Journal of the Korean Society of Manufacturing Technology Engineers - Vol. 34, No. 2, pp.113-119

ISSN: 2508-5107 (Online)

Print publication date 15 Apr 2025

Received 30 Dec 2024 Revised 24 Feb 2025 Accepted 07 Mar 2025

DOI: https://doi.org/10.7735/ksmte.2025.34.2.113

페이스 랜드마크 트래킹과 메타휴먼 재구성을 활용한 AI 기반 운전자 졸음 감지

김차엽^a ; 조춘묵^a ; 신진섭^b ; 서영호^a^{, b} ; 김병희^a^{, b}^{, *}

AI-Based Driver Drowsiness Detection via Facial Landmark Tracking and Metahuman Reconstruction

Kim, ChaYeop^a ; Jo, ChunMuk^a ; Shin, JinSeob^b ; Seo, YoungHo^a^{, b} ; Kim, ByeongHee^a^{, b}^{, *}

aDepartment of Mechanical Convergence Engineering, Major in Mechatronics Engineering, Kangwon National University
bDepartment of Smart Health Science and Technology, Kangwon National University

Correspondence to: ^*Tel.: +82-33-244-8910 E-mail address: kbh@kangwon.ac.kr (ByeongHee Kim).

Abstract

Drowsy driving poses a significant risk, resulting in an average of 2.9 fatalities per 100 accidents—nearly twice the 1.5 fatalities associated with drunk driving. Cognitive function declines when in-vehicle CO2 levels exceed 2,000 ppm, indicating the necessity for real-time drowsiness detection. This study proposes an AI-based face-tracking system that integrates drowsiness detection with CO2 measurement. Infrared cameras within VR HMDs capture subtle facial muscle movements, which are transmitted to a metahuman model within a 3D engine via Live Link. A supervised model, trained on 2,400 metahuman images, underpins drowsiness detection. The metahuman background color dynamically adjusts in response to CO2 concentration, facilitating intuitive monitoring. If prolonged drowsiness is detected, the system issues a warning. The performance of the AI model was validated using k-fold cross-validation and mean average precision. This approach enables real-time driver monitoring by delivering multistage warnings, immediate feedback, and vehicle control interventions when necessary.

Keywords:

VR HMD, 3D simulation engine, Face tracking, Object detection, CO2 concentration

Acknowledgments

본 연구는 2024년도 교육부의 재원으로 한국연구재단의 지원을 받아 수행된 지자체-대학 협력기반 지역혁신 사업의 연구(2022RIS-005) 및 강원대학교 산업기술연구소의 지원을 받은 연구입니다.

References

Korean National Police Agency, 2024, viewed 27 November 2024, Beware of Drowsy Driving in Spring, <https://www.police.go.kr/user/bbs/BD_selectBbs.do?q_bbsCode=1002&q_bbscttSn=20240404142330339&q_tab=&q_searchKeyTy=sj___1002&q_searchVal=%EB%B4%84%EC%B2%A0%20%EB%B6%88%EC%B2%AD%EA%B0%9D&q_rowPerPage=10&q_currPage=1&q_sortName=&q_sortOrder=&, >.
Satish, U., Mendell, M. J., Shekhar, K., Hotchi, T., Sullivan, D., Streufert, S., Fisk, W. J., 2012, Is CO2 an Indoor Pollutant? Direct Effects of Low-to-Moderate CO2 Concentrations on Human Decision-Making Performance, Environ. Health Perspect., 120:12 1671-1677. [https://doi.org/10.1289/ehp.1104789]
Ji, Q., Zhu, Z., Lan, P., 2004, Real-Time Nonintrusive Monitoring and Prediction of Driver Fatigue, IEEE Transactions on Vehicular Technology, 53:4 1052-1068. [https://doi.org/10.1109/TVT.2004.830974]
Patel, M., Lal, S. K. L., Kavanagh, D., Rossiter, P., 2011, Applying Neural Network Analysis on Heart Rate Variability Data to Assess Driver Fatigue, Expert Syst. Appl., 38:6 7235-7242. [https://doi.org/10.1016/j.eswa.2010.12.028]
Lin, C.-T., Huang, K.-C., Chao, C.-F., Chen, J.-A., Chiu, T.-W., Ko, L.-W., Jung, T.-P., 2010, Tonic and Phasic EEG and Behavioral Changes Induced by Arousing Feedback, NeuroImage, 52:2 633-642. [https://doi.org/10.1016/j.neuroimage.2010.04.250]
Tashakori, M., Nahvi, A., Ebrahimian, Kiashari, S. E. H., 2021, Driver Drowsiness Detection Using Facial Thermal Imaging in a Driving Simulator, Proc. Inst. Mech. Eng. Part H-J. Eng. Med. 236:1 43-55. [https://doi.org/10.1177/09544119211044232]
Viola, P., Jones, M., 2001, Rapid Object Detection Using a Boosted Cascade of Simple Features, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2001), 511-518. [https://doi.org/10.1109/CVPR.2001.990517]
Dalal, N., Triggs, B., 2005, Histograms of Oriented Gradients for Human Detection, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05), 886-893. [https://doi.org/10.1109/CVPR.2005.177]
Zhang, K., Zhang, Z., Li, Z., Qiao, Y., 2016, Joint Face Detection and Alignment using Multitask Cascaded Convolutional Networks, IEEE Signal Processing Letters, 23:10 1499-1503. [https://doi.org/10.1109/LSP.2016.2603342]
Deng, J., Guo, J., Ververas, E., Kotsia, I., Zafeiriou, S., 2020, RetinaFace: Single-shot Multi-Level Face Localisation in the Wild, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 5203-5212. [https://doi.org/10.1109/CVPR42600.2020.00525]
Cootes, T. F., Taylor, C. J., Cooper, D. H., Graham, J., 1995, Active Shape Models—Their Training and Application, Comput. Vis. Image Underst., 61:1 38-59. [https://doi.org/10.1006/cviu.1995.1004]
Kazemi, V., Sullivan, J., 2014, One Millisecond Face Alignment with an Ensemble of Regression Trees, 2014 IEEE Conference on Computer Vision and Pattern Recognition, 1867-1874. [https://doi.org/10.1109/CVPR.2014.241]
Newell, A., Yang, K., Deng, J., 2016, Stacked Hourglass Networks for Human Pose Estimation, 14th European Conference Computer Vision-ECCV2016, 483-499. [https://doi.org/10.1007/978-3-319-46484-8_29]
Blanz, V., Vetter, T., 1999, A Morphable Model for the Synthesis of 3D Faces, Proc. 26th Annu. Conf. Comput. Graph. Interactive Tech. (SIGGRAPH), 187-194. [https://doi.org/10.1145/311535.311556]
Ekman, P., Friesen, W. V., 1978, Facial Action Coding System, Environmental Psychology & Nonverbal Behavior. [https://doi.org/10.1037/t27734-000]
Lewis, J. P., Anjyo, K., Rhee, T., Zhang, M., Pighin, F. H., Deng, Z., 2014, Practice and Theory of Blendshape Facial Models, Eurographics Conference (State of the Art Reports), 1(8), 2, 199-218. [https://doi.org/10.2312/egst.20141042]
Epic Games, n.d., viewed 5 December 2024, MetaHuman Documentation, <https://dev.epicgames.com/documentation/en-us/metahuman/metahuman-documentation, >.
Epic Games, n.d., viewed 5 December 2024, Unreal Engine Official Website, <https://www.unrealengine.com/ko, >.
Larue, G. S., Rakotonirainy, A., Pettitt, A. N., 2011, Driving Performance Impairments Due to Hypovigilance on Monotonous Roads, Accid. Anal. Prev., 43:6 2037-2046. [https://doi.org/10.1016/j.aap.2011.05.023]
International Labour Organization (ILO) and World Health Organization (WHO), 2024, viewed 10 December 2024, Carbon Dioxide (ICSC: 0021), <https://chemicalsafety.ilo.org/dyn/icsc/showcard.display?p_card_id=0021&p_version=2&p_lang=en, >.

ChaYeop Kim

M.Sc. Candidate in the Department of Mechanical Convergence Engineering, Major in Mechatronics Engineering, Kangwon National University. His research interest is Digital Twin of Machine Tool.

E-mail: american33@kangwon.ac.kr

ChunMuk Jo

Ph.D. Candidate in the Department of Mechanical Convergence Engineering, Major in Mechatronics Engineering, Kangwon National University. His research interest is Mechanical Engineering and AI.

E-mail: ccm516@kangwon.ac.kr

JinSeob Shin

Ph.D. Candidate in the Department of Smart Health Science and Technology, Kangwon National University. His research interest is Mechanical Engineering.

E-mail: js_of_yadang@kangwon.ac.kr

YoungHo Seo

Professor in the Department of Smart Health Science and Technology, Kangwon National University. His research interest is Micro/Nanoscale Surface Texturing Technologies and their Applications in Various Sensor Systems.

E-mail: mems@kangwon.ac.kr

ByeongHee Kim

Professor in the Department of Smart Health Science and Technology, Kangwon National University. His research interest is Micro and Nano System Design and Precision Control of Machine Tools.

E-mail: kbh@kangwon.ac.kr