ellibsЭлектронные библиотекиRussian Digital Libraries Journal1562-5419Казанский (Приволжский) федеральный университет10.26907/1562-5419-2021-24-4-582-603ellibs-288Research ArticleСтатьиОценка усталости человека методом анализа фотографий лица с помощью сверточных нейронных сетейHuman Fatigue Evaluation by Face's Image Analysis Based upon Convolutional Neural NetworksБайрамовА. И.BairamovA. I.bairamovazat@gmail.comФасхутдиновТ. Р.FaskhutdinovT. R.groboment@gmail.comТимергалинД. М.TimergalinD. M.qwisan@gmail.comЯмиковР. Р.YamikovR. R.jamrustem@yandex.ruМуртазинВ. Р.MurtazinV. R.dweyker@bk.ruТумановН. А.TumanovN. A.grooov173@yandex.ruКазанский (Приволжский) Федеральный университетKazan (Volga region) Federal University202128082021244582603Copyright © Байрамов А.И., Фасхутдинов Т.Р., Тимергалин Д.М., Ямиков Р.Р., Муртазин В.Р., Туманов Н.А., 20212021Байрамов А.И., Фасхутдинов Т.Р., Тимергалин Д.М., Ямиков Р.Р., Муртазин В.Р., Туманов Н.А.Bairamov A.I., Faskhutdinov T.R., Timergalin D.M., Yamikov R.R., Murtazin V.R., Tumanov N.A.Данная работа распространяется под лицензией Creative Commons Attribution 4.0.This work is licensed under a Creative Commons Attribution 4.0 License.https://ellibs.elpub.ru/jour/article/view/288

Представлены решения проблемы распознавания усталости человека по изображению его лица. Сначала рассмотрены уже существующие алгоритмы, а затем предложена и реализована модель собственной архитектуры. В заключении приведены итоговые показатели работы модели.

This article presents solutions to the person's fatigue recognition problem by the face's image analysis based on convolutional neural networks. In the present paper, existing algorithms were considered. A new model's architecture was proposed and implemented. Resultant metrics of the model were evaluated.

степень усталостисверточные нейронные сетимашинное обучениераспознавание усталостиобработка изображенийоценка усталости по изображению лицаResNet-152v2ReferencesHüllermeier E., Rifqi M. A Fuzzy Variant of the Rand Index for Comparing Clustering Structures. 2009. P. 1294–1298.Hüllermeier E., Rifqi M. A Fuzzy Variant of the Rand Index for Comparing Clustering Structures. 2009. P. 1294–1298.Levashov S.P. Monitoring i analiz professional'nykh riskov v Rossii i za rubezhom. Kurganskii gosudarstvennyi universitet.Levashov S.P. Monitoring i analiz professional'nykh riskov v Rossii i za rubezhom. Kurganskii gosudarstvennyi universitet.Phillips R.O. et al. Fatigue in transport: a review of exposure, risks, checks and controls // Transport Reviews. Routledge, 2017. Vol. 37, No. 6. P. 742–766.Phillips R.O. et al. Fatigue in transport: a review of exposure, risks, checks and controls // Transport Reviews. Routledge, 2017. Vol. 37, No. 6. P. 742–766.Qiang Ji, Zhiwei Zhu, Lan P. Real-time nonintrusive monitoring and prediction of driver fatigue // IEEE Transactions on Vehicular Technology. 2004. Vol. 53, No. 4. P. 1052–1068.Qiang Ji, Zhiwei Zhu, Lan P. Real-time nonintrusive monitoring and prediction of driver fatigue // IEEE Transactions on Vehicular Technology. 2004. Vol. 53, No. 4. P. 1052–1068.Sharafutdinova A.F. Sistema opredeleniya ustalosti voditelya // Vestnik Sov-remennykh Issledovanii. 2018. № 7.1 (22).Sharafutdinova A.F. Sistema opredeleniya ustalosti voditelya // Vestnik Sov-remennykh Issledovanii. 2018. № 7.1 (22).Singh S., Papanikolopoulos N.P. Monitoring driver fatigue using facial analysis techniques // Proceedings 199 IEEE/IEEJ/JSAI International Conference on Intelligent Transportation Systems (Cat. No. 99TH8383). Tokyo, Japan: IEEE, 1999. P. 314–318.Singh S., Papanikolopoulos N.P. Monitoring driver fatigue using facial analysis techniques // Proceedings 199 IEEE/IEEJ/JSAI International Conference on Intelligent Transportation Systems (Cat. No. 99TH8383). Tokyo, Japan: IEEE, 1999. P. 314–318.Wen-Bing Horng et al. Driver fatigue detection based on eye tracking and dy-namic template matching // IEEE International Conference on Networking, Sensing and Control, 2004. Taipei, Taiwan: IEEE, 2004. P. 7–12.Wen-Bing Horng et al. Driver fatigue detection based on eye tracking and dy-namic template matching // IEEE International Conference on Networking, Sensing and Control, 2004. Taipei, Taiwan: IEEE, 2004. P. 7–12.Sklonnost' k neschastnym sluchayam i faktory, ee obuslavlivayushchie | Ekstremal'na ta krizova psikhologіya [Electronic resource]. URL: http://extrpsy.nuczu.edu.ua/statya18 (accessed: 31.03.2021).Sklonnost' k neschastnym sluchayam i faktory, ee obuslavlivayushchie | Ekstremal'na ta krizova psikhologіya [Electronic resource]. URL: http://extrpsy.nuczu.edu.ua/statya18 (accessed: 31.03.2021).Prichiny neschastnykh sluchaev – Okhrana truda [Electronic resource]. URL: http://okhrana-truda.com/neschastnye-sluchai-i-professionalnye-zabolevaniya/prichiny-neschastnykh-sluchaev.html (accessed: 31.03.2021).Prichiny neschastnykh sluchaev – Okhrana truda [Electronic resource]. URL: http://okhrana-truda.com/neschastnye-sluchai-i-professionalnye-zabolevaniya/prichiny-neschastnykh-sluchaev.html (accessed: 31.03.2021).Chelovecheskii faktor kak prichina proizvodstvennogo travmatizma | Inter-aktivnyi portal Departamenta truda i zanyatosti naseleniya Tyumenskoi oblasti [Elec-tronic resource]. URL: https://czn.admtyumen.ru/News/Detail/7efb0823-7fee-4366-836e-5f4affa7f6d5 (accessed: 12.04.2021).Chelovecheskii faktor kak prichina proizvodstvennogo travmatizma | Inter-aktivnyi portal Departamenta truda i zanyatosti naseleniya Tyumenskoi oblasti [Elec-tronic resource]. URL: https://czn.admtyumen.ru/News/Detail/7efb0823-7fee-4366-836e-5f4affa7f6d5 (accessed: 12.04.2021).Obyazan li rabotodatel' vyplachivat' kompensatsiyu rabotniku, poluchivshemu travmu na proizvodstve? // Upravlenie personalom [Electronic resource]. URL: https://www.top-personal.ru/lawissue.html?2294 (accessed: 31.03.2021).Obyazan li rabotodatel' vyplachivat' kompensatsiyu rabotniku, poluchivshemu travmu na proizvodstve? // Upravlenie personalom [Electronic resource]. URL: https://www.top-personal.ru/lawissue.html?2294 (accessed: 31.03.2021).Evija.Feierabende. Glavnaya prichina travm na rabote – legkomyslennoe otnoshenie k bezopasnosti [Electronic resource]: Text // www.balta.lv. 2017. URL: https://www.balta.lv/ru/content/glavnaya-prichina-travm-na-rabote-–-legkomyslennoe-otnoshenie-k-bezopasnosti (accessed: 31.03.2021).Evija.Feierabende. Glavnaya prichina travm na rabote – legkomyslennoe otnoshenie k bezopasnosti [Electronic resource]: Text // www.balta.lv. 2017. URL: https://www.balta.lv/ru/content/glavnaya-prichina-travm-na-rabote-–-legkomyslennoe-otnoshenie-k-bezopasnosti (accessed: 31.03.2021).Sikander G., Anwar S. Driver Fatigue Detection Systems: A Review // IEEE Trans. Intell. Transport. Syst. 2019. Vol. 20, No. 6. P. 2339–2352.Sikander G., Anwar S. Driver Fatigue Detection Systems: A Review // IEEE Trans. Intell. Transport. Syst. 2019. Vol. 20, No. 6. P. 2339–2352.David F. Dinges, Richard Grace. PERCLOS: A Valid Psychophysiological Meas-ure of Alertness As Assessed by Psychomotor Vigilance. P. 4.David F. Dinges, Richard Grace. PERCLOS: A Valid Psychophysiological Meas-ure of Alertness As Assessed by Psychomotor Vigilance. P. 4.Yadav N., Banerjee K., Bali V. A Survey on Fatigue Detection of Workers Using Machine Learning // International Journal of E-Health and Medical Communications. 2020. Vol. 11, No. 3. P. 1–8.Yadav N., Banerjee K., Bali V. A Survey on Fatigue Detection of Workers Using Machine Learning // International Journal of E-Health and Medical Communications. 2020. Vol. 11, No. 3. P. 1–8.Bengio Y., Lecun Y. Convolutional Networks for Images, Speech, and Time-Series. 1997.Bengio Y., Lecun Y. Convolutional Networks for Images, Speech, and Time-Series. 1997.He K. et al. Deep Residual Learning for Image Recognition // arXiv:1512.03385 [cs]. 2015.He K. et al. Deep Residual Learning for Image Recognition // arXiv:1512.03385 [cs]. 2015.Realizuem i sravnivaem optimizatory modelei v glubokom obuchenii [Electron-ic resource]. URL: https://habr.com/ru/company/skillfactory/blog/525214/ (accessed: 10.05.2021).Realizuem i sravnivaem optimizatory modelei v glubokom obuchenii [Electron-ic resource]. URL: https://habr.com/ru/company/skillfactory/blog/525214/ (accessed: 10.05.2021).Global Z. Brutalk – Funktsiya aktivatsii Softmax s Python [Electronic resource] // URL: https://www.brutalk.com/ru/brutalk-blog/view/funkciya-aktivacii-softmax-s-python-6046b4a524b71 (accessed: 10.05.2021).Global Z. Brutalk – Funktsiya aktivatsii Softmax s Python [Electronic resource] // URL: https://www.brutalk.com/ru/brutalk-blog/view/funkciya-aktivacii-softmax-s-python-6046b4a524b71 (accessed: 10.05.2021).Fatigue_detection (Dataset) [Electronic resource]. URL: https://kaggle.com/timmjy/fatigue-detection (accessed: 05.04.2021).Fatigue_detection (Dataset) [Electronic resource]. URL: https://kaggle.com/timmjy/fatigue-detection (accessed: 05.04.2021).Colaboratory – Google [Electronic resource]. URL: https://research.google.com/colaboratory/faq.html (accessed: 11.05.2021).Colaboratory – Google [Electronic resource]. URL: https://research.google.com/colaboratory/faq.html (accessed: 11.05.2021).Keras: the Python deep learning API [Electronic resource]. URL: https://keras.io/ (accessed: 10.05.2021).Keras: the Python deep learning API [Electronic resource]. URL: https://keras.io/ (accessed: 10.05.2021).TensorFlow [Electronic resource]. URL: https://www.tensorflow.org/ (ac-cessed: 10.05.2021).TensorFlow [Electronic resource]. URL: https://www.tensorflow.org/ (ac-cessed: 10.05.2021).Fatigure-detection [Electronic resource]. URL: https://github.com/bairamovazat/fatigure-detection (accessed: 10.05.2021).Fatigure-detection [Electronic resource]. URL: https://github.com/bairamovazat/fatigure-detection (accessed: 10.05.2021).Advances in Face Detection and Facial Image Analysis / ed. Kawulok M., Celebi M.E., Smolka B. Cham: Springer International Publishing, 2016.Advances in Face Detection and Facial Image Analysis / ed. Kawulok M., Celebi M.E., Smolka B. Cham: Springer International Publishing, 2016.NeuralSetStorage [Electronic resource]. URL: https://github.com/bairamovazat/NeuralSetStorage (accessed: 10.05.2021).NeuralSetStorage [Electronic resource]. URL: https://github.com/bairamovazat/NeuralSetStorage (accessed: 10.05.2021).

The authors declare that there are no conflicts of interest present.