ellibsЭлектронные библиотекиRussian Digital Libraries Journal1562-5419Казанский (Приволжский) федеральный университет10.26907/1562-5419-2020-23-5-1076-1092ellibs-249Research ArticleСтатьиРаспределенная тренировка ML-модели на мобильных устройствахDistributed Training of ML Model on Mobile DevicesСимонД. В.SimonD. V.denis.v.simon@gmail.comШаховаИ. С.ShakhovaI. S.is@it.kfu.ruВысшая школа информационных технологий и интеллектуальных систем Казанского (Приволжского) федерального университетаHigher Institute of Information Technology and Intelligent Systems20202810202023510761092Copyright © Симон Д.В., Шахова И.С., 20202020Симон Д.В., Шахова И.С.Simon D.V., Shakhova I.S.Данная работа распространяется под лицензией Creative Commons Attribution 4.0.This work is licensed under a Creative Commons Attribution 4.0 License.https://ellibs.elpub.ru/jour/article/view/249

В настоящее время потребность в наличии решений по распределенной тренировке ML-модели в мире возрастает. Однако существующие инструменты, в частности, TensorFlow Federated, – в самом начале своего развития, сложны в реализации и пригодны на текущий момент исключительно для симуляции на серверах. Для мобильных устройств надежно работающих подходов для достижения этой цели не существует. В статье спроектирован и представлен подход к такой распределенной тренировке ML-модели на мобильных устройствах, реализуемый с использованием существующих технологий. В его основе лежит концепция model personalization. В данном подходе эта концепция улучшена как следствие смягчения выявленных недостатков. Процесс реализации выстроен так, чтобы на всех этапах работы с ML-моделью использовать только один язык программирования Swift (применяются Swift for TensorFlow и Core ML 3), делая такой подход еще более удобным и надежным благодаря общей кодовой базе.

Currently, the need for distributed ML training solutions in the world is increasing. However, existing tools, in particular TensorFlow Federated, are at the very beginning of their development, difficult to implement, and currently suitable exclusively for simulation on servers. For mobile devices, reliable approaches for this purpose do not exist. This article has designed and presented an approach to such distributed training of the ML-model on mobile devices, implemented on existing technologies. It is based on the concept of model personalization. In this approach, this concept is improved as a consequence of mitigating the identified drawbacks. The implementation process is structured so that at all stages of working with the ML-model use only one Swift programming language (Swift for TensorFlow and Core ML 3 are used), making this approach even more convenient and reliable due to the common code base.

распределенная тренировка ML-моделимобильная разработкапрограммная инженериямашинное обучениеML-модельon-device MLon-device trainingedge computingReferencesBrendan McMahan and Daniel Ramage. Federated Learning: Collaborative Machine Learning without Centralized Training Data. 2017. URL: https://ai.googleblog.com/2017/04/federated-learning-collaborative.html (дата обращения: 03.03.2020)Brendan McMahan and Daniel Ramage. Federated Learning: Collaborative Machine Learning without Centralized Training Data. 2017. URL: https://ai.googleblog.com/2017/04/federated-learning-collaborative.html (дата обращения: 03.03.2020)Krishna Pillutla at all. Robust Aggregation for Federated Learning. 2019. arXiv:1912.13445 [stat.ML]Krishna Pillutla at all. Robust Aggregation for Federated Learning. 2019. arXiv:1912.13445 [stat.ML]What’s New in the iOS SDK // Apple. 2019. URL: https://developer.apple.com/ios/whats-new/ (дата обращения: 03.03.2020)What’s New in the iOS SDK // Apple. 2019. URL: https://developer.apple.com/ios/whats-new/ (дата обращения: 03.03.2020)Apple. What’s New in Core ML 3. 2019. URL: https://developer.apple.com/ machine-learning/core-ml/ (дата обращения: 03.03.2020)Apple. What’s New in Core ML 3. 2019. URL: https://developer.apple.com/ machine-learning/core-ml/ (дата обращения: 03.03.2020)MIT Technology Review. From cloud to the edge: On-device artificial intelligence boosts performance. 2019. URL: https://www.technologyreview.com/s/ 613527/from-cloud-to-the-edge-on-device-artificial-intelligence-boosts-performance/ (дата обращения: 03.03.2020)MIT Technology Review. From cloud to the edge: On-device artificial intelligence boosts performance. 2019. URL: https://www.technologyreview.com/s/ 613527/from-cloud-to-the-edge-on-device-artificial-intelligence-boosts-performance/ (дата обращения: 03.03.2020)B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas. Communication-Efficient Learning of Deep Networks from Decentralized Data. In Artificial Intelligence and Statistics, 2017. P. 1273–1282.B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas. Communication-Efficient Learning of Deep Networks from Decentralized Data. In Artificial Intelligence and Statistics, 2017. P. 1273–1282.K. Bonawitz, V. Ivanov, B. Kreuter, A. Marcedone, H. B. McMahan, S. Patel, D. Ramage, A. Segal, and K. Seth. Practical Secure Aggregation for Privacy-Preserving Machine Learning. In ACM SIGSAC Conference on Computer and Communications Security, стр. 1175–1191, 2017.K. Bonawitz, V. Ivanov, B. Kreuter, A. Marcedone, H. B. McMahan, S. Patel, D. Ramage, A. Segal, and K. Seth. Practical Secure Aggregation for Privacy-Preserving Machine Learning. In ACM SIGSAC Conference on Computer and Communications Security, стр. 1175–1191, 2017.Get Started with Federated Learning for Data privacy // Leapfrog. 2019. URL: https://www.lftechnology.com/blog/ai/federated-learning-data-privacy/ (дата обращения: 03.03.2020)Get Started with Federated Learning for Data privacy // Leapfrog. 2019. URL: https://www.lftechnology.com/blog/ai/federated-learning-data-privacy/ (дата обращения: 03.03.2020)Theo Ryffel at all. A generic framework for privacy preserving deep learning. 2018. arXiv:1811.04017 [cs.LG]Theo Ryffel at all. A generic framework for privacy preserving deep learning. 2018. arXiv:1811.04017 [cs.LG]Personalizing a Model with On-Device Updates // Apple. 2019. URL: https://developer.apple.com/documentation/coreml/core_ml_api/personalizing_a_model_with_on-device_updates (дата обращения: 03.03.2020)Personalizing a Model with On-Device Updates // Apple. 2019. URL: https://developer.apple.com/documentation/coreml/core_ml_api/personalizing_a_model_with_on-device_updates (дата обращения: 03.03.2020)Matthijs Hollemans. On-device training with Core ML – part 1. 2019. URL: https://machinethink.net/blog/coreml-training-part1/ (дата обращения: 03.03.2020)Matthijs Hollemans. On-device training with Core ML – part 1. 2019. URL: https://machinethink.net/blog/coreml-training-part1/ (дата обращения: 03.03.2020)Jason Brownlee. A Gentle Introduction to Transfer Learning for Deep Learning. 2017. URL: https://machinelearningmastery.com/transfer-learning- for-deep-learning/ (дата обращения: 03.03.2020)Jason Brownlee. A Gentle Introduction to Transfer Learning for Deep Learning. 2017. URL: https://machinelearningmastery.com/transfer-learning- for-deep-learning/ (дата обращения: 03.03.2020)Alexander Rakhlin. Online Methods in Machine Learning. 2016. URL: http://www.mit.edu/~rakhlin/6.883/Alexander Rakhlin. Online Methods in Machine Learning. 2016. URL: http://www.mit.edu/~rakhlin/6.883/Martijn Willemsen. Anonymizing Unstructured Data to Prevent Privacy Leaks during Data Mining. 2016. URL: https://www.semanticscholar.org/paper/ Anonymizing-Unstructured-Data-to-Prevent-Privacy-Willemsen/40781ab4856f3d50af8ecda8f9aa1851c2e027eb (дата обращения: 03.03.2020)Martijn Willemsen. Anonymizing Unstructured Data to Prevent Privacy Leaks during Data Mining. 2016. URL: https://www.semanticscholar.org/paper/ Anonymizing-Unstructured-Data-to-Prevent-Privacy-Willemsen/40781ab4856f3d50af8ecda8f9aa1851c2e027eb (дата обращения: 03.03.2020)Adam Drake. Scalable Machine Learning with Fully Anonymized Data. 2018. URL: https://adamdrake.com/scalable-machine-learning-with-fully- anonymized-data.html (дата обращения: 03.03.2020)Adam Drake. Scalable Machine Learning with Fully Anonymized Data. 2018. URL: https://adamdrake.com/scalable-machine-learning-with-fully- anonymized-data.html (дата обращения: 03.03.2020)Analytics Vidhya. Introduction to Apple’s Core ML 3 – Build Deep Learning Models for the iPhone. 2019. URL: https://www.analyticsvidhya.com/blog/2019/11/ introduction-apple-core-ml-3-deep-learning-models-iphone/ (дата обращения: 03.03.2020)Analytics Vidhya. Introduction to Apple’s Core ML 3 – Build Deep Learning Models for the iPhone. 2019. URL: https://www.analyticsvidhya.com/blog/2019/11/ introduction-apple-core-ml-3-deep-learning-models-iphone/ (дата обращения: 03.03.2020)

The authors declare that there are no conflicts of interest present.