ellibsЭлектронные библиотекиRussian Digital Libraries Journal1562-5419Казанский (Приволжский) федеральный университет10.26907/1562-5419-2020-23-6-1155-1171ellibs-254Research ArticleСтатьиОпределение дефектов на стальных листах с использованием сверточных нейронных сетейSteel Defects Analysis Using CNN (Convolutional Neural Networks)ГаскаровР. Д.GaskarovR. D.komnatakita@gmail.comБирюковА. М.BiryukovA. M.herou.public@gmail.comНиконовА.NikonovA. F.nikonowalex@gmail.comАгниашвилиД. В.AgniashviliD. V.daniilak@gmail.comХайрисламовД. А.KhayrislamovD. A.danilhayrislamov@gmail.com20202812202023611551171Copyright © Гаскаров Р.Д., Бирюков А.М., Никонов A.А., Агниашвили Д.В., Хайрисламов Д.А., 20202020Гаскаров Р.Д., Бирюков А.М., Никонов A.А., Агниашвили Д.В., Хайрисламов Д.А.Gaskarov R.D., Biryukov A.M., Nikonov A.F., Agniashvili D.V., Khayrislamov D.A.Данная работа распространяется под лицензией Creative Commons Attribution 4.0.This work is licensed under a Creative Commons Attribution 4.0 License.https://ellibs.elpub.ru/jour/article/view/254

Сталь в наши дни является одним из важнейших производственных материалов, который используется повсеместно, от медицины до промышленных отраслей. Своевременное обнаружение и распознавание дефектов на стальных листах после проката – одна из ключевых проблем этого производства с учетом его сложности и необходимости затрат большого количества времени на проведение вручную проверок каждого листа и каждой заготовки. Одними из целей настоящей работы были автоматизация и упрощение данного процесса. Для решения соответствующих задач была использована, в первую очередь, модель сверточной нейронной сети под названием UNet, которая уже зарекомендовала себя как отличный инструмент решения таких задач — при высокой результативности она требует меньшего количества учебных данных. В основе этой модели лежат последовательная, производимая в несколько шагов свертка изображения до приемлемого размера (иными словами, сжатие или кодирование), а затем развертка, восстановление изображения к исходному размеру и соотношению сторон, после чего на выходе будет получена маска изображения с классами элементов, которые необходимо было найти. В дополнение к этой нейронной сети в качестве кодирующего (сворачивающего) слоя была использована другая модель — ResNet34, предварительно обученная на датасете (наборе данных) ImageNet1000. В этой модели также был модифицирован выходной слой — вместо 34 слоев с классами на выходе возвращалось лишь 4, что сократило время обработки и позволило использовать наиболее удачные определения в результатах. Используя данный подход и проведя все необходимые проверки, при подведении итогов, мы получили результат в 94,8% точности определения дефектов на стальных листах.

Steel is one of the most important bulk materials these days. It is used almost everywhere - from medicine to industry. Detecting this material's defects is one of the most challenging problems for industries worldwide. This process is also manual and time-consuming. Through this study we tried to automate this process. A convolutional neural network model UNet was used for this task for more accurate segmentation with less training image data set for our model. The essence of this NN (neural network) is in step-by-step convolution of every image (encoding) and then stretching them to initial resolution, consequently getting a mask of an image with various classes on it. The foremost modification is changing an input image's size to 128x800 px resolution (original images in dataset are 256x1600 px) because of GPU memory size's limitation. Secondly, we used ResNet34 CNN (convolutional neural network) as encoder, which was pre-trained on ImageNet1000 dataset with modified output layer - it shows 4 layers instead of 34. After running tests of this model, we obtained 92.7% accuracy using images of hot-rolled steel sheets.

сверточные нейронные сетинейронные сетимашинное обучениестальискусственный интеллектопределение дефектовсегментацияклассификацияUNetResNetReferencesStavropoulos P. et al. Monitoring and control of manufacturing processes: A review // Procedia CIRP. 2013. V. 8. P. 421–425.Stavropoulos P. et al. Monitoring and control of manufacturing processes: A review // Procedia CIRP. 2013. V. 8. P. 421–425.Chao W. and al. Research on Classification of Surface Defects of Hot-rolled Steel Strip Based on Deep Learning // DEStech Transactions on Computer Science and Engineering. 2019. 2nd International Conference on Informatics, Control and Auto-mation (ICA 2019). 2019. P. 362–366.Chao W. and al. Research on Classification of Surface Defects of Hot-rolled Steel Strip Based on Deep Learning // DEStech Transactions on Computer Science and Engineering. 2019. 2nd International Conference on Informatics, Control and Auto-mation (ICA 2019). 2019. P. 362–366.Li J. and al. Real-time detection of steel strip surface defects based on im-proved yolo detection network // IFAC-PapersOnLine. 2018. V. 51, No. 21. P. 76–81.Li J. and al. Real-time detection of steel strip surface defects based on im-proved yolo detection network // IFAC-PapersOnLine. 2018. V. 51, No. 21. P. 76–81.Zhang Z., Zhang S., Li Q. Online Surface Defects Detection System for Cold-rolled Steel Strip // Recent Patents on Engineering. 2017. V. 11, No. 1. P. 62–67.Zhang Z., Zhang S., Li Q. Online Surface Defects Detection System for Cold-rolled Steel Strip // Recent Patents on Engineering. 2017. V. 11, No. 1. P. 62–67.He K. and al. Mask r-cnn // Proceedings of the IEEE international conference on computer vision, 2017. P. 2961–2969.He K. and al. Mask r-cnn // Proceedings of the IEEE international conference on computer vision, 2017. P. 2961–2969.Ren S. and al. Faster r-cnn: Towards real-time object detection with region proposal networks // Advances in Neural Information Processing Systems, 2015. P. 91–99.Ren S. and al. Faster r-cnn: Towards real-time object detection with region proposal networks // Advances in Neural Information Processing Systems, 2015. P. 91–99.Shang W. and al. Understanding and improving convolutional neural net-works via concatenated rectified linear units // International Conference on Machine Learning, 2016. P. 2217–2225.Shang W. and al. Understanding and improving convolutional neural net-works via concatenated rectified linear units // International Conference on Machine Learning, 2016. P. 2217–2225.Ferguson M.K. and al. Detection and segmentation of manufacturing defects with convolutional neural networks and transfer learning // Smart and Sustainable Manufacturing Systems. 2018. V. 2. arXiv:1808.02518 [cs.CV]Ferguson M.K. and al. Detection and segmentation of manufacturing defects with convolutional neural networks and transfer learning // Smart and Sustainable Manufacturing Systems. 2018. V. 2. arXiv:1808.02518 [cs.CV]Ronneberger O., Fischer P., Brox T. U-net: Convolutional networks for bio-medical image segmentation // International Conference on Medical Image Compu-ting and Computer-Assisted Intervention. Springer, Cham, 2015. P. 234–241.Ronneberger O., Fischer P., Brox T. U-net: Convolutional networks for bio-medical image segmentation // International Conference on Medical Image Compu-ting and Computer-Assisted Intervention. Springer, Cham, 2015. P. 234–241.Wu Z., Shen C., Van Den Hengel A. Wider or deeper: Revisiting the resnet model for visual recognition // Pattern Recognition. 2019. V. 90. P. 119–133.Wu Z., Shen C., Van Den Hengel A. Wider or deeper: Revisiting the resnet model for visual recognition // Pattern Recognition. 2019. V. 90. P. 119–133.He K. and al. Deep residual learning for image recognition // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016. P. 770–778.He K. and al. Deep residual learning for image recognition // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016. P. 770–778.Ronneberger O., Fischer P., Brox T. U-net: Convolutional networks for bio-medical image segmentation // International Conference on Medical Image Compu-ting and Computer-Assisted Intervention. Springer, Cham, 2015. P. 234–241.Ronneberger O., Fischer P., Brox T. U-net: Convolutional networks for bio-medical image segmentation // International Conference on Medical Image Compu-ting and Computer-Assisted Intervention. Springer, Cham, 2015. P. 234–241.Salehi S.S.M., Erdogmus D., Gholipour A. Tversky loss function for image segmentation using 3D fully convolutional deep networks // International Workshop on Machine Learning in Medical Imaging. Springer, Cham, 2017. P. 379–387.Salehi S.S.M., Erdogmus D., Gholipour A. Tversky loss function for image segmentation using 3D fully convolutional deep networks // International Workshop on Machine Learning in Medical Imaging. Springer, Cham, 2017. P. 379–387.Milletari F., Navab N., Ahmadi S.A. V-net: Fully convolutional neural net-works for volumetric medical image segmentation // 2016 Fourth International Con-ference on 3D Vision (3DV). IEEE, 2016. P. 565–571.Milletari F., Navab N., Ahmadi S.A. V-net: Fully convolutional neural net-works for volumetric medical image segmentation // 2016 Fourth International Con-ference on 3D Vision (3DV). IEEE, 2016. P. 565–571.Wang P., Chung A.C.S. Focal dice loss and image dilation for brain tumor segmentation // Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support. Springer, Cham, 2018. P. 119–127.Wang P., Chung A.C.S. Focal dice loss and image dilation for brain tumor segmentation // Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support. Springer, Cham, 2018. P. 119–127.Sudre C.H. and al. Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations // Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support. Springer, Cham, 2017. P. 240–248.Sudre C.H. and al. Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations // Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support. Springer, Cham, 2017. P. 240–248.Sitdikov V.D. and al. Application of X-ray Technique to Study the Structure of Ultrafine-Grained Ferritic/Martensitic Steel // J. of Materials Engineering and Per-formance. 2019. V. 28, No. 11. P. 7109–7118.Sitdikov V.D. and al. Application of X-ray Technique to Study the Structure of Ultrafine-Grained Ferritic/Martensitic Steel // J. of Materials Engineering and Per-formance. 2019. V. 28, No. 11. P. 7109–7118.

The authors declare that there are no conflicts of interest present.