ellibsЭлектронные библиотекиRussian Digital Libraries Journal1562-5419Казанский (Приволжский) федеральный университет10.26907/1562-5419-2025-28-5-1207-1229ellibs-616Research ArticleСтатьиГде находятся лучшие признаки? Послойный анализ слоев трансформера для эффективной классификации эндоскопических изображенийWhere Do the Best Features Lie? A Layer-Wise Analysis of Frozen Encoders for Efficient Endoscopic Image ClassificationТахаАхмадTahaAhmada.taha@innopolis.universityЛукмановРустам А.LukmanovRustam A.r.lukmanov@innopolis.universityУниверситет ИннополисInnopolis University20251912202528512071229Copyright © Таха А., Лукманов Р.А., 20252025Таха А., Лукманов Р.А.Taha A., Lukmanov R.A.Данная работа распространяется под лицензией Creative Commons Attribution 4.0.This work is licensed under a Creative Commons Attribution 4.0 License.https://ellibs.elpub.ru/jour/article/view/616

В поисках путей развития медицинского искусственного интеллекта показано, что предварительно обученный Vision Transformer с линейным классификатором может достигать высокой и конкурентоспособной производительности в классификации эндоскопических изображений. Представлен систематический послойный анализ, который выявляет источник наиболее важных признаков, оспаривая общепринятую эвристику использования только последнего слоя. Установлен отчетливый феномен «пика перед концом», когда поздне-промежуточный слой предлагает более обобщаемое представление для последующей медицинской задачи. На стандартных наборах данных Kvasir и HyperKvasir предложенный подход с малым количеством параметров не только получить достаточно высокую точность, но и значительно сокращает вычислительные затраты. Полученные работы могут быть рекомендованы в качестве практического руководства по эффективному использованию признаков общих базовых моделей в клинических условиях.

In our quest to advance medical AI, we demonstrate that a pre-trained and frozen Vision Transformer paired with a linear classifier can achieve highly competitive performance in endoscopic image classification. Our central contribution is a systematic, layer-wise analysis that identifies the source of the most powerful features, challenging the common heuristic of using only the final layer. We uncover a distinct "peak-before-the-end" phenomenon, where a late-intermediate layer offers a more generalizable representation for the downstream medical task. On the Kvasir and HyperKvasir benchmarks, our parameter-light approach not only achieves excellent accuracy but also drastically reduces computational overhead. This work provides a practical roadmap for efficiently leveraging the power of general foundation models in clinical environments.

классификация эндоскопических изображенийзамороженный кодировщикизвлечение признаковпослойный анализвизуальный трансформер (ViT)перенос обучениясамоконтролируемое обучение (SSL)медицинский искусственный интеллектendoscopic image classificationfrozen encoderfeature extractionlayer-wise analysisvision transformer (ViT)transfer learningself-supervised learning (SSL)medical AIReferencesAbusuliman M., Jamali T., Zuchelli T.E. Advances in gastrointestinal endoscopy: A comprehensive review of innovations in cancer diagnosis and management // World Journal of Gastrointestinal Endoscopy. 2025. Vol. 17, No. 5. P. 105468.Abusuliman M., Jamali T., Zuchelli T.E. Advances in gastrointestinal endoscopy: A comprehensive review of innovations in cancer diagnosis and management // World Journal of Gastrointestinal Endoscopy. 2025. Vol. 17, No. 5. P. 105468.Simadibrata D.M., Lesmana E., Fass R. Role of endoscopy in gastroesophageal reflux disease // Clinical Endoscopy. 2023. Vol. 56, No. 6. P. 681–692.Simadibrata D.M., Lesmana E., Fass R. Role of endoscopy in gastroesophageal reflux disease // Clinical Endoscopy. 2023. Vol. 56, No. 6. P. 681–692.Mathews A.A., Draganov P.V., Yang D. Endoscopic management of colorectal polyps: From benign to malignant polyps // World Journal of Gastrointestinal Endoscopy. 2021. Vol. 13, No. 9. P. 356.Mathews A.A., Draganov P.V., Yang D. Endoscopic management of colorectal polyps: From benign to malignant polyps // World Journal of Gastrointestinal Endoscopy. 2021. Vol. 13, No. 9. P. 356.Bernatchi I.N., Voidazan S., Petrut M.I., Gabos G., Balasescu M., Nicolau C. Inter-observer variability on the value of endoscopic images for the documentation of upper gastrointestinal endoscopy – our center experience // Acta Marisiensis – Seria Medica. 2023.Bernatchi I.N., Voidazan S., Petrut M.I., Gabos G., Balasescu M., Nicolau C. Inter-observer variability on the value of endoscopic images for the documentation of upper gastrointestinal endoscopy – our center experience // Acta Marisiensis – Seria Medica. 2023.Ghazi G.G.R.J.J. et al. Sampling error in the diagnosis of colorectal cancer is associated with delay to surgery: a retrospective cohort study // Surgical Endoscopy. 2022. Vol. 36. P. 4893–4902.Ghazi G.G.R.J.J. et al. Sampling error in the diagnosis of colorectal cancer is associated with delay to surgery: a retrospective cohort study // Surgical Endoscopy. 2022. Vol. 36. P. 4893–4902.Khalifa M., Albadawy M. Ai in diagnostic imaging: Revolutionising accuracy and efficiency // Computer Methods and Programs in Biomedicine Update. 2024. Vol. 5. P. 100146.Khalifa M., Albadawy M. Ai in diagnostic imaging: Revolutionising accuracy and efficiency // Computer Methods and Programs in Biomedicine Update. 2024. Vol. 5. P. 100146.LeCun Y., Bengio Y., Hinton G. Deep learning // Nature. 2015. Vol. 521, No. 7553. P. 436–444.LeCun Y., Bengio Y., Hinton G. Deep learning // Nature. 2015. Vol. 521, No. 7553. P. 436–444.He K., Zhang X., Ren S., Sun J. Deep residual learning for image recognition // 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2015. P. 770-778.He K., Zhang X., Ren S., Sun J. Deep residual learning for image recognition // 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2015. P. 770-778.Dosovitskiy A. et al. An image is worth 16x16 words: Transformers for image recognition at scale // 3rd Conference on Neural Information Processing Systems. 2021.Dosovitskiy A. et al. An image is worth 16x16 words: Transformers for image recognition at scale // 3rd Conference on Neural Information Processing Systems. 2021.Deng J., Dong W., Socher R., Li L.-J., Li K., Fei-Fei L. Imagenet: A large-scale hierarchical image database // 2009 IEEE Conference on Computer Vision and Pattern Recognition. 2009. P. 248–255.Deng J., Dong W., Socher R., Li L.-J., Li K., Fei-Fei L. Imagenet: A large-scale hierarchical image database // 2009 IEEE Conference on Computer Vision and Pattern Recognition. 2009. P. 248–255.Su S.S. et al. Democratizing protein language models with parameter-efficient fine-tuning // Proceedings of the National Academy of Sciences of the United States of America. 2024. Vol. 121. P. e2405840121.Su S.S. et al. Democratizing protein language models with parameter-efficient fine-tuning // Proceedings of the National Academy of Sciences of the United States of America. 2024. Vol. 121. P. e2405840121.Sanchez-V T.S., Rahimi A., Oktay O., Bharadwaj S. Addressing the exorbitant cost of labeling medical images with active learning // International Conference on Machine Learning and Medical Imaging Analysis.Sanchez-V T.S., Rahimi A., Oktay O., Bharadwaj S. Addressing the exorbitant cost of labeling medical images with active learning // International Conference on Machine Learning and Medical Imaging Analysis.Zhang Z.Z. et al. Active, continual fine tuning of convolutional neural networks for reducing annotation efforts // Medical Image Analysis. 2021. Vol. 71. P. 101997.Zhang Z.Z. et al. Active, continual fine tuning of convolutional neural networks for reducing annotation efforts // Medical Image Analysis. 2021. Vol. 71. P. 101997.Pogorelov K. et al. Kvasir: A multi-class image dataset for computer aided gastrointestinal disease detection // Proceedings of the 8th ACM on Multimedia Systems Conference. 2017. P. 164–169.Pogorelov K. et al. Kvasir: A multi-class image dataset for computer aided gastrointestinal disease detection // Proceedings of the 8th ACM on Multimedia Systems Conference. 2017. P. 164–169.Borgli H. et al. Hyperkvasir, a comprehensive multi-class image and video dataset for gastrointestinal endoscopy // Scientific Data. 2020. Vol. 7, No. 1. P. 283.Borgli H. et al. Hyperkvasir, a comprehensive multi-class image and video dataset for gastrointestinal endoscopy // Scientific Data. 2020. Vol. 7, No. 1. P. 283.Huang G., Liu Z., van der Maaten L., Weinberger K.Q. Densely connected convolutional networks // 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2018. P. 2261–2269.Huang G., Liu Z., van der Maaten L., Weinberger K.Q. Densely connected convolutional networks // 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2018. P. 2261–2269.Shah S.T. et al. Comparison of vision transformers and convolutional neural networks in medical image analysis: A systematic review // Journal of Medical Systems. 2024. Vol. 48, No. 1. P. 84.Shah S.T. et al. Comparison of vision transformers and convolutional neural networks in medical image analysis: A systematic review // Journal of Medical Systems. 2024. Vol. 48, No. 1. P. 84.Rosenthal J.T., Beecy A., Sabuncu M.R. Rethinking clinical trials for medical ai with dynamic deployments of adaptive systems // npj Digital Medicine. 2025. Vol. 8, No. 1. P. 252.Rosenthal J.T., Beecy A., Sabuncu M.R. Rethinking clinical trials for medical ai with dynamic deployments of adaptive systems // npj Digital Medicine. 2025. Vol. 8, No. 1. P. 252.Hu E.J. et al. Lora: Low-rank adaptation of large language models // International Conference on Learning Representations. 2021.Hu E.J. et al. Lora: Low-rank adaptation of large language models // International Conference on Learning Representations. 2021.Farina M., Ahmad U., Taha A., Younes H., Mesbah Y., Yu X., Pedrycz W. Sparsity in transformers: A systematic literature review // Neurocomputing. 2024. Vol. 582. P. 127468.Farina M., Ahmad U., Taha A., Younes H., Mesbah Y., Yu X., Pedrycz W. Sparsity in transformers: A systematic literature review // Neurocomputing. 2024. Vol. 582. P. 127468.Chen T., Kornblith S., Norouzi M., Hinton G. A simple framework for contrastive learning of visual representations // Proceedings of the 37th International Conference on Machine Learning. 2020. P. 1597–1607.Chen T., Kornblith S., Norouzi M., Hinton G. A simple framework for contrastive learning of visual representations // Proceedings of the 37th International Conference on Machine Learning. 2020. P. 1597–1607.Yan Y.C. et al. Brain tumor intelligent diagnosis based on auto-encoder and u-net feature extraction // PLOS ONE. 2025. Vol. 20, No. 3. P. e0315631.Yan Y.C. et al. Brain tumor intelligent diagnosis based on auto-encoder and u-net feature extraction // PLOS ONE. 2025. Vol. 20, No. 3. P. e0315631.Jawahar B.S.G., Seddah D. What does bert learn about the structure of language? // Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019. P. 3651–3657.Jawahar B.S.G., Seddah D. What does bert learn about the structure of language? // Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019. P. 3651–3657.Lin M., Chen Q., Yan S. Network in network // 2nd International Conference on Learning Representations, ICLR 2014. 2014.Lin M., Chen Q., Yan S. Network in network // 2nd International Conference on Learning Representations, ICLR 2014. 2014.Kingma D.P., Ba J. Adam: A method for stochastic optimization // 5th International Conference on Learning Representations, ICLR 2017. 2017.Kingma D.P., Ba J. Adam: A method for stochastic optimization // 5th International Conference on Learning Representations, ICLR 2017. 2017.Siddiqui S., Khan J.A., Algamdi S. Deep ensemble learning for gastrointestinal diagnosis using endoscopic image classification // PeerJ Computer Science. 2025. Vol. 11. P. e2809.Siddiqui S., Khan J.A., Algamdi S. Deep ensemble learning for gastrointestinal diagnosis using endoscopic image classification // PeerJ Computer Science. 2025. Vol. 11. P. e2809.Zoph B., Vasudevan V., Shlens J., Le Q.V. Learning transferable architectures for scalable image recognition // 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2018. P. 8697-8705.Zoph B., Vasudevan V., Shlens J., Le Q.V. Learning transferable architectures for scalable image recognition // 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2018. P. 8697-8705.Tan M., Le Q.V. Efficientnet: Rethinking model scaling for convolutional neural networks // Proceedings of the 36th International Conference on Machine Learning. 2020. P. 6105–6114.Tan M., Le Q.V. Efficientnet: Rethinking model scaling for convolutional neural networks // Proceedings of the 36th International Conference on Machine Learning. 2020. P. 6105–6114.Szegedy C., Vanhoucke V., Ioffe S., Shlens J., Wojna Z. Rethinking the inception architecture for computer vision // 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2015. P. 2818–2826.Szegedy C., Vanhoucke V., Ioffe S., Shlens J., Wojna Z. Rethinking the inception architecture for computer vision // 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2015. P. 2818–2826.Ben-Younes D. et al. Perception encoder: The best visual embeddings are not at the output of the network // The Twelfth International Conference on Learning Representations. 2025.Ben-Younes D. et al. Perception encoder: The best visual embeddings are not at the output of the network // The Twelfth International Conference on Learning Representations. 2025.Tishby N., Pereira F.C., Bialek W. The information bottleneck method // 37th Annual Allerton Conference on Communication, Control, and Computing. 2000.Tishby N., Pereira F.C., Bialek W. The information bottleneck method // 37th Annual Allerton Conference on Communication, Control, and Computing. 2000.van der Maaten L., Hinton G. Visualizing data using t-sne // Journal of Machine Learning Research. 2008. Vol. 9. P. 2579–2605.van der Maaten L., Hinton G. Visualizing data using t-sne // Journal of Machine Learning Research. 2008. Vol. 9. P. 2579–2605.Kamboj A.K., Gaddam S., Lo S.K., Rezaie A. Irregular z-line: To biopsy or not to biopsy? // Digestive Diseases and Sciences. 2024. Vol. 69, No. 8. P. 2734–2740.Kamboj A.K., Gaddam S., Lo S.K., Rezaie A. Irregular z-line: To biopsy or not to biopsy? // Digestive Diseases and Sciences. 2024. Vol. 69, No. 8. P. 2734–2740.

The authors declare that there are no conflicts of interest present.