ellibsЭлектронные библиотекиRussian Digital Libraries Journal1562-5419Казанский (Приволжский) федеральный университет10.26907/1562-5419-2025-28-6-1415-1434ellibs-626Research ArticleСтатьиНекоторые подходы к повышению точности прогнозирования с использованием ансамблевых методовSome Approaches to Improving Prediction Accuracy using Ensemble MethodsМаСиньюэMaXinyuexinyuema35@gmail.comСенькоОлег ВалентиновичSen’KoOleg ValentinovichOSenko@frccsc.ruМосковский государственный университет имени М. В. ЛомоносоваLomonosov Moscow State UniversityФедеральный исследовательский центр «Информатика и управление» Российской академии наукFederal Research Center “Computer Science and Control” of the Russian Academy of Sciences20251912202528614151434Copyright © Ма С., Сенько О.В., 20252025Ма С., Сенько О.В.Ma X., Sen’Ko O.V.Данная работа распространяется под лицензией Creative Commons Attribution 4.0.This work is licensed under a Creative Commons Attribution 4.0 License.https://ellibs.elpub.ru/jour/article/view/626

Представлены результаты экспериментального исследования эффективности использования сверхслучайных деревьев в моделях, основанных на градиентном бустинге, а также в новом ансамблевом методе, в котором лес генерируется, исходя из условия повышенной внутренней дивергенции. сследована эффективность сверхслучайных деревьев при использовании расширенных наборов признаков с включением новых признаков, вычисляемых как расстояния Идо набора описаний опорных объектов из обучающей выборки. Показано, что использование сверхслучайных деревьев в моделях градиентного бустинга и дивергентного леса позволяет улучшить обобщающую способность, а также, что к еще большему росту обобщающей способности приводит использование расширенных наборов признаков.

This study presents the results of an experimental analysis evaluating the effectiveness of Extra Trees within gradient boosting models, as well as in a newly proposed ensemble framework where the forest is generated under conditions of enhanced internal divergence. Additionally, the paper explores the performance of Extra Trees when applied to novel feature representations computed as distances to a selected set of reference examples. It has been shown that the use of Extra Randomized Trees in gradient boosting and divergent forest models improves generalization ability. The use of expanded feature sets leads to even greater generalization ability.

регрессионное моделированиеансамблевое обучениеметрическое пространствометод сверхслучайных деревьевregression modelingensemble learningmetric spaceextremely randomized trees methodReferencesHabr. Open Machine Learning Course. Topic 5. Ensembles: Bagging, Random Forest. Available at: https://habr.com/ru/companies/ods/articles/324402/ (accessed 6 November 2025). (In Russ.).Habr. Open Machine Learning Course. Topic 5. Ensembles: Bagging, Random Forest. Available at: https://habr.com/ru/companies/ods/articles/324402/ (accessed 6 November 2025). (In Russ.).Dmitriev A.I., Zhuravlev Yu.I., Krendelev F.P. O matematicheskikh printsipakh klassifikatsii predmetov ili yavlenii [On the Mathematical Principles of the Classification of Objects and Phenomena] // Diskretnyi analiz [Discrete Analysis]. 1967. No. 7. P. 3–17 (In Russ.).Dmitriev A.I., Zhuravlev Yu.I., Krendelev F.P. O matematicheskikh printsipakh klassifikatsii predmetov ili yavlenii [On the Mathematical Principles of the Classification of Objects and Phenomena] // Diskretnyi analiz [Discrete Analysis]. 1967. No. 7. P. 3–17 (In Russ.).Vaintsvaig M.N. Algoritm obucheniya raspoznavaniyu obrazov “Kora” [Algorithm for pattern recognition learning “Kora”] // Algoritmy obucheniya raspoznavaniyu obrazov [Algorithms for pattern recognition learning]. Moscow: Sovetskoe radio, 1973. P. 8–12 (In Russ.).Vaintsvaig M.N. Algoritm obucheniya raspoznavaniyu obrazov “Kora” [Algorithm for pattern recognition learning “Kora”] // Algoritmy obucheniya raspoznavaniyu obrazov [Algorithms for pattern recognition learning]. Moscow: Sovetskoe radio, 1973. P. 8–12 (In Russ.).Heath D., Kasif S., Salzberg S. k-DT: A multi-tree learning method // Proceedings of the Second International Workshop on Multistrategy Learning. 1993. P. 138–149. https://doi.org/10.1007/0-387-34296-6_10Heath D., Kasif S., Salzberg S. k-DT: A multi-tree learning method // Proceedings of the Second International Workshop on Multistrategy Learning. 1993. P. 138–149. https://doi.org/10.1007/0-387-34296-6_10Breiman L. Random Forests // Machine Learning. 2001. Vol. 45, No. 1. P. 5–32. https://doi.org/10.1023/A:1010933404324Breiman L. Random Forests // Machine Learning. 2001. Vol. 45, No. 1. P. 5–32. https://doi.org/10.1023/A:1010933404324Breiman L. Bagging predictors // Machine Learning. 1996. Vol. 24, No. 2. P. 123–140. https://doi.org/10.1007/BF00058655Breiman L. Bagging predictors // Machine Learning. 1996. Vol. 24, No. 2. P. 123–140. https://doi.org/10.1007/BF00058655Ho T.K. The Random Subspace Method for Constructing Decision Forests // IEEE Transactions on Pattern Analysis and Machine Intelligence. 1998. Vol. 20, No. 8. P. 832–844. https://doi.org/10.1109/34.709601Ho T.K. The Random Subspace Method for Constructing Decision Forests // IEEE Transactions on Pattern Analysis and Machine Intelligence. 1998. Vol. 20, No. 8. P. 832–844. https://doi.org/10.1109/34.709601Freund Y., Schapire R.E. A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting // Journal of Computer and System Sciences. 1997. Vol. 55. P. 119–139. https://doi.org/10.1006/jcss.1997.1504Freund Y., Schapire R.E. A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting // Journal of Computer and System Sciences. 1997. Vol. 55. P. 119–139. https://doi.org/10.1006/jcss.1997.1504Friedman J.H. Stochastic Gradient Boosting // Computational Statistics & Data Analysis. 2002. Vol. 38, No. 4. P. 367–378. https://doi.org/10.1016/S0167-9473(01)00065-2Friedman J.H. Stochastic Gradient Boosting // Computational Statistics & Data Analysis. 2002. Vol. 38, No. 4. P. 367–378. https://doi.org/10.1016/S0167-9473(01)00065-2Zhou Z.H. Ensemble Methods: Foundations and Algorithms. New York: Chapman and Hall/CRC, 2012. 446 p. ISBN 978-1-4398-3003-1.Zhou Z.H. Ensemble Methods: Foundations and Algorithms. New York: Chapman and Hall/CRC, 2012. 446 p. ISBN 978-1-4398-3003-1.Hastie T., Tibshirani R., Friedman J. The Elements of Statistical Learning: Data Mining, Inference and Prediction. New York: Springer, 2009. 745 p. https://doi.org/10.1007/978-0-387-84858-7Hastie T., Tibshirani R., Friedman J. The Elements of Statistical Learning: Data Mining, Inference and Prediction. New York: Springer, 2009. 745 p. https://doi.org/10.1007/978-0-387-84858-7Beja-Battais P. Overview of AdaBoost: Reconciling its Views to Better Understand its Dynamics // arXiv preprint arXiv:2310.18323 [cs.LG]. 2023. https://doi.org/10.48550/arXiv.2310.18323Beja-Battais P. Overview of AdaBoost: Reconciling its Views to Better Understand its Dynamics // arXiv preprint arXiv:2310.18323 [cs.LG]. 2023. https://doi.org/10.48550/arXiv.2310.18323Chen T., Guestrin C. XGBoost: A Scalable Tree Boosting System // Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2016. P. 785–794. https://doi.org/10.48550/arXiv.1603.02754Chen T., Guestrin C. XGBoost: A Scalable Tree Boosting System // Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2016. P. 785–794. https://doi.org/10.48550/arXiv.1603.02754Ke G., Meng Q., Finley T., Wang T., Chen W., Ma W., Ye Q., Liu T.-Y. LightGBM: A Highly Efficient Gradient Boosting Decision Tree // Advances in Neural Information Processing Systems (NeurIPS). 2017. Vol. 30.Ke G., Meng Q., Finley T., Wang T., Chen W., Ma W., Ye Q., Liu T.-Y. LightGBM: A Highly Efficient Gradient Boosting Decision Tree // Advances in Neural Information Processing Systems (NeurIPS). 2017. Vol. 30.Hancock J.T., Khoshgoftaar T.M. CatBoost for big data: an interdisciplinary review // Journal of Big Data. 2020. Vol. 7, No. 94. 45 p. https://doi.org/10.1186/s40537-020-00369-8Hancock J.T., Khoshgoftaar T.M. CatBoost for big data: an interdisciplinary review // Journal of Big Data. 2020. Vol. 7, No. 94. 45 p. https://doi.org/10.1186/s40537-020-00369-8Zhuravlev Yu.I., Senko O.V., Dokukin A.A., Kiselyova N.N., Saenko I.A. Two-Level Regression Method Using Ensembles of Trees with Optimal Divergence // Doklady Mathematics. 2021. Vol. 103, No. 1. P. 1–4.Zhuravlev Yu.I., Senko O.V., Dokukin A.A., Kiselyova N.N., Saenko I.A. Two-Level Regression Method Using Ensembles of Trees with Optimal Divergence // Doklady Mathematics. 2021. Vol. 103, No. 1. P. 1–4.https://doi.org/10.1134/S1064562421040177https://doi.org/10.1134/S1064562421040177Dokukin A.A., Sen’ko O.V. A New Two-Level Machine Learning Method for Evaluating the Real Characteristics of Objects // Journal of Computer and Systems Sciences International. 2023. Vol. 62, No. 4. P. 607–614. https://doi.org/10.1134/S1064230723040020Dokukin A.A., Sen’ko O.V. A New Two-Level Machine Learning Method for Evaluating the Real Characteristics of Objects // Journal of Computer and Systems Sciences International. 2023. Vol. 62, No. 4. P. 607–614. https://doi.org/10.1134/S1064230723040020Senko O.V., Dokukin A.A., Kiselyova N.N., Dudarev V.A., Kuznetsova Yu.O. New Two-Level Ensemble Method and Its Application to Chemical Compounds Properties Prediction // Lobachevskii Journal of Mathematics. 2023. Vol. 44, No. 1. P. 188–197. https://doi.org/10.1134/S1995080223010341Senko O.V., Dokukin A.A., Kiselyova N.N., Dudarev V.A., Kuznetsova Yu.O. New Two-Level Ensemble Method and Its Application to Chemical Compounds Properties Prediction // Lobachevskii Journal of Mathematics. 2023. Vol. 44, No. 1. P. 188–197. https://doi.org/10.1134/S1995080223010341Geurts P., Ernst D., Wehenkel L. Extremely Randomized Trees // Machine Learning. 2006. Vol. 63, No. 1. P. 3–42. https://doi.org/10.1007/s10994-006-6226-1Geurts P., Ernst D., Wehenkel L. Extremely Randomized Trees // Machine Learning. 2006. Vol. 63, No. 1. P. 3–42. https://doi.org/10.1007/s10994-006-6226-1López-Iñesta E., Grimaldo F., Arevalillo-Herráez M. Combining feature extraction and expansion to improve classification-based similarity learning // Pattern Recognition Letters. 2016. Vol. 85. P. 84–90. https://doi.org/10.1016/j.patrec.2016.11.005López-Iñesta E., Grimaldo F., Arevalillo-Herráez M. Combining feature extraction and expansion to improve classification-based similarity learning // Pattern Recognition Letters. 2016. Vol. 85. P. 84–90. https://doi.org/10.1016/j.patrec.2016.11.005Breiman L., Friedman J., Olshen R.A., Stone C.J. Classification and Regression Trees. Monterey, CA: Wadsworth & Brooks/Cole, 1984. 358 p. https://doi.org/10.1201/9781315139470Breiman L., Friedman J., Olshen R.A., Stone C.J. Classification and Regression Trees. Monterey, CA: Wadsworth & Brooks/Cole, 1984. 358 p. https://doi.org/10.1201/9781315139470Mahalanobis P.C. On the Generalised Distance in Statistics (reprint of 1936) // Sankhya A. 2018. Vol. 80, Suppl. 1. P. 1–7. https://doi.org/10.1007/s13171-019-00164-5Mahalanobis P.C. On the Generalised Distance in Statistics (reprint of 1936) // Sankhya A. 2018. Vol. 80, Suppl. 1. P. 1–7. https://doi.org/10.1007/s13171-019-00164-5

The authors declare that there are no conflicts of interest present.