ellibsЭлектронные библиотекиRussian Digital Libraries Journal1562-5419Казанский (Приволжский) федеральный университет10.26907/1562-5419-2025-28-5-1070-1084ellibs-609Research ArticleСтатьиАвтоматическое извлечение аргументативных отношений из текстов научной коммуникацииAutomatic Extraction of Argumentative Relations from Scientific Communication TextsЗагорулькоЮрий АлексеевичZagorulkoYury Alekseevichzagor@iis.nsk.suСидороваЕлена АнатольевнаSidorovaElena Anatolievnalsidorova@iis.nsk.suАхмадееваИрина РавильевнаAkhmadeevaIrina Ravilevnai.r.akhmadeeva@iis.nsk.suИнститут систем информатики им. А.П. Ершова СО РАНA.P. Ershov Institute of Informatics Systems of Siberian Branch of RAS20251912202528510701084Copyright © Загорулько Ю.А., Сидорова Е.А., Ахмадеева И.Р., 20252025Загорулько Ю.А., Сидорова Е.А., Ахмадеева И.Р.Zagorulko Y.A., Sidorova E.A., Akhmadeeva I.R.Данная работа распространяется под лицензией Creative Commons Attribution 4.0.This work is licensed under a Creative Commons Attribution 4.0 License.https://ellibs.elpub.ru/jour/article/view/609

Сложность задачи извлечения аргументативных структур связана с такими проблемами, как выделение аргументативных сегментов, прогнозирование дальних связей между неконтактными сегментами, обучение на данных, размеченных с низкой степенью согласованности между аннотаторами. В настоящей работе рассмотрен подход к извлечению аргументативных отношений из достаточно больших текстов, относящихся к области научной коммуникации. Проведен сравнительный анализ методов тонкой настройки с использованием предобученной языковой модели типа Longformer, позволяющей учитывать длинные контексты, и двух методов, позволяющих учитывать расхождения аннотаторов в разметке аргументов за счет использования так называемых мягких меток, полученных путем равномерного сглаживания меток и усреднения экспертных оценок. Эксперименты проводились на четырех наборах данных, содержащих положительные и отрицательные примеры пар утверждений (посылка, заключение) и различающихся способами сегментации и средним размером текста. Наилучшие результаты получены на модели с усреднением экспертных оценок. В то же время отмечено, что модель, использующая сглаженные метки, также повышает точность классификаторов, но ухудшает полноту.

The complexity of the problem of extracting argumentative structures is associated with such problems as selecting argumentative segments, predicting long-range connections between non-contact segments, and training on data labeled with a low degree of inter-annotator consistency. In this paper, we consider an approach to extracting argumentative relations from fairly large texts related to scientific communication. A comparative analysis was performed of fine-tuning methods using a pre-trained Longformer-type language model that takes into account long contexts and two methods that take into account annotator discrepancies in argument labeling by using the so-called soft labels obtained by uniformly smoothing labels and averaging expert assessments. The experiments were conducted on four datasets containing positive and negative examples of statement pairs (premise, conclusion) and differing in segmentation methods and average text size. The best results were obtained using the model with averaging expert assessments. At the same time, it is noted that the model using smoothed labels also increases the accuracy of classifiers, but worsens the recall.

анализ аргументацииизвлечение аргументативных отношенийнаучная коммуникацияпроблемы сегментациимягкая меткасглаживание метокязыковая модельargument miningargumentative relation extractionscientific communicationsegmentation problemsoft labellabel smoothinglanguage modelReferencesMeissner J.M., Thumwanit N., Sugawara S., Aizawa A. Embracing Ambiguity: Shifting the Training Target of NLI Models // Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, Online, August 2021. Association for Computational Linguistics: Vol. 2: Short Papers, P. 862–869. https://doi.org/10.18653/v1/2021.acl-short.109Meissner J.M., Thumwanit N., Sugawara S., Aizawa A. Embracing Ambiguity: Shifting the Training Target of NLI Models // Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, Online, August 2021. Association for Computational Linguistics: Vol. 2: Short Papers, P. 862–869. https://doi.org/10.18653/v1/2021.acl-short.109Lukasik M., Bhojanapalli S., Menon A., Kumar S. Does label smoothing mitigate label noise? // Proceedings of the 37th International Conference on Machine Learning, Virtual, 13–18 July 2020, Vol. 119, P. 6448–6458. URL: https://proceedings.mlr.press/v119/lukasik20a.htmlLukasik M., Bhojanapalli S., Menon A., Kumar S. Does label smoothing mitigate label noise? // Proceedings of the 37th International Conference on Machine Learning, Virtual, 13–18 July 2020, Vol. 119, P. 6448–6458. URL: https://proceedings.mlr.press/v119/lukasik20a.htmlHaque S., Bansal A., McMillan C. Label smoothing improves neural source code summarization // 2023 IEEE/ACM 31st International Conference on Program Comprehension (ICPC), Melbourne, Australia, 15–16 May 2023. Institute of Electrical and Electronics Engineers: 2023. P. 101–112. https://doi.org/10.1109/ICPC58990.2023.00025Haque S., Bansal A., McMillan C. Label smoothing improves neural source code summarization // 2023 IEEE/ACM 31st International Conference on Program Comprehension (ICPC), Melbourne, Australia, 15–16 May 2023. Institute of Electrical and Electronics Engineers: 2023. P. 101–112. https://doi.org/10.1109/ICPC58990.2023.00025Szegedy C., Vanhoucke V., Ioffe S., Shlens J., Wojna Z. Rethinking the Inception Architecture for Computer Vision // 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 27–30 June, 2016. Institute of Electrical and Electronics Engineers: P. 2818–2826. https://doi.org/10.1109/CVPR.2016.308Szegedy C., Vanhoucke V., Ioffe S., Shlens J., Wojna Z. Rethinking the Inception Architecture for Computer Vision // 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 27–30 June, 2016. Institute of Electrical and Electronics Engineers: P. 2818–2826. https://doi.org/10.1109/CVPR.2016.308Wang Y., Wang M., Chen Y., Tao S., Guo J., Su C., Zhang M., Yang H. Capture Human Disagreement Distributions by Calibrated Networks for Natural Language Inference // Findings of the Association for Computational Linguistics: ACL 2022, Dublin, Ireland. May 2022. Association for Computational Linguistics: 2022, P. 1524–1535. https://doi.org/10.18653/v1/2022.findings-acl.120Wang Y., Wang M., Chen Y., Tao S., Guo J., Su C., Zhang M., Yang H. Capture Human Disagreement Distributions by Calibrated Networks for Natural Language Inference // Findings of the Association for Computational Linguistics: ACL 2022, Dublin, Ireland. May 2022. Association for Computational Linguistics: 2022, P. 1524–1535. https://doi.org/10.18653/v1/2022.findings-acl.120Timofeeva M.K., Ilina D.V., Kononenko I.S. Argumentative Annotation of the Scientific Internet-Communication Corpus: Genre Analysis and Study of Typical Reasoning Models based on the ArgNetBank Studio Platform // NSU Vestnik. Series: Linguistics and Intercultural Communication. 2024. Vol. 22, No. 1. P. 27–49. (In Russ.) https://doi.org/10.25205/1818-7935-2024-22-1-27-49Timofeeva M.K., Ilina D.V., Kononenko I.S. Argumentative Annotation of the Scientific Internet-Communication Corpus: Genre Analysis and Study of Typical Reasoning Models based on the ArgNetBank Studio Platform // NSU Vestnik. Series: Linguistics and Intercultural Communication. 2024. Vol. 22, No. 1. P. 27–49. (In Russ.) https://doi.org/10.25205/1818-7935-2024-22-1-27-49Shestakov V.K., Kononenko I.S., Sidorova E.A., Zagorulko Yu.A. Assessing Inter-Annotator Agreement on Argumentative Markup // 2024 IEEE International Multi-Conference on Engineering, Computer and Information Sciences (SIBIRCON). IEEE: 2024, P. 309–313. https://doi.org/10.1109/SIBIRCON63777.2024.10758535Shestakov V.K., Kononenko I.S., Sidorova E.A., Zagorulko Yu.A. Assessing Inter-Annotator Agreement on Argumentative Markup // 2024 IEEE International Multi-Conference on Engineering, Computer and Information Sciences (SIBIRCON). IEEE: 2024, P. 309–313. https://doi.org/10.1109/SIBIRCON63777.2024.10758535Akhmadeeva I., Sidorova E., Ilina D. Argument mining in scientific communication: Comparative study // Internet and modern society. Human-computer communication. Cham: Springer Nature Switzerland, 2026. P. 152–166. https://doi.org/10.1007/978-3-031-96177-9_13Akhmadeeva I., Sidorova E., Ilina D. Argument mining in scientific communication: Comparative study // Internet and modern society. Human-computer communication. Cham: Springer Nature Switzerland, 2026. P. 152–166. https://doi.org/10.1007/978-3-031-96177-9_13Beltagy I., Peters M. E., Cohan A. Longformer: The long-document transformer //arXiv preprint arXiv:2004.05150. 2020.Beltagy I., Peters M. E., Cohan A. Longformer: The long-document transformer //arXiv preprint arXiv:2004.05150. 2020.

The authors declare that there are no conflicts of interest present.