ellibsЭлектронные библиотекиRussian Digital Libraries Journal1562-5419Казанский (Приволжский) федеральный университет10.26907/1562-5419-2024-27-5-730-744ellibs-569Research ArticleСтатьиАвтоматическое аннотирование html-документов по стандарту MicrodataAutomatic Annotation of HTML Documents using the Microdata StandardИбрагимовТимур ФердинандовичIbragimovTimur Ferdinandovichi.timur0701@gmail.comФеренецАлександр АндреевичFerenetsAlexander Andreevichist.kazan@gmail.comКазанский (Приволжский) федеральный университетKazan (Volga region) Federal University202428052025275730744Copyright © Ибрагимов Т.Ф., Ференец А.А., 20252025Ибрагимов Т.Ф., Ференец А.А.Ibragimov T.F., Ferenets A.A.Данная работа распространяется под лицензией Creative Commons Attribution 4.0.This work is licensed under a Creative Commons Attribution 4.0 License.https://ellibs.elpub.ru/jour/article/view/569

Описана разработка на основе методов машинного обучения приложения для автоматического аннотирования веб-страниц по стандарту Microdata с возможностью расширения для других стандартов и с внедрением данных в JSX-файлы. Собраны и подготовлены датасеты для обучения моделей Machine Learning (ML). Собраны и проанализированы метрики модели ML.

The development of an application based on machine learning methods for automatic annotation of web pages according to the Microdata standard is described, with the possibility of extension to other standards and injecting data to JSX files. Datasets were collected and prepared for training Machine Learning (ML) models. The ML model metrics were collected and analyzed.

Microdataсемантическая разметкаHTML5поисковая оптимизация (SEO)поисковые системымашинное обучениеschema.orgсемантический вебстандарты разметкиавтоматизация SEOMicrodatasemantic markupHTML5search engine optimization (SEO)search enginesmachine learningschema.orgsemantic webmarkup standardsSEO automationReferencesHTML5 (HyperText Markup Language). URL: https://html.spec.whatwg.org/multipage/introduction.html.HTML5 (HyperText Markup Language). URL: https://html.spec.whatwg.org/multipage/introduction.html.JSX. URL: https://www.typescriptlang.org/docs/handbook/jsx.html.JSX. URL: https://www.typescriptlang.org/docs/handbook/jsx.html.Microdata. URL: https://html.spec.whatwg.org/multipage/microdata.html.Microdata. URL: https://html.spec.whatwg.org/multipage/microdata.html.JSON-LD. URL: https://json-ld.org.JSON-LD. URL: https://json-ld.org.Brinkmann A., Primpeli A., Bizer Ch. The Web Data Commons Schema.org Data Set Series. URL: https://www.uni-mannheim.de/media/Einrichtungen/dws/Files_Research/Web-based_Systems/pub/Brinkmann-etal-TheWDCSchemaorgDataSetSeries-WWW2023.pdf.Brinkmann A., Primpeli A., Bizer Ch. The Web Data Commons Schema.org Data Set Series. URL: https://www.uni-mannheim.de/media/Einrichtungen/dws/Files_Research/Web-based_Systems/pub/Brinkmann-etal-TheWDCSchemaorgDataSetSeries-WWW2023.pdf.Schemas. URL: https://schema.org/docs/schemas.html.Schemas. URL: https://schema.org/docs/schemas.html.RDFa. URL: https://www.w3.org/TR/html-rdfa.RDFa. URL: https://www.w3.org/TR/html-rdfa.Microformats. URL: https://microformats.org.Microformats. URL: https://microformats.org.Local Business Schema Generator – MicroData & JSON-LD. URL: https://microdatagenerator.org/localbusiness-microdata-generator.Local Business Schema Generator – MicroData & JSON-LD. URL: https://microdatagenerator.org/localbusiness-microdata-generator.Structured Data Markup Helper. URL: https://www.google.com/webmasters/markup-helper/u/0.Structured Data Markup Helper. URL: https://www.google.com/webmasters/markup-helper/u/0.Entity SEO Tools. URL: https://inlinks.com/.Entity SEO Tools. URL: https://inlinks.com/.Web-segment. URL: https://github.com/liaocyintl/web-segment.Web-segment. URL: https://github.com/liaocyintl/web-segment.Utiu N., Ionescu V.-S. Learning Web Content Extraction with DOM Features. URL: http://dx.doi.org/10.1109/ICCP.2018.8516632.Utiu N., Ionescu V.-S. Learning Web Content Extraction with DOM Features. URL: http://dx.doi.org/10.1109/ICCP.2018.8516632.Peters M. E., Lecocq D. Content extraction using diverse feature sets // WWW '13 Companion: Proceedings of the 22nd International Conference on World Wide Web, Rio de Janeiro, Brazil, May 13–17, 2013. Association for Computing Machinery, New York, NY, United States: 2013, pages 89–90.Peters M. E., Lecocq D. Content extraction using diverse feature sets // WWW '13 Companion: Proceedings of the 22nd International Conference on World Wide Web, Rio de Janeiro, Brazil, May 13–17, 2013. Association for Computing Machinery, New York, NY, United States: 2013, pages 89–90.Gongqing Wu, Li Li, Xuegang Hu, Xindong Wu Web news extraction via path ratios. URL: https://dl.acm.org/doi/abs/10.1145/2505515.2505558.Gongqing Wu, Li Li, Xuegang Hu, Xindong Wu Web news extraction via path ratios. URL: https://dl.acm.org/doi/abs/10.1145/2505515.2505558.Vadrevu S., Gelgi F., Davulcu H. Semantic partitioning of web pages // Web Information Systems Engineering–WISE 2005: 6th International Conference on Web Information Systems Engineering, New York, NY, USA, November 20–22, 2005. Proceedings 6. – Springer Berlin Heidelberg, 2005. P. 107–118.Vadrevu S., Gelgi F., Davulcu H. Semantic partitioning of web pages // Web Information Systems Engineering–WISE 2005: 6th International Conference on Web Information Systems Engineering, New York, NY, USA, November 20–22, 2005. Proceedings 6. – Springer Berlin Heidelberg, 2005. P. 107–118.Extraction Results from the October 2022 Common Crawl Corpus. URL: https://webdatacommons.org/structureddata/#results-2022-1.Extraction Results from the October 2022 Common Crawl Corpus. URL: https://webdatacommons.org/structureddata/#results-2022-1.Common Crawl September/October 2022 Crawl Archive (CC-MAIN-2022-40). URL: https://data.commoncrawl.org/crawl-data/CC-MAIN-2022-40/index.html.Common Crawl September/October 2022 Crawl Archive (CC-MAIN-2022-40). URL: https://data.commoncrawl.org/crawl-data/CC-MAIN-2022-40/index.html.SPARQL Query Language. URL: https://www.w3.org/TR/sparql11-query.SPARQL Query Language. URL: https://www.w3.org/TR/sparql11-query.BERT: https://arxiv.org/abs/1810.04805.BERT: https://arxiv.org/abs/1810.04805.Babel. URL: https://babeljs.io/.Babel. URL: https://babeljs.io/.TypeScript. URL: https://www.typescriptlang.org/.TypeScript. URL: https://www.typescriptlang.org/.

The authors declare that there are no conflicts of interest present.