{"id":35446,"date":"2026-09-03T08:42:28","date_gmt":"2026-09-03T06:42:28","guid":{"rendered":"https:\/\/opi.org.pl\/ai-lab-opi-udostepnia-polish-modernbert-nowa-rodzine-modeli-jezykowych-w-ramach-projektu-gaia-ai-factory\/"},"modified":"2026-10-21T09:01:35","modified_gmt":"2026-10-21T07:01:35","slug":"ai-lab-opi-udostepnia-polish-modernbert-nowa-rodzine-modeli-jezykowych-w-ramach-projektu-gaia-ai-factory","status":"publish","type":"post","link":"https:\/\/opi.org.pl\/en\/ai-lab-opi-udostepnia-polish-modernbert-nowa-rodzine-modeli-jezykowych-w-ramach-projektu-gaia-ai-factory\/","title":{"rendered":"OPI PIB\u2019s AI LAB releases Polish ModernBERT, a new family of language models as part of the Gaia AI Factory project"},"content":{"rendered":"\n

Researchers at AI LAB have recently released the PolDense<\/a> and EuroDense<\/a> retrievers, as well as the PUMA<\/a> benchmark. Their newest addition is a family of next-generation representation models developed as part of the Gaia AI Factory<\/a> project.<\/p>\n\n\n\n

Polish ModernBERT is a family of encoder-only models designed for a wide range of natural language processing tasks, such as text classification, proper name recognition, and regression. It also serves as the foundation for retrieval-focused encoders. With support for context windows of up to 8,192 tokens, the models are capable of processing substantially longer texts than many previously available Polish language models.<\/p>\n\n\n\n

The model was developed through a comprehensive process that ranged from cleaning and preparing training datasets to devising a novel pre-training framework procedure. Researchers trained the model using a dedicated Polish corpus consisting of 44.5 billion tokens.<\/p>\n\n\n\n

So far, four Base and Large model variants have been made available, each offered with support for either 512-token or 8,000-token context windows. Their performance was measured using 30 Polish benchmark natural language processing tasks. The researchers also introduced LongContext, a new benchmark for assessing the analysis of long documents in the Polish language.<\/p>\n\n\n\n

With a score of 85.11 points, Large-8K emerged as the leading Polish encoder model assessed in the study. Base-8K ranked as the best-performing 8K model in its size class, achieving similar effectiveness despite relying on substantially fewer parameters.<\/p>\n\n\n\n

The results obtained on an NVIDIA H100 indicated that Polish ModernBERT exhibited higher computational efficiency than the compared RoBERTa models. The Base variants can operate on conventional CPU hardware and are well suited for use in public administration, science, and business.<\/p>\n\n\n\n

Polish ModernBERT was developed as part of the Gaia AI Factory project, which is co-funded by the EuroHPC Joint Undertaking through the European Union\u2019s Digital Europe programme, as well as by the participating state, Poland. The computational tasks were completed with the aid of the PLGrid infrastructure at the Cyfronet Academic Computer Centre at AGH University of Science and Technology.<\/p>\n\n\n\n

The models are available to the public free of charge on the Hugging Face platform at<\/strong>:<\/strong> https:\/\/huggingface.co\/collections\/OPI-PIB\/pl-modernbert<\/a><\/p>\n\n\n\n

The related research paper by the AI Lab team<\/strong>:<\/strong> https:\/\/arxiv.org\/abs\/2609.01379<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

The AI LAB team at OPI PIB has released Polish ModernBERT, a family of language models designed to analyse short texts and extensive documents in Polish. The models are available to the public on the Hugging Face platform<\/a>, and detailed information pertaining to their architecture, training, and evaluation results is presented in a related research paper<\/a>.<\/p>\n","protected":false},"author":32,"featured_media":35448,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":"","_links_to":"","_links_to_target":""},"categories":[411],"tags":[1099,1120,876],"class_list":["post-35446","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-news-en","tag-ailab","tag-gaiaaifactory","tag-languagemodels-2"],"_links":{"self":[{"href":"https:\/\/opi.org.pl\/en\/wp-json\/wp\/v2\/posts\/35446","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/opi.org.pl\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/opi.org.pl\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/opi.org.pl\/en\/wp-json\/wp\/v2\/users\/32"}],"replies":[{"embeddable":true,"href":"https:\/\/opi.org.pl\/en\/wp-json\/wp\/v2\/comments?post=35446"}],"version-history":[{"count":1,"href":"https:\/\/opi.org.pl\/en\/wp-json\/wp\/v2\/posts\/35446\/revisions"}],"predecessor-version":[{"id":35450,"href":"https:\/\/opi.org.pl\/en\/wp-json\/wp\/v2\/posts\/35446\/revisions\/35450"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/opi.org.pl\/en\/wp-json\/wp\/v2\/media\/35448"}],"wp:attachment":[{"href":"https:\/\/opi.org.pl\/en\/wp-json\/wp\/v2\/media?parent=35446"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/opi.org.pl\/en\/wp-json\/wp\/v2\/categories?post=35446"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/opi.org.pl\/en\/wp-json\/wp\/v2\/tags?post=35446"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}