# ModernBERT The[ original BERT paper](https://arxiv.org/abs/1810.04805) came out in 2018, around 7 years ago at time of writing. However, it is still referred to and used as a strong baseline in a number of NLP tasks. [ModernBERT](https://huggingface.co/blog/modernbert) was created by HuggingFace. ModernBERT is a drop-in replacement for use in problems where BERT may have previously been used and like the original, has a base and a large variant. ModernBERT also outperforms [DEBERTA-v3-base](https://huggingface.co/microsoft/deberta-v3-base) which has been a favourite of NLP practitioners for a few years thanks to its few-shot and zero-shot capabilities. Comparison Table
**BERT Base**
**ModernBERT Base**
**BERT Large**
**ModernBERT Large**
**\# Params**
110M
149M
340M
395M
**Context Size**
512
8192
512
8192
**BEIR**
38.9
41.6
38.9
44.0
**[MLDR](https://infiniflow.org/blog/multi-way-retrieval-evaluations-on-infinity-database)OOD **
23.9
27.4
23.3
34.3
**MLDRID**
32.2
44.0
31.7
48.6
**BEIR (ColBERT)**
49.0
51.3
49.5
52.4
**MLDROOD (ColBERT)**
28.1
80.2
28.5
80.4
**GLUE**
84.7
88.5
85.2
90.4
**CSN**
41.2
56.4
41.6
59.5
**SQA**
59.5
73.6
60.8
83.9
OOD=out of domain ID=In Domain [mmBERT](https://arxiv.org/abs/2509.06888) is a modern multi-lingual encoder-only model based on ModernBERT.