Yandex releases AliceAI-Foundation-80B-A3B-Base, an Apache 2.0 base MoE for Russian and English

Yandex published AliceAI-Foundation-80B-A3B-Base, a base language model with a hybrid architecture and MoE layers. The card says it has 80B parameters with 3B active per token, 262,144 tokens of context, and was trained entirely from scratch. The layout repeats three KDA plus MoE layers then one gated attention plus MoE layer, for 48 layers, with 512 experts (10 routed and one shared). The languages are Russian and English.

The card says the team rebuilt the training corpus, picked the architecture and hyperparameters, and validated key choices in separate 2-trillion-token runs. It claims results on par with larger open models on maths, coding and reasoning, and particular strength on Russian factual knowledge. Alongside the weights, Yandex released two Russian-language factual benchmarks, WikiWebFacts and HardMultiQA, with their evaluation protocols. The claims are the maker's own.

The licence is Apache 2.0. This is the pre-trained stage only, so it will not follow instructions out of the box.

The card gives no release date; the date used is the Hugging Face repo creation date.

A base model and its benchmarks, both released. The benchmarks may outlast the model.

Sources

• Model card on Hugging Face
• Hugging Face repo metadata (creation date, licence, size)

Editor's first pass (AI-assisted)

• Real 4: Apache 2.0, ungated, custom_code tag; this is a pre-trained base, not a chat model.
• Useful 3: A foundation for fine-tuning; strongest on Russian factual knowledge, per the card.
• Notable 3.5: 395 likes; ships two new Russian-language benchmarks.
• Novel 3.5: A hybrid of KDA and gated attention layers over a 512-expert MoE, trained from scratch.