Ai2 Publishes Bolmo Research in Nature and Releases New Byte-Level…

At a Glance
| Item | Value |
|---|---|
| Publisher | Allen Institute for AI |
| Published | 2026-10-07 |
| Source type | Primary source (the publisher itself) |
Values determined by this site’s code when the information was collected. Dates are JST.
Overview
Allen Institute for AI (Ai2) has announced that its research paper on “Bolmo," a language model operating at the byte-level, has been published in Nature. Alongside this, they have released new checkpoints on Hugging Face that apply the same conversion technique beyond the original Olmo family to other model families such as Qwen 3 8B and Llama 3 8B.
Bolmo is an open byte-level language model family announced in December 2025. This latest development focuses on providing academic backing for the research and releasing new models with a broader scope of application. Additionally, “Stage 1" checkpoints, which freeze the original global model while training only the byte-level components, have been released as a starting point for researchers looking to extend or experiment with the architecture.
Specifications
The names and parameter scales of each model identified from the materials are as follows:
- Bolmo 1B (Olmo byteified)
- Bolmo 7B (Olmo byteified)
- Bwen 8B (created by byteifying Qwen 3 8B)
- BlamaLlama-B 8B (created by byteifying Llama 3 8B)
- Stage 1 checkpoints (intermediate-stage models where the original global model is frozen and only the byte-level portion is trained)
Details regarding the architecture (whether MoE or Dense) and context length are omitted as they are not mentioned in the materials. A key characteristic of these models is that, instead of the conventional approach of splitting text into a fixed vocabulary of subwords, they directly process sequences of “bytes," which are the fundamental units computers use to represent characters. Ai2 refers to this conversion process as “byteifying" and describes it as a method for converting existing, capable subword models into byte-level models with relatively short additional training.
Performance
While the materials do not include benchmark tables with numerical values or structured scores, Ai2 reports the following qualitative evaluation results:
- Bwen 8B is described as “the most powerful byteified model to date," and is reported to have outperformed Bolmo 7B across Ai2’s aggregate evaluation suite.
- Both Bwen 8B and BlamaLlama-B 8B are reported to have reached performance levels close to matching their source models, Qwen 3 8B and Llama 3 8B respectively.
- Regarding Bolmo 1B and Bolmo 7B, Ai2 positions them as “the first fully open byte-level models capable of competing with strong subword models across a wide range of tasks."
It is important to note that these are evaluation results from Ai2 itself and are not independent verifications by third parties. Because specific numerical scores cannot be confirmed from the published materials, presentation as a comparative table is withheld. Overall, the evaluation emphasizes how closely models created via the byteifying method can approach the performance of original subword models, with the focal point of verification being whether performance approaching the base model can be achieved through the lower-cost means of additional training.
Strengths and Use Cases
It is expected that the Bolmo series and the newly released byteified models will demonstrate their characteristics in domains where conventional subword-based language models struggle.
General language models process text by splitting it into pre-defined chunks of fixed vocabulary (subwords). While this approach has achieved widespread success, it faces challenges such as a lack of flexibility when recognizing spelling, handling unusual strings or rare words, or dealing with text that does not neatly fit into the defined vocabulary chunks. In contrast, byte-level models like Bolmo directly process the raw bytes of computers that represent characters, punctuation, and symbols. This is touted as a strength because it allows them to capture fine-grained text structures more accurately—ranging from the handling of whitespace to diverse writing systems—without being constrained by a fixed vocabulary.
According to the materials, developers and researchers have already begun applying them to specific tasks. A concrete example cited is “Russian-to-English poetry and lyric translation," a fine-tuned version of Bolmo 7B. In domains like poetry and lyrics, where character-level acoustics, subtle notations, and specialized phrasing are emphasized, the byte-level approach is finding practical application.
Uses as a platform for research and model development are also anticipated. Traditional byte-level models required training from scratch to achieve performance comparable to subword models, presenting a major barrier due to high computational costs. By utilizing this byteifying method and the released Stage 1 checkpoints, researchers can experiment with new architectures and expand capabilities without repeating massive training costs. It is reported that they are already being used as a foundation for experiments on transferring capabilities between byte-level models, and for research verifying the trade-off between processing efficiency and fine-grained character understanding in hierarchical byte architectures.
Furthermore, because byte representations are the foundational expression of all digital data—not limited to text—applications toward multimodal foundations that comprehensively handle other modalities such as images and audio are also envisioned for the future.
How to Get It
Checkpoints for each of the newly announced models are available on Hugging Face.
The main deliverables published are:
- Checkpoints for Bolmo 1B and Bolmo 7B
- Checkpoint for Bwen 8B, converted from Qwen 3 8B
- Checkpoint for BlamaLlama-B 8B, converted from Llama 3 8B
- Stage 1 checkpoints, where byte-level components were trained while freezing the original global model
Depending on their use case, researchers and developers can utilize not only the final models but also the Stage 1 checkpoints as a starting point for architectural experimentation and extension. Because specific details regarding distribution formats, supported inference engines, and terms of use agreement requirements are not specified in the material, please check the respective repositories on Hugging Face.

