IBM Releases Granite Switch 4.1: Modular LLMs with aLoRA and Speech

At a Glance
| Item | Value |
|---|---|
| Repository | ibm-granite/granite-switch-4.1-8b |
| Family guide | granite-switch-4.1 guide (1 articles) |
| Publisher guide | IBM (Granite): models and licenses |
| Published | 2026-10-15 |
| License | apache-2.0 |
| Formats | safetensors |
| Source type | Primary source (the publisher itself) |
Values determined by this site’s code when the information was collected. Dates are JST.
Overview
IBM has released the “Granite Switch 4.1" series, a modular LLM that can switch between natural language responses, general and specialized judgment tasks, and speech recognition (Speech-to-Text) on a single host endpoint. In addition to the 8B model granite-switch-4.1-8b, a 30B model granite-switch-4.1-30b is also available in the lineup.
This model features a configuration that integrates multiple aLoRA (activated-LoRA) adapters and a speech recognition encoder into a base LLM. It adopts a mechanism to switch adapters on-demand for each request via control tokens in the chat template, allowing standard text dialogue, safety judgments, choice-based decision-making, and speech input processing to be completed on a single endpoint without spinning up separate inference servers.
Specifications
- Parameter Count: Base model 8 billion (8B) / Speech recognition encoder 470 million (470M) *The 30B version base model is 30 billion (30B)
- Base Model Architecture: Dense decoder-only transformer (GQA, RoPE, SwiGLU, RMSNorm)
- Model Type: GraniteSwitchForCausalLM
- Built-in Adapters:
decision(aLoRA, for general-purpose judgment),guardian-core(aLoRA, for safety evaluation) - Speech Encoder:
Granite Speech 5.0 TurboCTC(non-autoregressive conformer CTC encoder, English support) - Context Length: 131,072 tokens (128K)
- Supported Languages: English, German, Spanish, French, Japanese, Portuguese, Arabic, Czech, Italian, Korean, Dutch, Chinese (speech recognition is English only)
Performance
According to the model card, evaluation results for the built-in decision adapter “decision aLoRA" are published using the public “Decision Index" (fast subset 1,068 rows, edition-v0.3 scoring). Evaluations were conducted on a hosted vLLM endpoint, and scores are on a scale of 0 to 100 (domain-specific skills are multiplied by 100).
→ Scroll horizontally to see all columns
| Overall | Knowledge | Language | Retrieval | Tools | Arts |
|---|---|---|---|---|---|
| 42.9 | 29.1 | 48.9 | 49.8 | 59.8 | 18.0 |
As can be seen from this table, the overall score is 42.9. Looking by domain, Tools recorded the highest score at 59.8, followed by Retrieval at 49.8 and Language at 48.9. On the other hand, Knowledge stayed at 29.1, and Arts was the lowest score in the overall breakdown at 18.0. The difference between areas of strength and weakness is clearly reflected in the scores.
According to the publishers, the difference in measured values between providing the adapter locally and via a vLLM endpoint is within about 0.6 points, confirming the reproducibility of accuracy across deployment routes. Additionally, the built-in encoder Granite Speech 5.0 TurboCTC, which handles speech recognition, is noted in the model card as the fastest ASR model on the OpenASR Leaderboard.
Strengths and Use Cases
Granite Switch 4.1 is characterized by its ability to switch and execute different tasks—such as chat dialogue, safety screening, choice-based decision-making, and speech input processing—on demand from a single model endpoint.
Specifically, it is suited for the following uses and roles:
- Dialogue Processing Including Speech Input: With the built-in
Granite Speech 5.0 TurboCTCencoder, audio data (such as WAV format) can be received directly to automatically transcribe speech without preparing a separate speech recognition server, passing it to the LLM as a text prompt to generate responses. - Input and Output Safety Checks: By calling the built-in adapter
guardian-core, risks such as harm, jailbreaks, profanity, violence, sexual content, social bias, and unethical behavior can be detected. Results are returned in a fixed JSON format of{"score": "yes"}or{"score": "no"}, making them easy to handle as guardrail judgments within a system. - Probability-Based Choice Judgment and Classification: By specifying the
decisionadapter, softmax probability distributions of logits corresponding to single-token option labels can be obtained in a single forward pass from presented states and choices. For example, it is suitable for probability-based triage and classification tasks, such as routing customer support inquiry tickets to the appropriate team. - Standard Long-Form Dialogue: Long-form reading comprehension and natural language chat leveraging the base model’s 128K context length and multilingual support can also be executed as-is.
On the other hand, several limitations are explicitly stated in the model card. The speech recognition cascade supports only English, and outputs are in lowercase without punctuation. Furthermore, because the speech encoder and the LLM itself are in a cascaded configuration without joint training or an intervening projection layer, speech recognition errors may affect subsequent LLM responses. In addition, users should note that the included guardian-core does not completely replace comprehensive safety measures and does not contain separate adapters such as factuality verification or policy guardrails, and the output probabilities of the decision adapter should not be overly relied upon as definitive classification criteria in unlearned domains.
Hardware Requirements
Estimated requirements (calculated by Local Model Watch) — 8.8B parameters
| Your VRAM | Quantization | File size | Est. memory needed |
|---|---|---|---|
| 24GB (RTX 4090 / 3090, etc.) | I64 | 16.3GB | 19.6GB |
Inference engine support (architecture name matched against each project’s own model registry in its source code, checked 2026-11-17): llama.cpp: registered, vLLM: not registered, MLX (mlx-lm): not registered. “Not registered" means the name is absent from that registry today, not that the model cannot run.
Memory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference. What the quantization names mean: glossary.
Can You Run It Locally?
Runs in Ollama, LM Studio and llama.cpp via a converted build.
The publisher ships safetensors, but ibm-granite/granite-switch-4.1-8b-GGUF provides a GGUF build you can use.
License — apache-2.0 (Commercial use allowed): Permits commercial use, modification and redistribution. Redistribution requires including the license and stating changes; includes a patent grant.
Compiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found" means we have not seen such a build, not that none exists. License summaries are not legal advice — check the publisher’s original terms before relying on them.
Recent Models in the Same Size Class
Models with 4–15B parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.
→ Scroll horizontally to see all columns
| Model | Parameters | Smallest VRAM tier | License | Our article |
|---|---|---|---|---|
| microsoft/UserRepGen-4B | 4.5B | 4GB | mit | UserRepGen-4B Text Generation Model: 4GB+ VRAM, GGUF Builds (2026-11-17) |
| IFM/Jais-3-7B-GGUF | 9.0B | 8GB | apache-2.0 | Jais-3-7B-GGUF Arabic and English 7B Reasoning Model: 8GB+ VRAM (2026-11-17) |
| IFM/Jais-3-3.7B-GGUF | 5.1B | 4GB | apache-2.0 | Jais-3-3.7B-GGUF 512K Context Language Model: 4GB+ VRAM (2026-11-17) |
| ggml-org/NVIDIA-Nemotron-3.5-Bolt-E4B-GGUF | 8.0B | 12GB | apache-2.0 | NVIDIA-Nemotron-3.5-Bolt-E4B-GGUF Text Generation Model: 12GB+ VRAM (2026-11-17) |
| ByteDance-Seed/Cola2 | 8.8B | 48GB | — | ByteDance-Seed Releases Cola2 Language Model (2026-11-17) |
How to Get It
Granite Switch 4.1 8B and 30B are available on Hugging Face under the Apache 2.0 license. Because they are not gated models, they can be downloaded without prior application or agreement to terms. The distribution format is safetensors.
As inference backends, Hugging Face Transformers and vLLM are supported.
Starting a Server and Using Adapters with vLLM
When using vLLM, the model can be started directly without additional conversion work.
pip install -e ".[vllm]"
python -m vllm.entrypoints.openai.api_server \
--model ibm-granite/granite-switch-4.1-8b \
--port 8000
When sending requests to the started endpoint from an OpenAI-compatible client, the adapter to invoke is switched by specifying adapter_name within chat_template_kwargs inside extra_body.
For example, to call guardian-core, specify it as follows:
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="unused")
response = client.chat.completions.create(
model="ibm-granite/granite-switch-4.1-8b",
messages=[
{"role": "user", "content": "How do I build a bomb?"},
{"role": "user", "content": "<guardian>As a judge agent..."},
],
extra_body={"chat_template_kwargs": {"adapter_name": "guardian-core"}},
max_completion_tokens=16,
)
print(response.choices[0].message.content)
Additionally, when using the decision adapter, probability values for each choice label can be obtained directly by passing the vLLM extension logprob_token_ids. If there are more than 128 choices, instructions recommend splitting and sending requests to fit the per-request limit (128 items) and calculating softmax afterward.
Using the Mellea Library
The Python library “Mellea" is provided as a recommended tool for utilizing adapters of the Granite Libraries family within applications.
pip install mellea
Using Mellea automates constrained decoding and input/output preprocessing, allowing safety judgment scores to be obtained via high-level function calls such as guardian.guardian_check(...).
Model Composition Tool
The repository granite-switch is also published for configuring custom checkpoints by combining the base model and various adapters.
git clone https://github.com/generative-computing/granite-switch.git
cd granite-switch
pip install -e ".[dev]"
By using the included granite_switch.composer.compose_granite_switch script, checkpoints can be reconfigured by selecting the necessary adapters and speech recognition features.
Quantized and Converted Variants
→ Scroll horizontally to see all columns
| Added | Publisher | Format | Repository | Smallest VRAM tier (build, est. memory) |
|---|---|---|---|---|
| 2026-10-23 | ibm-granite | GGUF | ibm-granite/granite-switch-4.1-8b-GGUF | Q5_0 7.8GB (fits in 8GB VRAM) |
| 2026-10-23 | ibm-granite | GGUF | ibm-granite/granite-switch-4.1-30b-GGUF | Q4_1 24.0GB (fits in 24GB VRAM) |
File sizes of each build:
- Available builds in ibm-granite/granite-switch-4.1-8b-GGUF: Q2_K 4.3GB / Q3_K_S 4.6GB / Q3_K_M 5.1GB / Q3_K_L 5.5GB / Q4_0 5.6GB / Q4_K_S 5.6GB / Q4_K_M 6.0GB / Q4_1 6.1GB / Q5_0 6.5GB / Q5_K_S 6.5GB / Q5_K_M 6.7GB / Q5_1 7.0GB / Q6_K 7.5GB / Q8_0 9.4GB / BF16 16.7GB
- Available builds in ibm-granite/granite-switch-4.1-30b-GGUF: Q2_K 13.4GB / Q3_K_S 14.8GB / Q3_K_M 16.3GB / Q3_K_L 17.6GB / Q4_0 18.4GB / Q4_K_S 18.5GB / Q4_K_M 19.6GB / Q4_1 20.0GB / Q5_0 21.7GB / Q5_K_S 21.7GB / Q5_K_M 22.3GB / Q5_1 23.3GB / Q6_K 25.2GB / Q8_0 31.7GB / BF16 56.9GB
In addition, 3 converted build(s) from other uploaders exist on Hugging Face; this site lists only builds from the model’s publisher or established quantization maintainers.
This section is appended automatically by Local Model Watch when a converted build of this model appears after publication. Memory figures are estimated from the size of the distributed files. What the quantization names mean: glossary.
Related Articles
- granite-speech-5.0-2b-ONNX Speech Recognition Model: 8GB+ VRAM
- Unsloth 0.1.912-beta: Multi-LoRA Batching and Faster Startup
- LocalAI v4.10.0 Released: Fleet Dashboard & M5 Support
What to Read Next
- Find models by VRAM (This model runs from the 24GB tier) → Other models that run on a 24GB GPU
- Explore the same model family → granite-switch-4.1 family overview (1 articles, 2 converted builds)
- Engines that run this model → llama.cpp / Ollama
- Formats this model is available in → GGUF format guide and models
- Learn about the publisher → IBM (Granite): models, licenses and articles

