inclusionAI Releases SingProbe Models for Real-Time LLM Guardrails

At a Glance
| Item | Value |
|---|---|
| Repository | inclusionAI/Ling-3.1-flash-1019-singprobe |
| Publisher guide | inclusionAI (Ant Group): models and licenses |
| Published | 2026-11-19 |
| License | mit |
| Formats | safetensors |
| Paper | arXiv:2608.30703 |
| Source type | Primary source (the publisher itself) |
Values determined by this site’s code when the information was collected. Dates are JST.
Overview
inclusionAI has released two lightweight models, inclusionAI/Ling-3.1-flash-1019-singprobe and inclusionAI/DeepSeek-V4.1-Flash-1017-singprobe, designed to perform real-time guardrail evaluations by referencing the internal states of LLMs during generation.
These models employ “SingProbe" technology, which directly utilizes the hidden states of the base model performing autoregressive decoding. This allows them to score inappropriate expressions and hallucination risks in real time, concurrently with token generation, without needing to invoke a separate independent model each time.
According to the abstract of the paper (SingProbe Technical Report), SingProbe is developed to avoid the redundant processing overhead of traditional standalone guardrail models. By integrating with inference engines (SGLang and vLLM), it aims to continuously output query intent, response safety, and hallucination risks with minimal execution overhead (less than 0.5% in their implementation).
Specifications
- Parameter count: 8.13M (probe portion)
- Architecture: SingProbeAttnModel (8 attention heads, head dimension 64)
- Context length: 2,048 tokens (probe window size)
Performance
The structure and evaluation data for Ling-3.1-flash-1019-singprobe presented in the release model card are as follows:
Model configuration information:
| Base model | Probe parameters | Tapped layers (zero-based) | Outputs |
| — | —: | —: | — |
| Ling-3.1-flash-1019 | 8.13M | [23, 47, 71] | 8 query intents + response unsafety + hallucination |
Benchmark evaluation comparison:
| Task | Metric | Ling-3.1-flash-1019-singprobe | Reference baseline |
| — | — | —: | — |
| Query intent classification (6 benchmarks) | F1 | 0.8687 | YuFeng-XGuard-Reason-8B (0.8714) |
| Response safety classification (8 benchmarks) | F1 | 0.8707 | Qwen3Guard-Gen-8B-strict (0.8604) |
| Streaming safety (3 benchmarks) | Response AUC / token AUC | 0.9844 / 0.9411 | Qwen3Guard-Stream-8B-strict (0.9640 / 0.8893) |
| Hallucination detection (6 benchmarks) | AUC | 0.8041 | DRIFT (0.8000) |
Deployment characteristic evaluation:
| Deployment characteristic | Result |
| — | — |
| Benign-response false-positive rate | 0.008% averaged over 5 datasets at the evaluation’s pre-calibration response threshold of 0.983085 |
| Calibrated release thresholds | 0.5 for response unsafety and 0.5 for hallucination |
From the measurement results conducted independently by the publishers, trends for this model in various fields can be confirmed. In streaming safety evaluation, it recorded a Response AUC of 0.9844 and a token AUC of 0.9411, showing extremely high detection accuracy that clearly surpasses the comparison target Qwen3Guard-Stream-8B-strict (0.9640 / 0.8893). In response safety classification, it also achieved an F1 score of 0.8707, exceeding the 0.8604 of Qwen3Guard-Gen-8B-strict. Furthermore, in hallucination detection, it recorded an AUC of 0.8041, outperforming DRIFT’s 0.8000. On the other hand, the F1 score in query intent classification was 0.8687, which is a slightly lower result compared to the reference baseline YuFeng-XGuard-Reason-8B at 0.8714. In actual operation, the benign-response false-positive rate is kept as low as 0.008% averaged over 5 datasets, possessing a good balance that can detect risks in real time without hindering normal inputs and outputs.
Strengths and Use Cases
This model group excels at streaming guardrails and hallucination detection functions that monitor safety in real time, concurrently with LLM generation processing (autoregressive decoding).
By referencing three layers from the hidden states of the base models Ling-3.1-flash-1019 and DeepSeek-V4.1-Flash-1017, and without separately calling an external independent guardrail model, it scores the following elements in real time for each generated token:
- Query intent classification: Evaluates the intent of the input query across 8 categories (
label_0tolabel_7). - Response unsafety detection: Detects inappropriate expressions and safety risks (
label_8) included in the tokens being generated. - Hallucination risk detection: Monitors the risk of factually incorrect outputs or logical breakdowns (
label_9) in the model’s generated content.
Assumed primary use cases include online monitoring in chatbots and conversational systems that stream responses to users in real time. Because it directly utilizes the internal states of the base model without routing through an independent guardrail model, risks can be grasped at high speed while suppressing the overhead of redundant processing.
Furthermore, since it can detect inappropriate generations while keeping false positives on benign contexts low, it is also suitable for applications such as immediately halting generation when problematic output is detected, or incorporating into safety control mechanisms like constrained decoding and specific medical risk interventions (SingProbe-Med).
Hardware Requirements
Estimated requirements (calculated by Local Model Watch) — 8M parameters
| Your VRAM | Quantization | File size | Est. memory needed |
|---|---|---|---|
| 4GB (laptop iGPU / phone class) | BF16 | 0.0GB | 0.0GB |
Inference engine support (architecture name matched against each project’s own model registry in its source code, checked 2026-11-20): llama.cpp: not registered, vLLM: not registered, MLX (mlx-lm): not registered. “Not registered" means the name is absent from that registry today, not that the model cannot run.
Memory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference. What the quantization names mean: glossary.
Can You Run It Locally?
The publisher distributes this model as safetensors.
License — mit (Commercial use allowed): Permits commercial use, modification and redistribution, provided the copyright notice and license text are retained.
Compiled by this site’s code from the published formats and the license field. License summaries are not legal advice — check the publisher’s original terms before relying on them.
How to Get It
The model weights are distributed in safetensors format and published on Hugging Face. The MIT license applies, and going through a agreement procedure (gated) is not required to obtain them.
Integration with inference engines such as SGLang and vLLM is assumed for these models. The model card introduces server launch procedures using a specific branch of SGLang (branch 261029).
An example command to launch inclusionAI/Ling-3.1-flash-1019-singprobe is as follows:
python3 -m sglang.launch_server \
--model-path /home/admin/Ling-3.1-flash-1019 \
--probe-ckpt /yijian3/536716/SingGuardStream-Final/release/Ling-3.1-flash-1019-singprobe \
--trust-remote-code --tp 8 --ep-size 8 \
--context-length 16384 --chunked-prefill-size 1024 \
--max-prefill-tokens 8192 --max-total-tokens 32768 \
--max-running-requests 16 --mem-fraction-static 0.85 \
--cuda-graph-backend-prefill disabled --cuda-graph-backend-decode disabled \
--port 30000
When using inclusionAI/DeepSeek-V4.1-Flash-1017-singprobe, you can also launch the server with a similar configuration by replacing the paths for --model-path and --probe-ckpt with the respective model’s directory.
After the server is running, you can obtain per-token scores by referencing the meta_info.token_probe_probs field included in the response from the /generate endpoint. The keys label_0 to label_7 correspond to query intents, label_8 to response unsafety, and label_9 to the probability score of hallucination risk.
Related Articles
- ZoomGain-4B: 4GB+ VRAM, GGUF Builds
- Ling-3.0-flash-Sante-fp8 Text Generation Model: ~144GB Memory
- Ling-3.1-flash-1019 Text Generation Model: ~641GB Memory, GGUF Builds
- Ling-3.0-tiny-UBTree Draft Model for Speculative Decoding: 4GB+ VRAM
What to Read Next
- Find models by VRAM (This model runs from the 4GB tier) → Other models that run on a 8GB GPU
- Formats this model is available in → Safetensors format guide and models
- Learn about the publisher → inclusionAI (Ant Group): models, licenses and articles

