SmolDataEnvs: RL Tasks for Small Model Optimization

September 26, 2026September 27, 2026

At a Glance

Values determined by this site’s code at collection time. Dates are JST.

Overview

FineEnvs/SmolDataEnvs is a dataset containing over 5.5K reinforcement learning (RL) tasks designed for hill-climbing optimization of small models in the code and data science domains. The dataset is provided in a plain format that requires no execution environment (runtime) or specific framework, allowing it to be used immediately simply by loading it. Each row includes an actual tabular dataset, a corresponding question, and a gold answer that can be deterministically verified with the included grader, making it easy to test and evaluate prompts even with small models in a local environment.

Claims and Evidence

Presenter FineEnvs claims that this dataset can provide extremely reliable, deterministic signals for the reinforcement learning and evaluation of small models. To support this claim, the following features and data composition are presented.

1. Tasks Subject to Rigorous Verification Processes

All included tasks are built on top of the jupyter-agent dataset, which was extracted from actual data science notebooks spanning 471 Kaggle datasets. Every extracted question-answer pair has passed verification confirming that a strong agent model can actually solve the task in a live sandbox environment and reproduce the gold answer under deterministic grading. Since all ambiguous or unverifiable tasks have been eliminated, every included task is guaranteed to be “solvable" and “objectively gradable."

2. LLM-Independent Deterministic Evaluation System

No LLM-based judgment (LLM-as-a-judge) is involved in the evaluation (reward signal) pathway of this dataset. The included grader.py performs step-by-step rule-based checks such as exact matching, numerical comparison taking tolerances (atol/rtol) into account, list or percentage normalization, and symbolic equivalence verification. This prevents evaluation metric drift caused by changes in the evaluation model and allows model performance to be measured against a constant standard at all times.

3. Intentionally Hardened Evaluation Splits

The test and evaluation splits are designed to be more difficult than the training split. Specifically, while the train split consists of 29% easy and 14% hard tasks, the held-out evaluation splits (test and eval) consist of 11–13% easy and 38–40% hard tasks. This makes it possible to measure the generalization performance of models more rigorously. The details of the data split are as follows:

4. Data Structure Including Rich Metadata

Each row of the dataset comprehensively covers the information required to run and evaluate tasks.

5. Deployment of Family Repositories According to Use Case

Rather than simply being provided as a basic dataset, SmolDataEnvs is composed of multiple repositories tailored to specific purposes, such as trajectory data for supervised fine-tuning (SFT) and the Harbor suite executable as a sandbox environment.

Hardware Requirements

Regarding the use of this dataset and evaluation code, the prerequisites indicated in the materials are as follows:

• Target Models: Primarily targets reinforcement learning and evaluation for small models (the description mentions an example of a 2B model).
• License: Released under the MIT license.
• Data Format: Provided in Parquet format, handling text and tabular data.
• Required Software and Libraries: Based on a Python environment, requiring libraries such as datasets and huggingface_hub. Tag information lists pandas, polars, and mlcroissant.
• Execution Environment Constraints: The dataset itself requires no runtime or specific execution framework (“no runtime, no framework required") and can be loaded as independent data row by row. Note that the input tabular data files are distributed via the Hugging Face Bucket feature.

How to Reproduce Locally

Acquiring the dataset, downloading task tabular data, running the scoring script, and referencing the training script can be executed locally based on the published code.

1. Loading the Dataset

Using the datasets library, you can load specifying the split as follows:

from datasets import load_dataset

ds = load_dataset("FineEnvs/SmolDataEnvs", split="test")
row = ds[0]
print(row["question"], "→", row["answer"], f"({row['reward_mode']})")

2. Acquiring Task Data Files

Since the input tabular files used in each task are stored on Hugging Face Buckets, they are retrieved using the huggingface_hub Bucket API.

from huggingface_hub import list_bucket_tree, download_bucket_files

prefix = row["bucket_prefix"].rstrip("/") + "/"
items = [i for i in list_bucket_tree(row["hf_bucket"], prefix=prefix, recursive=True)
         if getattr(i, "type", None) == "file"]
download_bucket_files(row["hf_bucket"],
                      files=[(i.path, "input/" + i.path.split("/")[-1]) for i in items])

3. Grading Inference Results with the Grader

Download grader.py included in the repository, dynamically import it, and evaluate model outputs by matching them against the gold values. Tolerances (atol/rtol) and comparison modes (reward_mode) are specified to be passed from the metadata within the dataset.

from huggingface_hub import hf_hub_download
import importlib.util, sys

path = hf_hub_download("FineEnvs/SmolDataEnvs", "grader.py", repo_type="dataset")
spec = importlib.util.spec_from_file_location("grader", path)
grader = importlib.util.module_from_spec(spec)
sys.modules["grader"] = grader          # Required for internal dataclass behavior
spec.loader.exec_module(grader)

r = grader.grade(row["answer"], my_prediction, reward_mode=row["reward_mode"],
                 abs_tol=row["atol"], rel_tol=row["rtol"])
print(r.reward, r.method)   # 1.0 exact | 0.0 miss

4. Training Environment and Peripheral Repositories

Training scripts and notebooks are published as single-file scripts on GitHub at FineEnvs/04-smoldataenvs, formatted to be passed and executed on HF Jobs or similar platforms. Additionally, as purpose-specific derivative repositories, TRL-compatible agent trajectory data SmolDataEnvs-sft and Harbor suites for sandbox environments (SmolDataEnvs-harbor-train, SmolDataEnvs-harbor-test, SmolDataEnvs-harbor-eval) are available.

What the Materials Do Not Cover

• Details of models used in verification experiments: Regarding the 2B model mentioned in the explanation as “A 2B model on these tasks," the specific base model name or model family is not explicitly stated.
• Specific numerical figures of experimental results: Specific numerical data such as achieved scores or performance differences for the two training runs executed with shuffled order and difficulty-curriculum order (“Two runs over the same 5,000 tasks") are not documented in the materials.
• Names of agent models used for task verification: The specific model names or system prompt configurations of the “strong agent models" used to verify task answerability during data extraction are not disclosed.

Related Articles

• XiaomiMiMo Releases Agentic RL Training Environment and Dataset

Sources

• FineEnvs/SmolDataEnvs (Hugging Face Datasets)
• FineEnvs/04-smoldataenvs (GitHub)
• FineEnvs/SmolDataEnvs-sft (Hugging Face Datasets)
• FineEnvs/SmolDataEnvs-harbor-train (Hugging Face Datasets)
• FineEnvs/SmolDataEnvs-harbor-test (Hugging Face Datasets)
• FineEnvs/SmolDataEnvs-harbor-eval (Hugging Face Datasets)
• jupyter-agent dataset (Hugging Face Datasets)