{"id":4616,"date":"2026-09-26T18:13:08","date_gmt":"2026-09-26T09:13:08","guid":{"rendered":"https:\/\/localmodelwatch.tsuchitsuchi.com\/2026\/09\/26\/smoldataenvs-rl-tasks-small-model-optimization\/"},"modified":"2026-09-27T22:09:41","modified_gmt":"2026-09-27T13:09:41","slug":"smoldataenvs-rl-tasks-small-model-optimization","status":"publish","type":"post","link":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/09\/26\/smoldataenvs-rl-tasks-small-model-optimization\/","title":{"rendered":"SmolDataEnvs: RL Tasks for Small Model Optimization"},"content":{"rendered":"

<\/p>\n

At a Glance<\/h2>\n
\n\n\n\n\n\n\n\n
Item<\/th>\nValue<\/th>\n<\/tr>\n<\/thead>\n
Repository<\/td>\nFineEnvs\/SmolDataEnvs<\/a><\/td>\n<\/tr>\n
Published<\/td>\n2026-09-24<\/td>\n<\/tr>\n
Source type<\/td>\nPrimary source (the publisher itself)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n

Values determined by this site’s code at collection time. Dates are JST.<\/em><\/p>\n

<\/p>\n

Overview<\/h2>\n

FineEnvs\/SmolDataEnvs<\/code> is a dataset containing over 5.5K reinforcement learning (RL) tasks designed for hill-climbing optimization of small models in the code and data science domains. The dataset is provided in a plain format that requires no execution environment (runtime) or specific framework, allowing it to be used immediately simply by loading it. Each row includes an actual tabular dataset, a corresponding question, and a gold answer that can be deterministically verified with the included grader, making it easy to test and evaluate prompts even with small models in a local environment.<\/p>\n

Claims and Evidence<\/h2>\n

Presenter FineEnvs claims that this dataset can provide extremely reliable, deterministic signals for the reinforcement learning and evaluation of small models. To support this claim, the following features and data composition are presented.<\/p>\n

1. Tasks Subject to Rigorous Verification Processes<\/h3>\n

All included tasks are built on top of the jupyter-agent dataset<\/a>, which was extracted from actual data science notebooks spanning 471 Kaggle datasets. Every extracted question-answer pair has passed verification confirming that a strong agent model can actually solve the task in a live sandbox environment and reproduce the gold answer under deterministic grading. Since all ambiguous or unverifiable tasks have been eliminated, every included task is guaranteed to be “solvable” and “objectively gradable.”<\/p>\n

2. LLM-Independent Deterministic Evaluation System<\/h3>\n

No LLM-based judgment (LLM-as-a-judge) is involved in the evaluation (reward signal) pathway of this dataset. The included grader.py<\/code> performs step-by-step rule-based checks such as exact matching, numerical comparison taking tolerances (atol<\/code>\/rtol<\/code>) into account, list or percentage normalization, and symbolic equivalence verification. This prevents evaluation metric drift caused by changes in the evaluation model and allows model performance to be measured against a constant standard at all times.<\/p>\n

3. Intentionally Hardened Evaluation Splits<\/h3>\n

The test and evaluation splits are designed to be more difficult than the training split. Specifically, while the train split consists of 29% easy and 14% hard tasks, the held-out evaluation splits (test<\/code> and eval<\/code>) consist of 11\u201313% easy and 38\u201340% hard tasks. This makes it possible to measure the generalization performance of models more rigorously. The details of the data split are as follows:<\/p>\n

\n

\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\n\n\n\n\n
Split<\/th>\nTasks<\/th>\nEasy<\/th>\nMedium<\/th>\nHard<\/th>\nWhat it’s for<\/th>\n<\/tr>\n<\/thead>\n
train<\/code><\/td>\n5,000<\/td>\n1,433<\/td>\n2,845<\/td>\n722<\/td>\ntraining<\/td>\n<\/tr>\n
test<\/code><\/td>\n250<\/td>\n33<\/td>\n118<\/td>\n99<\/td>\nheld-out benchmark, deliberately harder<\/td>\n<\/tr>\n
eval<\/code><\/td>\n144<\/td>\n16<\/td>\n74<\/td>\n54<\/td>\nquick validation during a run<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n

4. Data Structure Including Rich Metadata<\/h3>\n

Each row of the dataset comprehensively covers the information required to run and evaluate tasks.<\/p>\n

\n\n\n\n\n\n\n\n\n\n\n\n\n\n
Column<\/th>\nMeaning<\/th>\n<\/tr>\n<\/thead>\n
task_id<\/code>, source_row_id<\/code><\/td>\nidentifiers<\/td>\n<\/tr>\n
question<\/code><\/td>\nthe question to answer<\/td>\n<\/tr>\n
answer<\/code><\/td>\nthe gold answer<\/td>\n<\/tr>\n
reward_mode<\/code>, atol<\/code>, rtol<\/code><\/td>\nhow to grade it: match type and numeric tolerances<\/td>\n<\/tr>\n
difficulty_level<\/code> (1\u20135), difficulty_tier<\/code><\/td>\ndifficulty<\/td>\n<\/tr>\n
kaggle_dataset<\/code><\/td>\nthe source dataset<\/td>\n<\/tr>\n
hf_bucket<\/code>, bucket_prefix<\/code>, files<\/code><\/td>\nwhere the input files live and what they are<\/td>\n<\/tr>\n
instruction<\/code><\/td>\nthe full agent prompt<\/td>\n<\/tr>\n
package_tier<\/code><\/td>\nenvironment sizing hint<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n

5. Deployment of Family Repositories According to Use Case<\/h3>\n

Rather than simply being provided as a basic dataset, SmolDataEnvs is composed of multiple repositories tailored to specific purposes, such as trajectory data for supervised fine-tuning (SFT) and the Harbor suite executable as a sandbox environment.<\/p>\n

\n\n\n\n\n\n\n\n\n\n
Repo<\/th>\nWhat it is<\/th>\n<\/tr>\n<\/thead>\n
SmolDataEnvs<\/code><\/a><\/td>\nthe tasks as plain rows, load it and prompt any model<\/td>\n<\/tr>\n
SmolDataEnvs-sft<\/code><\/a><\/td>\n4,677 verified agent trajectories, TRL-ready<\/td>\n<\/tr>\n
SmolDataEnvs-harbor-train<\/code><\/a><\/td>\n5,000 tasks as Harbor environments<\/td>\n<\/tr>\n
SmolDataEnvs-harbor-test<\/code><\/a><\/td>\n250 held-out, deliberately harder<\/td>\n<\/tr>\n
SmolDataEnvs-harbor-eval<\/code><\/a><\/td>\n144 for quick validation during a run<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n

Hardware Requirements<\/h2>\n

Regarding the use of this dataset and evaluation code, the prerequisites indicated in the materials are as follows:<\/p>\n