ReJev is an independent reproduction of a Jev-style decision-model training pipeline on top of MiniCPM5-2B, exploring whether a small model can learn stable structured decision behavior without autoregressive free-form generation.

The full training-and-evaluation pipeline at a glance; the technical report walks through it in §1.

Experiment

Task format: state + question + options → one decision.

• MiniCPM5-2B + LoRA post-training
• Sealed holdout with leakage checks
• Constrained vs. unconstrained decoding evaluation
• Statistical significance and grouped bootstrap analysis
• Experiment records, cost tracking, and reproducibility boundaries

Result

51.11% baseline → 80.50% ReJev-2B on a sealed holdout (1,892 items / 924 groups):

+29.39pp · 0% invalid output · $5.31 cumulative app billing

On cost: the $5.31 figure is the cumulative billing of the rejev Modal app (including all earlier mis-run overhead for that app); it is not the cost of this training run alone. Cost conventions differ across rounds and are not comparable or additive.

The goal is not to claim equivalence with Jev, but to understand and reproduce the mechanics of lightweight decision models through controlled experiments. Focus: Decision Models · Post-training · Evaluation · Reproducibility.

What this repository actually demonstrates

1. Model understanding — what separates a decision model from a generative LLM.
2. A complete post-training loop — data → LoRA → evaluation, not an API call.
3. Experiment design — sealed holdout, leakage control, baselines, significance testing, and a capacity probe that failed informatively.
4. Engineering discipline — cost tracking, checkpoints, source locking, reproducibility boundaries, and limitations stated rather than smoothed over.

What is not claimed

• No claim of equivalence with the official Jev model, nor that Tev1-4B was reproduced.
• The numbers above are not extrapolated to out-of-distribution capability — the current evidence covers this exam and its kin, nothing wider.
• The cost of removing restricted sources is not attributed to a single variable (two factors co-vary in that run). The −2.33pp figure previously cited is void (corrected 2026-09-28) — it was measured on a contaminated split; the corrected reading on the clean holdout is +0.63pp (CI [−1.13, +2.37]).

Every experiment record carries a validity boundaries section stating what was measured and what cannot be read from it.

Scope of the release

The repository ships the bulk of the experiment records, not all of them. Three paths are excluded in full:

1. exp005 (the four-way comparison against the real Jev): its conclusions rest on measured numbers for a third-party checkpoint (Tev1-4B) whose license is undecided.
2. plan/007 and 008 (the weight-release plans): their text is "which internal files were checked for what", so redacting them leaves a hollow shell.
3. src/publish/ (the weight-release toolchain): it ships via the HF/ModelScope channel, depends on files not in this export, and its test fixtures contain literal malformed-credential patterns that redaction would break.

Separately, the upstream data-build pipeline is not included: it is tev1's MIT code and must be obtained separately. The repository contains no source datasets, model weights, or full per-item result files.

Layout

Plain Text

src/align/          pre-training alignment checks (tokenizer / template / loss mask / parsing), zero GPU
src/data/           renderer and sealed splitting (records → instruction; stratified split + zero-leak assertions)
src/train/          training and evaluation (LoRA SFT on Modal, constrained-decoding eval, aggregation and stats)
tools/              remote-job trigger scripts (deploy + spawn pattern) and the repro-path checker
docs/plan/          plans and decision records (who decided what, and on what evidence)
docs/experiments/   per-round records (config, results, validity boundaries, repro path, billing)

Quick start

Bash

uv sync --locked --group align --python 3.14.3
uv run --locked --group align python src/align/v1a_check.py --offline   # 7 alignment checks

Training and evaluation run on Modal (account and GPU credit required). Use deploy + spawn — not modal run — for the training script: the latter binds the local process to the remote job, and killing it locally cancels the remote run.

Bash

modal deploy src/train/clean_train.py
modal run tools/trigger_clean.py        # returns in seconds; training is fully decoupled

Upstream & acknowledgements

• tev1 (MIT, commit 1dde778) — task protocol spec (system instruction, option-labelling convention, inference parameters), state normalisation, and stratified splitting all originate there.
• MiniCPM5-2B (Apache-2.0) — base model and its official TRL LoRA SFT recipe.

Links

• GitHub: Joe-rq/ReJev
• Technical report: technical-report.md