MetaCog wraps any language model — local or API — in an inference-time metacognition loop. The model (the thinker) writes one answer; a separate judge scores it. If the judge is confident, that is the answer. If not, the model writes several more thought paths in parallel, the judge scores each path and each distinct final answer, and the best one is returned. No training, no weights touched.

The project's quick start uses openbmb/MiniCPM5-2B as the thinker, and any OpenAI-compatible server or in-process Hugging Face checkpoint works.

Plain Text

problem ──▶ greedy thought path ──▶ judge confident? ──yes──▶ answer
                                          │ no
                                          ▼
                       2–6 more thought paths (sized by uncertainty), in parallel
                                          │
                       judge scores every path + every distinct final answer
                                          │
                                          ▼
                                    best path ──▶ answer

Measured results

On 180 paired GPQA Diamond + AIME rows across three hosted thinkers, the thinker alone scores 76.7%, while MetaCog reaches 86.7% (+11 fixed / −2 lost vs the previous MetaCog default, p = 0.02, 0.89× wall time, 1.39× thinker tokens). Pooled over 856 rows on four benchmarks, the earlier configuration lifted accuracy from 75.0% to 79.4%.

Looking for an agent? The author's Heuristic is an async coding-agent harness (a fork of Unreal Agent) with MetaCog built in.

How it works

Config() defaults are the measured v0.3 pipeline (mode="adaptive"):

1. Greedy path. The thinker answers once.
2. Stop check. The judge scores it; a score ≥ stop_confidence (0.95) is returned immediately. Measured: a Jev score ≥ 0.95 was correct 226/229 times.
3. Branch. Otherwise u = 1 − score sets how many more thought paths to sample, round(n_min + u·(n_max − n_min)) (defaults 2..6), all in parallel.
4. Judge. Every full path is scored independently ("is this correct?" — one call per path). Each distinct bare final answer is scored too (Final answer: X, no reasoning) and added with weight answer_prior (0.5).
5. Pick. The highest combined score wins. Only finished paths can win.

Modes

Mode	What happens	When
adaptive	As above; optional sketch level and multi-round tree	Default
best_of_n	Sample n_paths full answers, the judge picks one	Cheapest; any chat endpoint
stepwise	Sample n_paths continuations of step_tokens, judge picks, extend, repeat	Tighter steering; needs prefix continuation
race	All paths stream at once; the judge scores partial text and the first stream ≥ race_confidence (0.8) wins, cancelling the losers mid-flight	Lowest latency; needs a streaming thinker and server parallelism

race is the low-latency variant: every path streams concurrently while the judge re-scores the partial text roughly every race_score_chars of new output. The first stream to cross the threshold wins and the rest are cancelled — so wall-clock is one generation rather than two serial rounds, and losing paths stop paying tokens early. Two streams that finish on the same answer also end the race early. The thinker server must serve the N streams in parallel (e.g. llama-server --parallel N).

Several opt-in knobs are available (all measured; none beat the default significantly): triage lets the judge rate the bare problem first so hard ones skip the greedy stage; sketch_tokens makes branches cheap outlines that are pruned before expansion; max_rounds keeps branching from the best path; diversity_hints gives each extra path a different approach; escalate_thinker lets a stronger model write one path when still unsure; and strategy="choice" uses a single comparative judge call.

Judges

The one rule: the judge must not be the thinker — self-grading measured worse than no judge at all.

Judge	Class	Needs
Jev (TypeSafe, hosted) — default, best measured	SystemOneJudge.jev()	TYPESAFE_API_KEY
Reflex (MIT, 4B, local)	SystemOneJudge.reflex(...)	A local Qwen3.5-4B server, ~8 GB GPU
Local logprob judge (any open model, no key)	LogitJudge	A .gguf or any server returning top_logprobs
Your own	Any object with score(problem, candidates) -> Verdict	—

LogitJudge reads P(yes) straight from a frozen open model's next-token logits — one forward pass per path, no text generated. On 127 saved pools it made 68/127 correct picks vs Jev's 72/127 (p = 0.50, a statistical tie, slightly weaker). It is slow on CPU (~2 min per pool); use a GPU.

Quick start

Python

from metacog import MetaCog, Config, OpenAICompatThinker, SystemOneJudge

thinker = OpenAICompatThinker("http://localhost:8000", model="openbmb/MiniCPM5-2B")
judge = SystemOneJudge.jev()  # hosted; reads $TYPESAFE_API_KEY

mc = MetaCog(thinker, judge, Config())  # defaults = the measured v0.3 pipeline
result = mc.run("How many positive integers below 1000 have digits summing to 9?")

print(result.answer)

The thinker can be any OpenAI-compatible server (vLLM, llama.cpp, Ollama, LM Studio, OpenAI, OpenRouter, …), an in-process Hugging Face checkpoint via TransformersThinker("openbmb/MiniCPM5-2B"), or any object implementing generate(...). OpenAICompatThinker tolerates imperfect servers (ignored n, wrong finish_reason, HTTP 200 error bodies, dropped connections).

A metacog-eval harness (e.g. HumanEval) is included. It runs model-generated code, so use a container, and always report the coverage ceiling next to accuracy.

Links

• GitHub: ItIsCuthNotCup/MetaCog
• Related: Heuristic