deku is a MiniCPM5-1B-oriented local task harness for small factual jobs: web facts, URL summaries, repo lookup, git / diff questions and weak multi-step plans, with explicit refusals when a request is out of scope. The core idea: planning and tool choice live in code, and the model only does short, grounded completions.

Status: the Phase 2 agent core is in-tree (deku ask, route / refuse / tools). Measure capability smokes on the default GGUF path; do not copy MLX-only scores.

Design at a Glance

Layer	Responsibility
Agent / harness	Route, refuse, tools, weak multi-hop orchestration, integrate answers
LLM client	OpenAI-compatible HTTP (/v1/chat/completions) only
Default serve	Official GGUF + llama-server

The harness does not care how the HTTP API is implemented (llama.cpp, oMLX or any other OpenAI-compatible server), but its semantics are tuned for MiniCPM5-1B: English-first, no free-form long reasoning, no code authoring.

Default backend:

• Weights: openbmb/MiniCPM5-1B-GGUF
• Recommended quant: MiniCPM5-1B-Q4_K_M.gguf (~657 MB)
• Server: vanilla llama.cpp llama-server with --jinja (matching OpenBMB's llama.cpp cookbook)

Any other OpenAI-compatible MiniCPM5-1B endpoint can be selected via environment variables; MLX is an optional alternate path for Apple Silicon developers, not the default.

How It Works

The pipeline:

1. User question → route / refuse;
2. When needed, call a tool: web / url / dir / git / diff / multi-hop;
3. Tool output becomes grounded evidence;
4. MiniCPM does a short completion, only compressing or extracting short grounded spans;
5. Return an answer or a refusal.

The harness picks tools and builds evidence; MiniCPM does not author multi-step plans.

Intended Capabilities

• Public short facts (web_search)
• Summarize a given URL or local README (url_read / dir_search + hierarchical summary)
• Repo constants / overview (dir_search)
• Last commit / blame-style questions (git_search); staged / unstaged diffs (diff_search)
• Weak multi-step: rule-built plans, optional pronoun binding across hops, numbered or paragraph integration
• Refuse with a reason: math, code authoring, chitchat, open-ended essays / proofs

Quick Start

Requires mise (pins Python and uv) and llama-server on PATH. Weights download once on first serve (~657 MB).

Bash

mise trust && mise install    # python 3.12 + uv
mise run sync                 # uv sync → .venv + editable install
brew install llama.cpp        # provides llama-server
mise run serve                # downloads Q4_K_M, starts llama-server --jinja

Then ask (refusals and some lexical paths also work offline):

Bash

uv run deku ask "Who is the CEO of Apple?"
uv run deku ask "What is the last commit message?" --root .

Common environment variables:

Env	Role	Default
DEKU_URL	API base (no /v1)	http://127.0.0.1:8080
DEKU_MODEL	Model id sent to the server	MiniCPM5-1B
DEKU_MODEL_DIR	GGUF cache directory	~/.cache/deku/models
DEKU_RERANK_URL	MiniCPM-Reranker sidecar base	unset → lexical web rank
DEKU_AUDIENCE	Default refusal wording (human / agent)	human

Unit tests need no model or GPU (mise run test); mise run capability-smoke runs live GGUF smokes for refuse / web / url / hop / git / dir.

Limitations

• Tuned for MiniCPM5-1B on English (and Chinese to some extent). Japanese input is refused (non_english).
• Prefers short answers; long essays and open-ended reasoning are refused with an explicit reason.
• Not a general coding agent; code authoring and math are refused with a reason.
• The default extract path uses chat completions (GGUF + llama-server --jinja); prefill via /v1/completions degenerates on this stack (measured).
• Live web / URL quality depends on search snippets and the network; weak evidence leads to abstention rather than invention.
• Capability claims must be re-measured on GGUF; do not copy MLX-only scores.

Non-goals

• General coding agent / SWE-bench runner
• Model-authored multi-step chain-of-thought planning
• Japanese as a supported input language
• Bundling MLX conversion stacks or large research eval matrices

Links

• GitHub: penta2himajin/deku