declarative pi agent harness experiments
  • TypeScript 70.9%
  • JavaScript 20%
  • Vue 3.6%
  • Nix 2.8%
  • Python 2%
  • Other 0.7%
Find a file
Repository files (latest commit first)
Filename Latest commit message Latest commit date
Göran Heinemann be0e2f8abf
vision: surface provider errors, 300s default timeout, one retry on transient failures
analyzeImage() previously returned empty text when the shared llama-server
timed out (pi-ai resolves provider failures as stopReason 'error' results,
not rejections), so browser_view showed a misleading '(vision model returned
empty text)' after exactly 120s of waiting. Warm VLM calls take ~47-65s,
leaving no headroom under concurrent agent load on the same server.

- throw 'vision provider error: <reason>' on stopReason error/aborted so
  callers report the real cause (browser_view: '(visual analysis failed: ...)')
- per-request timeout configurable via VISION_TIMEOUT_MS, default 300_000
  (was hard-coded 120_000)
- retry once on transient failures (timeout/abort/provider error/no
  finish_reason); auth errors and clean empty completions fail fast

Verified against the live server: forced 1.5s timeout -> one retry, then
descriptive throw at ~3.1s; normal call -> full report in 47.5s.
2026-09-04 12:04:39 +02:00
agent send session id headers 2026-09-03 12:28:23 +02:00
doc add skillforge plugin: experience → wiki → skills loop 2026-09-03 22:48:42 +02:00
extensions vision: surface provider errors, 300s default timeout, one retry on transient failures 2026-09-04 12:04:39 +02:00
packages add research skill and refine webbrowser 2026-09-03 13:59:29 +02:00
scripts add research skill and refine webbrowser 2026-09-03 13:59:29 +02:00
.gitignore initial commit 2026-09-02 19:20:53 +02:00
flake.lock initial commit 2026-09-02 19:20:53 +02:00
flake.nix add basic playwright browser tool 2026-09-03 12:26:56 +02:00
presets.nix add skillforge plugin: experience → wiki → skills loop 2026-09-03 22:48:42 +02:00
README.md add skillforge plugin: experience → wiki → skills loop 2026-09-03 22:48:42 +02:00

pi — reproducible agent setup as a Nix flake

Packages the full contents of ~/.pi/agent (settings, themes, knowledge base, all custom extensions as toggleable modules, and the installed pi packages) together with the pi binary from numtide/llm-agents.nix (pinned npm release, bun-compiled standalone binary).

Usage — presets

Command What you get
nix run .# or nix run .#full full preset: everything enabled (default)
nix run .#minimal minimal preset: no knowledge graph, no gauntlet
nix run .#minimal-browser minimal + the headless Chromium web-browser plugin
nix run .#researcher lean instance + browser + scientific-research skill (full-text literature workflow) + skillforge (experience→wiki→skills loop)
nix develop / nix develop .#minimal same presets as a dev shell (pi on PATH + nodejs/git/imagemagick/ripgrep/fd)
nix run .# -- -p "one-shot prompt"     # args pass through to pi

Presets live in presets.nix — plain attrsets of option values. Adding a new preset there automatically gives it packages.agent-<name>, apps.<name> and devShells.<name>.

Plugin module system

Every extension is one self-contained directory under extensions/:

extensions/<name>/
  index.ts          (or package.json with "pi.extensions" for multi-file plugins)
  module.nix        ← declares its options + build data (auto-discovered!)
  package*.json     optional: own npm lockfile when the plugin has heavy deps

extensions/default.nix auto-discovers every subdirectory that contains a module.nix — dropping in a new plugin directory registers it, no other file needs editing. Each module declares Nix options under its name:

  • extensions.<name>.enable (mkEnableOption) — on/off switch used by presets
  • typed per-plugin settings in the same namespace, e.g. extensions."kg-dgraph".endpoint, readable from anywhere nix can evaluate (including nvim tooling via passthru.mkAgent / passthru.lib, below)

Unknown plugin names or subkeys fail with did-you-mean suggestions — the option declarations close the namespace, so preset typos are caught at eval time.

packages/agent.nix runs lib.evalModules over base + all plugin modules + the active preset, then assembles the agent dir from enabled plugins only: disabled plugins' code and npm trees are never copied or even built.

Plugin Core Deps Notes
llama-server yes provider registration, sampling params, /llama menu
model-router yes shared route_model task→model selection
shared yes shared ai-sdk transport (used by vision/model-router)
vision no shared* image analysis via llama-server
context-footer no footer: actual tokens + percent
web-search no SearXNG search, localhost:8080
web-link-extractor no link extraction for research
web-browser no self headless Chromium via playwright-core (navigate/view+VLM/click/type/select/press/scroll/read/tabs/evaluate); nixpkgs chromium is injected as BROWSER_EXECUTABLE by packages/agent.nix
gl-v4 no shared* gauntlet loop tool; contributes npm:pi-gauntlet to settings
herdr-agent-state no agent state socket
warcraft-sound no sound effects on events
kg-dgraph no self knowledge graph, dgraph localhost:8081
pi-mermaid no self mermaid rendering; contributes npm:pi-mermaid to settings
autonomous-run no self standalone loop engine (not loaded by pi; dev artifact)
scientific-research no skill-only, off by default: bundles the scientific-research Agent Skill under <src>/skills/; enabled in the researcher preset
skillforge no skill-only, off by default (python3-stdlib scripts): mines session traces into per-domain KB wikis and derives gated skills from them (/skill:skillforge mine|propose|lint); enabled in the researcher preset; design in doc/plans/2026-09-03-automatic-skill-derivation.md

* "shared" plugins resolve through the root tree built from extensions/_shared/package-lock.json; most imports actually come from pi's jiti virtual modules, so only plugins with real npm deps need a lock strategy.

Settings & environment (nix options)

Beyond on/off, three piAgent.* options drive the files generated into the agent dir:

Option Type / default Effect
piAgent.settings override layer over agent/settings.base.json merged (recursiveUpdate) into the generated settings.json; e.g. piAgent.settings.theme = "rose-pine";. The generated packages key always wins.
piAgent.models override layer over agent/models.json merged into the generated models.json; plugins patch fields here (llama-server writes its baseUrl).
piAgent.env attrsOf str, default {} written to env.sh in the store tree, sourced by scripts/activate.sh before pi starts. Keys must be valid shell names (checked). Free-form: piAgent.env.GAUNTLET_TIMEOUT_MS = "60000";.

Each generated env line uses a ${VAR:-default} fallback, so an export from caller's shell wins over the nix value — ad-hoc overrides like DGRAPH_ENDPOINT=http://… nix run .#full work without rebuilding. Precedence: shell export > nix option > code default.

Typed per-plugin options (each contributes to piAgent.env, gated on enable):

Option Env var / file Default
extensions."kg-dgraph".endpoint DGRAPH_ENDPOINT http://localhost:8081
extensions."web-search".searxngUrl SEARXNG_URL (instance base, no trailing path) http://localhost:8080
extensions."llama-server".baseUrl LLAMA_SERVER_URL + models.json provider baseUrl https://ai.goeranh.de/v1
extensions."model-router".defaultModel ROUTER_DEFAULT_MODEL (per-category: piAgent.env.ROUTER_<CAT>_MODEL) ornith-ai/Ornith-1.5-35B-A3B-GGUF:Q4_K_M
extensions.vision.model VISION_MODELonly when non-default: the vision preflight treats a set VISION_MODEL as an explicit pin (stricter model matching), so exporting the built-in default would change registration behavior unsloth/Qwen3.8-27B-GGUF:Q8_0
extensions."web-browser".executable BROWSER_EXECUTABLEonly when non-empty; empty means packages/agent.nix injects nixpkgs chromium (or chromium on PATH) nixpkgs chromium
extensions."web-browser".headless / .viewport / .navigationTimeoutMs / .disableSandbox BROWSER_HEADLESS / BROWSER_VIEWPORT / BROWSER_NAV_TIMEOUT_MS / BROWSER_DISABLE_SANDBOX 1 / 1440x900 / 60000 / 0

Secrets stay out of nix by design (LLAMA_SERVER_API_KEY is still a shell export; models.json references it via $LLAMA_SERVER_API_KEY).

How presets disable things completely

Two mechanisms compose:

  1. Extension code: disabled plugin dirs are simply absent from the store tree — pi never discovers them.
  2. settings.json is generated (base agent/settings.base.json + package contributions of enabled plugins). pi loads an installed npm package's extensions/skills only when it appears in this list, so e.g. disabling gl-v4 also drops npm:pi-gauntlet and its skills vanish — the files may still sit in node_modules, unused.

Bundling skills (convention: <name>/skills/)

A plugin can bundle Agent Skills without any npm package: put one or more skill directories under extensions/<name>/skills/ each containing a SKILL.md. When the plugin is enabled, packages/agent.nix copies them into $out/skills/; the symlink farm then exposes that directory as <agentDir>/skills/, where pi discovers them as global skills for the instance. Disabling the plugin removes its skills — same toggle semantics as extension code and settings.json package entries. Skill-only plugins need no index.ts (pi only loads extension code from *.ts/index.ts) and no npm lock (lock = null). See extensions/scientific-research/ for the canonical example; presets opt in via presets.nix (e.g. researcher).

Custom configs from outside (e.g. nvim)

The flake exposes the builder function:

nix build --impure -E \
  '(builtins.getFlake "path:/home/goeranh/projects/nix/pi").passthru.mkAgent { extensions."kg-dgraph".enable = false; }' \
  --print-out-paths

Any attrset of extensions.* options works — presets are just named instances.

Adding a new extension (walkthrough)

  1. Create extensions/<name>/ with the code:

    • single file → <name>/index.ts
    • multi-file → any layout + "pi": { "extensions": ["./path.ts"] } in its package.json (pi's native manifest form — loader verified)
  2. Add <name>/module.nix:

    { config, lib, ... }:
    let cfg = config.extensions.<name>; in
    {
      options.extensions.<name>.enable = lib.mkEnableOption "what it does";
    
      config = lib.mkMerge [
        {
          extensions.<name>.enable = lib.mkDefault true;
          piAgent.plugin.<name> = {
            src = ./.;            # this plugin directory (code + manifests)
            lock = null;          # "self" | "shared" | null — npm dep strategy
          };
        }
      ];
    }
    
  3. If it needs npm deps: run npm install --package-lock-only inside the dir, set lock = "self" + npmDepsHash = "…" (derive per "Updating dependency hashes" below); if none of its imports need the root tree, keep lock = null.

  4. If it needs an npm pi package in settings.json: add (lib.mkIf cfg.enable { piAgent.packages = [ "npm:<pkg>" ]; }) to its mkMerge.

  5. If the code reads env vars, declare typed options next to enable and export them from the module (gated on enable):

    options.extensions.<name>.endpoint = lib.mkOption {
      description = "…; exported to pi as FOO_ENDPOINT.";
      type = lib.types.str;
      default = "http://localhost:1234";   # must match the code fallback
    };
    
    config = lib.mkMerge [
      { /* …plugin descriptor, as above… */ }
      (lib.mkIf cfg.enable {
        piAgent.env.FOO_ENDPOINT = cfg.endpoint;   # exported to pi via env.sh
      })
    ];
    
  6. nix build .#agent-full — done. It is registered, toggleable and preset-able with zero other edits (auto-discovery).

The store tree holds all content read-only: extensions, themes, pre-built node_modules, the git package checkout, generated settings. On first run per preset (and on every nix update) scripts/activate.sh builds a symlink farm at $XDG_STATE_HOME/pi-agent[-<preset>]: one symlink per store entry, plus real writable copies of only the files pi must be able to lock or modify:

File Why it must be a real file
settings.json (copy) pi acquires a proper-lockfile lock (mkdirs settings.json.lock) on every read — against the store this EROFSes and pi silently discards your entire settings
auth.json ({} seed) credential-store reads; the llama-server key comes from $LLAMA_SERVER_API_KEY in models.json, nothing secret is stored here
trust.json (copy) project-trust decisions are appended at runtime
models-store.json (copy) custom-model cache written by pi
pi-debug.log (empty) append-only debug log

Env vars set for each preset:

Env var Value
PI_CODING_AGENT_DIR $XDG_STATE_HOME/pi-agent-<preset> (the farm)
PI_CODING_AGENT_SESSION_DIR $XDG_STATE_HOME/pi/sessions — conversation history, the only growing state, shared across presets
PI_OFFLINE 1 — no runtime package/tool network operations; updates go through this repo + rebuild

Consequences (all by design):

  • Runtime settings changes in the TUI persist until the next nix update, which re-seeds them from this repo.
  • New trusted project roots outside /home/goeranh get prompted once and are then stored in the writable trust.json copy.

Secrets

LLAMA_SERVER_API_KEY is intentionally not in this repo. Export it in your shell profile (as today) — the activation warns if it is missing. models.json references it via $LLAMA_SERVER_API_KEY, so nothing secret needs to be committed. Do not commit auth.json.

Runtime prerequisites

No network needed at startup (the git theme package ships pre-fetched and PI_OFFLINE=1). Your enabled plugins expect:

Service Endpoint (default) Needed by
llama.cpp server https://ai.goeranh.de/v1 — option extensions."llama-server".baseUrl chat + vision models (always)
SearXNG http://localhost:8080 — option extensions."web-search".searxngUrl web-search plugin
Dgraph http://localhost:8081 — option extensions."kg-dgraph".endpoint kg-dgraph plugin

Layout

flake.nix              inputs, presets → packages/apps/devShells, passthru.mkAgent
presets.nix            full / minimal option sets (add your own here)
packages/agent.nix     evalModules + assembly of the agent directory
scripts/activate.sh    symlink farm + env activation (shared by run & develop)
extensions/            ← THE PLUGIN REGISTRY (auto-discovered)
  default.nix          aggregator: readDir → all module.nix files
  base.nix             declares the piAgent.* option tree
  <name>/module.nix    one per plugin — options + build data
  _shared/             root npm manifest for lightweight plugins
agent/                 declarative base config (no extensions anymore)
  settings.base.json   settings minus "packages" (those are generated)
  models.json          llama-server provider + model catalogue
  AGENTS.md            global instructions
  knowledge-base/      NixOS/Nix reference docs referenced by AGENTS.md
  themes/              custom themes (dracula, rose-pine…)
  npm/                 manifests for the always-on pi "packages" set

Updating things

  • pi binary / nixpkgs: nix flake update llm-agents — the pinned pi version lives in their packages/pi/hashes.json and has a declarative updater.
  • Plugin code/config: edit under extensions/<name>/ or agent/, commit, done.
  • Always-on pi packages (agent/npm/package*.json): after changing versions and regenerating the lockfile locally, re-derive npmDepsHash in packages/agent.nix (below).
  • Preset membership: edit presets.nix.

Updating dependency hashes

Each npm tree pins an npmDepsHash: self-locked plugins carry theirs in their own module.nix; the shared root tree and the always-on pi-packages tree live in packages/agent.nix. After touching a lockfile:

# 1. put any well-formed placeholder into the affected hash, then build;
nix build .#agent-full 2>&1 | grep -A3 "hash mismatch"
#    → the error prints `got sha256-…`; copy that value back in and rebuild.

What is NOT included (and why)

Item Reason
~/.pi/agent/sessions/ 200MB+ of personal history; lives in $XDG_STATE_HOME/pi/sessions (the one writable dir, shared across presets)
auth.json secret — the llama-server provider authenticates via $LLAMA_SERVER_API_KEY instead
bin/{fd,rg} already on PATH via the pi wrapper and devShell; pi finds them there and never downloads its own copies
all node_modules/ rebuilt reproducibly from lockfiles by the module system
per-plugin module.nix in the runtime tree build-time metadata only — stripped from the store output

(Committed as static seeds: trust.json, models-store.json — see consequences above.)