# End-to-End Audit: Delegation Router + Research Versioning — Gap Analysis

**Date:** 2026-06-24
**Auditor:** Pi
**Scope:** Both design systems delivered this session

---

## Audit Method: Simulated End-to-End Trace

I traced 5 user flows across both systems, examining every step from prompt submission to output persistence. Verified against the actual files on disk (not just the design docs).

---

## Findings

### Finding 1 (HIGH) — UserPromptSubmit hook output injection is unverified

**System:** Delegation Router — Claude-side trigger
**Flow:** Task-shaped prompt → dispatch-trigger.sh → [dispatch-check] marker
**Gap:** The dispatch-trigger.sh script prints its output to stdout, expecting it to be injected into Claude's conversation context. But the actual behavior of Claude Code's `UserPromptSubmit` hook is **not documented in this setup**. If the hook's stdout goes to a log, stderr, or is silently discarded, the `[dispatch-check]` marker never reaches the model and the trigger never fires.

**Why it matters:** This is the entire trigger mechanism. If the hook doesn't inject stdout into context, the system has no trigger at all.

**Fix needed before building:** Claude must verify how `UserPromptSubmit` hooks work in Claude Code — specifically whether command stdout is injected into the conversation. Test with a simple echo hook before building the real one.

---

### Finding 2 (HIGH) — dispatch-trigger.sh $1 assumption unverified

**System:** Delegation Router — Claude-side trigger
**Flow:** UserPromptSubmit hook → bash script → $1
**Gap:** The script assumes the prompt is passed as the first command argument (`PROMPT="$1"`). If Claude Code's`UserPromptSubmit` hook passes the prompt via environment variable, stdin, or not at all, `$1` is empty and the script always exits without firing.

**Why it matters:** Same as Finding 1 — the trigger silently never works.

**Fix:** Verify hook argument API as part of Finding 1.

---

### Finding 3 (MEDIUM) — Pi extension hooks tool_call, not tool_result (auto-stage never fires)

**System:** Research Versioning — Layer 3 (Pi extension)
**Flow:** Pi writes research file → extension auto-stages
**Gap:** The `research-versioning.ts` extension hooks `tool_call`, which fires **before** the tool executes. The file doesn't exist yet, so `git add` fails silently (caught by try/catch). The auto-stage never actually works for Pi.

**Why it matters:** Layer 3 is completely non-functional for Pi mid-session writes. Only the Stop hook (Layer 1) catches Pi's research artifacts, and only at session end.

**Fix:** Change from `tool_call` to `tool_result` event, which fires after the file has been written.

---

### Finding 4 (MEDIUM) — Dispatch thresholds need merge, not append (conflicting guidance)

**System:** Delegation Router — dispatch skill edits
**Flow:** Model reads dispatch skill → two trigger mechanisms
**Gap:** The dispatch skill currently has an `Auto-invoke` section that says "Auto-invoke (even without being typed) when..." — which relies on the model's own judgment. The redesign replaces this with "When you see the `[dispatch-check]` marker in context..." — which relies on an external hook. The design says "append Thresholds section" but the old and new auto-invoke rules coexist and can conflict:

- Old rule: model decides when to auto-invoke (broad, implicit)
- New rule: hook decides when to flag, model only acts on explicit marker (narrow, explicit)

If both sections survive, the model gets mixed signals. The old section must be fully replaced, not just supplemented.

**Why it matters:** Model confusion = inconsistent delegation behavior.

**Fix:** When Claude edits dispatch SKILL.md, the old `Auto-invoke` bullet must be completely deleted and replaced by the new `Auto-invoke (dispatch check fired by UserPromptSubmit hook)` section. Do NOT leave both.

---

### Finding 5 (LOW) — signal/docs/ is not covered by research auto-versioning

**System:** Research Versioning — all three layers
**Flow:** Pi writes design doc to signal/docs/ → auto-versioning
**Gap:** The research-versioning extension watches `research/`, `docs/compliance/`, and `content/` — but NOT `signal/docs/`. Design documents written by Pi or Claude to `signal/docs/` are not auto-staged by Layer 3, not auto-committed by Layer 1 (Stop hook), and only covered by the manual discipline of Layer 2.

The `design-review-persist` skill tells Pi to manually commit — but Claude has no equivalent prompt to do the same.

**Why it matters:** signal/docs/ is the canonical home for design documents. If neither agent auto-versions it, design docs drift.

**Fix:** Add `signal/docs/` to Layer 1 stop hook directory scope and Layer 3 extension watched paths. The exclusion patterns already prevent raw data and large files from being committed.

---

### Finding 6 (LOW) — Mid-session research commits stay local until next push cycle

**System:** Research Versioning — Layer 1 (Stop hook)
**Flow:** Agent commits mid-session → Stop hook sees clean tree → exits without pushing
**Gap:** When an agent follows the research-versioning discipline and commits mid-session, the working tree is clean. The Stop hook (research-autopush.sh) exits immediately because `git diff --quiet` shows no changes. Those commits are never pushed until the NEXT session triggers a research change that creates a new commit-and-push cycle.

**Why it matters:** If the machine crashes or the session is interrupted between sessions, mid-session commits exist only in the local repo and are lost.

**Fix (optional):** The Stop hook could run `git push` even when it doesn't commit. But this adds latency to every session close. Risk is low — acceptable for now.

---

### Finding 7 (LOW) — Chief-of-staff sequential dependency handling is untested

**System:** Delegation Router — dispatch thresholds
**Flow:** Multi-domain sequential task → chief-of-staft orchestration
**Gap:** The thresholds route "sequential handoffs between specialists" to chief-of-staff. But the actual capability of the skyremote `chief-of-staff-kit` plugin to handle dependencies (specialist B depends on specialist A's output) is unknown. The plugin may only support parallel dispatch, not sequential workflows.

**Why it matters:** If chief-of-staff can't sequence, multi-domain sequential tasks break.

**Fix:** Discovery task for Claude: test chief-of-staff with a sequential brief. If it handles ordering, document it. If not, add a note that sequential multi-domain tasks need manual chaining.

---

### Finding 8 (LOW) — Pi model behavior with injected messages is untested

**System:** Delegation Router — Pi-side trigger
**Flow:** Pi extension injects [dispatch-check-pi] → model may or may not respond
**Gap:** The dispatch-trigger.ts extension injects a customType message with `display: true`. Whether the model (deepseek-v4-flash) treats this as an actionable instruction or ignores it as session metadata is unknown without testing.

**Why it matters:** The Pi-side trigger may silently have zero effect.

**Fix:** Testing step: submit a task-shaped prompt to Pi and verify the model responds to the dispatch-check. If ignored, switch from `message` injection to `systemPrompt` modification.

---

### Finding 9 (LOW/KNOWN) — Skills directory is not git-versioned

**System:** Both — shared infrastructure
**Flow:** Any skill file in ~/.claude/skills/
**Gap:** All deployed skills (dispatch edits, delegation-discipline, design-review-persist, research-versioning) live in `~/.claude/skills/` which is NOT under version control. A machine failure loses all skill work.

**Status:** TRACKED. This is phase 2 "skills-as-git-repo" from the workflow-consolidation plan.

---

## Summary

| # | Severity | Finding | System | Fix needed |
|---|----------|---------|--------|------------|
| 1 | **HIGH** | UserPromptSubmit hook output injection unverified | Delegation Router (Claude) | Verify hook behavior before building |
| 2 | **HIGH** | dispatch-trigger.sh $1 assumption unverified | Delegation Router (Claude) | Verify hook argument API before building |
| 3 | **MEDIUM** | Pi extension hooks tool_call not tool_result | Research Versioning (Pi) | Change event to `tool_result` |
| 4 | **MEDIUM** | Dispatch thresholds need merge, not append | Delegation Router (dispatch) | Delete old Auto-invoke section, don't append |
| 5 | **LOW** | signal/docs/ not covered by research auto-versioning | Research Versioning (all layers) | Add docs/ to watched directories |
| 6 | **LOW** | Mid-session commits stay local until next push | Research Versioning (Stop hook) | Optional: push even on clean tree |
| 7 | **LOW** | Chief-of-staff sequential handling untested | Delegation Router (thresholds) | Discovery test by Claude |
| 8 | **LOW** | Pi model behavior with injected messages untested | Delegation Router (Pi trigger) | Testing step |
| 9 | LOW/KNOWN | Skills not git-versioned | Both | Phase 2: skills-as-git-repo |

## Recommended Next Steps

**Critical (blockers to resolve before building):**

1. Claude: Verify UserPromptSubmit hook behavior — does stdout inject into context? Is `$1` the prompt?
2. Pi: Fix research-versioning extension from `tool_call` to `tool_result`

**Design refinements (fix in design docs before handoff):**
3. Clarify dispatch threshold merge instructions (replace old section, not append)
4. Add `signal/docs/` to research versioning scope (all 3 layers)

**Discovery tasks for Claude:**
5. Test chief-of-staff sequential dependency handling
6. Document the hook argument API after verification

---

## Files Affected by Fixes

| Finding | File to Change | Change Type |
|---------|---------------|-------------|
| 3 | `~/.pi/agent/extensions/research-versioning.ts` | Edit: tool_call → tool_result |
| 4 | `signal/docs/delegation-router-design-2026-06-24.md` | Edit: clarify merge vs append |
| 5 | `signal/docs/research-versioning-design-2026-06-24.md` | Edit: add docs/ to scope |
| 5 | `~/.pi/agent/extensions/research-versioning.ts` | Edit: add docs/ to RESEARCH_PATTERNS |
| 5 | `signal/docs/research-versioning-design-2026-06-24.md` | Edit: add docs/ to Stop hook |
| 1,2 | `signal/docs/delegation-router-design-2026-06-24.md` | Add: verification step before build