Signal/docs/research-versioning-design-2026-06-24.md

# Option B: Research Artifact Auto-Versioning — Pi Design

**Date:** 2026-06-24
**Design by:** Pi
**Builds by:** Claude (Stop hook) + Pi (skill)
**Approves:** Kisa
**Source:** docs/delegation-router-design-2026-06-24.md, task (2) — broaden auto-versioning from current-state.md to research artifacts

---

## Problem

Research artifacts — policy analyses, regulatory intelligence from TriLane, competitive research, automated blog drafts — are currently versioned only when someone remembers to commit them. The workflow-consolidation work scoped the auto-push Stop hook to a single file (`context/current-state.md`). Research artifacts fall outside that scope and drift out of sync.

## Design: Hybrid Research Versioning System

Three complementary layers, each with a different trigger and scope:

| Layer | Trigger | Scope | Guarantee |
|-------|---------|-------|-----------|
| **Stop hook** (automatic) | Session stop | Known research directories | Committed every session |
| **Skill discipline** (agent-guided) | Research task completion | Any research output | Agents remember to commit |
| **Pi extension** (event-driven) | Write to research path | Research files written by Pi | Auto-stages mid-session |

---

## Layer 1: Stop Hook — Broadened Research Auto-Push

A second Stop hook entry in `settings.json` that auto-commits research directories. Separate from `state-autopush.sh` so research artifacts and session state have independent commit histories.

### Directories Scoped

```
signal/research/          — CGM/DMEPOS market research, compliance intelligence, competitive analysis
signal/docs/              — Design documents, architecture specifications, whitepapers
signal/docs/compliance/  — Compliance docs (data handling, privacy, incident response)
signal/content/           — Blog drafts, outlines, published posts (created if doesn't exist)
```

### Exclusion Rules

Do NOT version:

- Raw scraped HTML or JSON from external sources
- Files over 500KB
- Temporary or scratch files (`*.tmp`, `scratch-*`, `draft-*-wip.md`)
- Files in `node_modules/`, `venv/`, `__pycache__/`, `.venv/`
- Files matching `.gitignore` patterns

### The Script

File: `~/.claude/hooks/research-autopush.sh`

```bash
#!/bin/bash
# research-autopush.sh — on Claude Code session stop, commit + push research artifacts.
# Broader scope than state-autopush.sh: covers signal/research/, docs/compliance/, content/.
# Deliberately scoped to known directories. Never `git add .`.

REPO="/Users/sttil-solutions/projects/signal"
EXCLUDE_PATTERNS="*.tmp scratch-* draft-*-wip.md *.json *.html"
DIRS=("research" "docs" "content")

cd "$REPO" 2>/dev/null || exit 0

# Check each directory for changes
HAS_CHANGES=0
for DIR in "${DIRS[@]}"; do
  [ -d "$DIR" ] || continue
  if ! git diff --quiet -- "$DIR" || ! git diff --cached --quiet -- "$DIR"; then
    HAS_CHANGES=1
    break
  fi
done

[ "$HAS_CHANGES" -eq 0 ] && exit 0

# Build exclude args
EXCLUDE_ARGS=()
for PATTERN in $EXCLUDE_PATTERNS; do
  EXCLUDE_ARGS+=(-- "(:(exclude)$PATTERN)")
done

# Add only the scoped directories, respecting exclude patterns
for DIR in "${DIRS[@]}"; do
  [ -d "$DIR" ] || continue
  git add "$DIR" "${EXCLUDE_ARGS[@]}" 2>/dev/null
done

# Check if anything was actually staged
if git diff --cached --quiet; then
  exit 0
fi

git commit -q -m "research: auto-push research artifacts on session stop [hook]" || exit 0

# Sync from canonical (Forgejo origin) before pushing; never force
if ! git pull --rebase origin main >/dev/null 2>&1; then
  echo "research-autopush: rebase conflict — nothing pushed, resolve manually in $REPO" >&2
  exit 0
fi

if git push origin main >/dev/null 2>&1; then
  echo "research-autopush: research artifacts pushed to origin (Forgejo)" >&2
else
  echo "research-autopush: push failed — run 'git push' in $REPO" >&2
fi
exit 0
```

### Settings.json Addition

Add to the `hooks.Stop` array in `~/.claude/settings.json`:

```json
{
  "hooks": {
    "Stop": [
      {
        "hooks": [
          {
            "type": "command",
            "command": "bash /Users/sttil-solutions/.claude/hooks/research-autopush.sh",
            "timeout": 30,
            "statusMessage": "Syncing research artifacts to git"
          }
        ]
      },
      {
        "hooks": [
          {
            "type": "command",
            "command": "bash /Users/sttil-solutions/.claude/hooks/state-autopush.sh",
            "timeout": 30,
            "statusMessage": "Syncing current-state.md to git"
          }
        ]
      }
    ]
  }
}
```

**Order matters:** research-autopush.sh runs first (more data, longer potential conflict window), then state-autopush.sh (single file, fast). Both are non-blocking — failures are logged, not fatal.

---

## Layer 2: Skill Discipline — Research Versioning Skill

A shared skill that agents follow when producing research artifacts. Makes versioning a conscious step in the research workflow, not just an automatic afterthought.

File: `~/.claude/skills/research-versioning/SKILL.md`

```markdown
---
name: research-versioning
description: Agents follow this when producing research artifacts — regulatory analysis, TriLane intelligence, competitive research, blog content. Ensures research is committed before session end, not lost to drift.
metadata:
  type: process
---

# Research Versioning

## Where Research Lives

| Artifact Type | Directory | Example |
|---------------|-----------|---------|
| Market / compliance research | `signal/research/` | `signal/research/cgm-payer-policy-2026-06.md` |
| Compliance documentation | `signal/docs/compliance/` | `signal/docs/compliance/data-handling.md` |
| Blog content | `signal/content/` | `signal/content/blog/why-readiness-matters.md` |

File naming: `<topic>-YYYY-MM.md` for research, `<topic>-YYYY-MM-DD.md` for blog posts.

## When to Save

1. **After every meaningful research output** — not only at session end
2. **Before switching context** — if you move from research to build, commit research first
3. **At session end** — the Stop hook catches anything missed

## What to Exclude

Do NOT commit:
- Raw scraped data (HTML, JSON dumps, API responses)
- Working drafts (use `-wip.md` suffix to auto-exclude)
- Files over 500KB
- Files already covered by `.gitignore`

## Speed

`git commit -m "research: <brief description> [agent]"` is fast enough. Use `git push` deliberately — pushing is not required every commit, but all commits should be pushed by session end.

## Interaction with Stop Hook

The Stop hook (research-autopush.sh) catches anything you missed. Treat it as a safety net, not your primary commit strategy — regular commits give you better history and conflict resolution.
```

---

## Layer 3: Pi Extension — Research Write Watcher

For Pi sessions, an extension that auto-stages research files when they're written. Catches mid-session writes before the Stop hook fires.

File: `~/.pi/agent/extensions/research-versioning.ts`

```typescript
import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";

/**
 * research-versioning.ts — Auto-stages research artifacts when written via Pi's tools.
 * Catches mid-session writes before the Stop hook fires at session end.
 * Reversible: remove or rename this file, then run /reload.
 */

const RESEARCH_PATTERNS = [
  /^\/Users\/sttil-solutions\/projects\/signal\/research\//,
  /^\/Users\/sttil-solutions\/projects\/signal\/docs\/compliance\//,
  /^\/Users\/sttil-solutions\/projects\/signal\/content\//,
];

const EXCLUDE_PATTERNS = [/\.tmp$/, /scratch-/, /draft-.*-wip\.md$/, /\.json$/, /\.html$/];

export default function (pi: ExtensionAPI) {
  pi.on("tool_result", async (event, ctx) => {
    if (event.toolName !== "write" && event.toolName !== "edit") return;

    const path = event.input.path as string | undefined;
    if (!path) return;

    // Check if path matches a research or docs directory
    const isResearch = RESEARCH_PATTERNS.some((p) => p.test(path));
    if (!isResearch) return;

    // Check exclusion patterns
    const isExcluded = EXCLUDE_PATTERNS.some((p) => p.test(path));
    if (isExcluded) return;

    // Silent auto-stage — no UI noise, just ensures the file is tracked
    try {
      const repoDir = "/Users/sttil-solutions/projects/signal";
      const { execSync } = await import("node:child_process");
      execSync(`git add "${path}"`, { cwd: repoDir, stdio: "ignore" });
    } catch {
      // Non-blocking: silent failure, the Stop hook will catch it
    }
  });
}
```

---

## Directory Setup

Create the content directory if it doesn't exist:

```bash
mkdir -p /Users/sttil-solutions/projects/signal/content/blog
```

The research and compliance directories already exist.

---

## Summary: What Gets Versioned When

| Scenario | Layer | Trigger | What Happens |
|----------|-------|---------|-------------|
| Claude writes a research file mid-session | Skill (Layer 2) | Agent follows discipline | Git commit |
| Claude finishes a session with research unsaved | Stop hook (Layer 1) | Session stop | `research-autopush.sh` catches it |
| Pi writes a research file mid-session | Extension (Layer 3) | `tool_call` event | Silent git add |
| Pi finishes a session with research unsaved | Stop hook (Layer 1) | Session stop | `research-autopush.sh` catches it |

---

## Implementation Sequence

| Step | Who | What | Depends On |
|------|-----|------|------------|
| 1 | Claude | Create `research-autopush.sh` + chmod | Nothing |
| 2 | Claude | Edit `settings.json`: add research hook to `Stop` array (before state-autopush) | Step 1 |
| 3 | Claude | Create `~/projects/signal/content/` directory | Step 2 |
| 4 | Claude | Update `.gitignore` if needed (ensure patterns don't conflict) | Step 3 |
| 5 | Pi | Create `research-versioning.ts` extension | Nothing |
| 6 | Pi | Run `/reload` to activate the extension | Step 5 |
| 7 | Pi | Create `research-versioning/SKILL.md` in ~/.claude/skills/ | Nothing |

---

## Reversibility

- **Stop hook**: Remove the research-autopush entry from `settings.json`. Delete `research-autopush.sh`.
- **Extension**: Remove `research-versioning.ts` from `~/.pi/agent/extensions/` and run `/reload`.
- **Skill**: Delete `~/.claude/skills/research-versioning/SKILL.md`.
- **No state affected**: Stateless design. Old commits remain in git history; new files simply stop being auto-staged.

---

## Future Considerations (Phase 2)

- **File size detection**: Auto-skip files over 500KB with a check in the bash script
- **External repo**: Research artifacts could eventually live in a separate `signal-research` repo for cleaner Signal commit history
- **TriLane sync**: Research artifacts written from TriLane queries could auto-tag the source session ID in commit messages
- **Content publishing**: Blog posts committed to `signal/content/blog/` could trigger a Railway deploy or webhook for the STTIL website