sttil/Signal
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
Signal/docs/convex-signal-spike-design-2026-06-24.md

22 KiB
Raw Blame History

Convex-Signal Spike — Build Design

Date: 2026-06-24 Design by: Pi Builds by: Claude Approves: Kisa Source: docs/convex-signal-instance-brief-2026-06-24.md Status: Design input. This is a SEPARATE, parallel spike in repo signal-convex. The current Signal build keeps running.

1. What This Spike Proves

One question answered with a build, not a whiteboard:

Can Convex's reactive platform serve as Signal's data + backend layer (at $25/mo with a BAA) without losing the readiness model's integrity, the auth flexibility we need, or the ability to walk away if Convex doesn't work out?

The spike proves three things:

  1. Core readiness loop — import CSV → grade → display worklist — on Convex
  2. Auth seam works — the spike runs on Convex auth; swapping to Clerk is a config change, not a rewrite
  3. Data-access interface holds — the readiness engine doesn't call Convex directly; it calls an interface

2. Module Map (The Seams)

Six modular seams per the brief, but Koaned to ensure they're actually buildable, not over-granular.

convex/                              ← Convex server-functions directory
├── schema.ts                       ← Convex schema (tables + indexes)
├── auth/
│   ├── AuthProvider.ts             ← Interface: login, getUser, hasPermission
│   └── convexAuth.ts               ← Default: Convex auth
│   └── clerkAuth.ts                ← Seam: Clerk JWT validation (future)
│
├── data/                            ← DATA-ACCESS LAYER (the keystone)
│   ├── Repository.ts               ← Generic interface: get, list, create, update, delete
│   ├── PatientRepository.ts        ← Patient + CoverageLine + Shipment queries
│   ├── DocItemRepository.ts        ← DocItem CRUD
│   ├── OverrideRepository.ts       ← Override CRUD
│   └── README.md                   ← How to swap the backend
│
├── readiness/                       ← PORTABLE READINESS ENGINE
│   ├── engine.ts                   ← gradeLine(), rollupPatient(), computeVerdict()
│   ├── rules.ts                    ← load payer_rules.json, resolve rule by ID
│   ├── citations.ts                ← build CascadeEntry records
│   ├── overrides.ts                ← apply override, recompute, re-rollup
│   ├── payer_rules.json            ← COPIED from signal/python-backend/config/
│   └── README.md                   ← This is portable; review before every copy
│
├── storage/
│   ├── FileStorage.ts              ← Interface: upload, getUrl, delete
│   └── convexStorage.ts            ← Default: Convex file storage
│   └── s3Storage.ts                ← Seam: S3 (future)
│
├── automations/
│   ├── ScheduledChecks.ts          ← Cron jobs: resupply reminders, expiry alerts
│   └── crons.ts                    ← Convex cron definitions
│
└── notifications/
    ├── NotificationService.ts      ← Interface: sendAlert, notifyWorklistChange
    └── navBotNotify.ts             ← Default: NavBot pattern
    └── emailNotify.ts              ← Seam: SES/SendGrid (future)

src/                                 ← React app
├── App.tsx                          ← Auth provider + routing
├── convexClient.ts                 ← Convex client config
├── components/
│   ├── worklist/
│   │   ├── WorklistTable.tsx        ← Reimagined on Convex reactive queries
│   │   └── PatientDetail.tsx        ← Nested patient → device → doc view
│   ├── import/
│   │   └── CSVImport.tsx           ← CSV upload → Convex mutation
│   └── auth/
│       └── AuthGate.tsx            ← Auth seam adapter
│
└── lib/
    └── readinessQueries.ts         ← Convex-specific reactive query hooks

Analogy

Think of it like a modular synthesizer:

  • Convex = the powered Eurorack case (provides power, bus, physical space)
  • Readiness engine = the oscillator module (does the core work, same module in any case)
  • Auth = the MIDI interface (different models, same CV/gate output)
  • Data layer = the patch cables (the interface is the jack standard; swap the module behind it)
  • Storage/notifications/automations = utility modules (envelope generators, mixers)
  • convexClient.ts + reactive queries = the speakers (Convex-specific output; swap the whole signal chain)

The readiness engine (oscillator) is the valuable module. Everything else exists to make it produce sound. Designing for swappable patch cables means you can move the oscillator to a different case without rewiring the whole rig.

3. Data-Access Interface (The Lock-In Hedge)

This is the real design tension. Convex's core value is reactivity — useQuery() auto-re-fetches when data changes. A naive DAO interface that returns plain data loses that.

Compromise: Two-layer design.

Layer A — Convex-specific reactive queries (UI-facing)

These ARE Convex queries — they subscribe to data, auto-update, and ARE NOT portable. This is the Cost Of Doing Business with Convex.

// convex/readiness/queries.ts — Convex-specific, reactive
export const getWorklist = query({
  args: { orgId: v.id("organizations") },
  handler: async (ctx, args) => {
    const patients = await ctx.db
      .query("patients")
      .withIndex("by_org", (q) => q.eq("orgId", args.orgId))
      .collect();
    const readinessEngine = new ReadinessEngine(ctx); // ctx.db → repository
    return patients.map((p) => readinessEngine.rollupPatient(p));
  },
});

Layer B — Repository interface (readiness-engine-facing)

The readiness engine never calls ctx.db directly. It calls a repository interface, which is implemented by a ConcreteRepository that wraps ctx.db. If the backend changes, implement a PostgresRepository that implements the same interface.

// convex/data/Repository.ts — portable interface
export interface Repository {
  getPatient(patientId: Id<"patients">): Promise<Patient | null>;
  listPatientsByOrg(orgId: Id<"organizations">): Promise<Patient[]>;
  getCoverageLines(patientId: Id<"patients">): Promise<CoverageLine[]>;
  createCoverageLine(line: NewCoverageLine): Promise<Id<"coverageLines">>;
  updateCoverageLine(id: Id<"coverageLines">, fields: Partial<CoverageLine>): Promise<void>;
  listDocItems(lineId: Id<"coverageLines">): Promise<DocItem[]>;
  upsertDocItem(item: NewDocItem): Promise<Id<"docItems">>;
  getOverride(patientHash: string, deviceType: string, docType: string): Promise<Override | null>;
  setOverride(override: NewOverride): Promise<void>;
  // ... etc
}

// convex/data/convexRepository.ts — Convex-specific implementation
export class ConvexRepository implements Repository {
  constructor(private ctx: QueryCtx | MutationCtx) {}

  async getPatient(patientId: Id<"patients">) {
    return this.ctx.db.get(patientId);
  }

  async listPatientsByOrg(orgId: Id<"organizations">) {
    return this.ctx.db
      .query("patients")
      .withIndex("by_org", (q) => q.eq("orgId", orgId))
      .collect();
  }
  // ...
}

The readiness engine:

// convex/readiness/engine.ts — portable, takes a Repository
export class ReadinessEngine {
  constructor(private repo: Repository) {}

  async gradeLine(patientId: Id<"patients">, deviceType: string, planId: string) {
    const lines = await this.repo.getCoverageLines(patientId);
    // ... grading logic, calls this.repo for all data access
  }
}

What portability costs: The reactive queries (getWorklist, getPatientDetail) ARE Convex-specific and would be rewritten if we move off Convex. The grading logic in engine.ts is fully portable. The cost of swappability is roughly 20% more code (the interface + implementation split) in exchange for 100% portable business logic. Worth it for a data-critical compliance engine.

BYOD Postgres note: Convex supports connecting an external Postgres database (bring-your-own-database). If we eventually want to own the data layer separately from Convex's runtime, this preserves that path without changing the repository interface at all — just swap which database the interface talks to.

4. Convex Schema + Readiness Model

Schema

// convex/schema.ts
import { defineSchema, defineTable } from "convex/server";
import { v } from "convex/values";

export default defineSchema({
  organizations: defineTable({
    name: v.string(),
    clerkOrgId: v.optional(v.string()), // seam: Clerk org ID, null for Convex auth
    createdAt: v.number(),
  }),

  patients: defineTable({
    orgId: v.id("organizations"),
    patientId: v.string(),      // client's UUID
    status: v.string(),          // rolled-up status (derived)
  }).index("by_org", ["orgId"]),

  coverageLines: defineTable({
    patientId: v.id("patients"),
    deviceType: v.string(),      // "dexcom_g7"
    planType: v.string(),        // REQUIRED: medicare | medicare_advantage | medicaid | commercial | unknown
    planId: v.string(),          // plan identifier from CSV mapping
    gradeable: v.boolean(),      // true only for CGM this phase
    lineStatus: v.optional(v.string()), // derived
    timingFlag: v.optional(v.string()),
  }).index("by_patient", ["patientId"]),

  shipments: defineTable({
    lineId: v.id("coverageLines"),
    dateOfService: v.string(),   // YYYY-MM-DD
    quantity: v.number(),
    components: v.array(v.string()),
    orderNumber: v.string(),
    hcpcs: v.string(),
    isCurrent: v.boolean(),
  }).index("by_line", ["lineId"])
    .index("by_line_dos", ["lineId", "dateOfService"]),

  docItems: defineTable({
    lineId: v.id("coverageLines"),
    docType: v.string(),          // swo | visit | pecos | pa | diagnosis
    requiredState: v.string(),    // REQUIRED | NOT_REQUIRED | NOT_EVALUATED
    inputState: v.string(),       // SUPPLIED | ABSENT
    value: v.optional(v.string()),
    source: v.string(),           // csv | staff_override | computed
    citation: v.optional(v.string()),
  }).index("by_line", ["lineId"]),

  overrides: defineTable({
    orgId: v.id("organizations"),
    patientId: v.id("patients"),
    deviceType: v.string(),
    docType: v.string(),
    status: v.string(),
    statusDate: v.optional(v.number()),
    expiryDate: v.optional(v.number()),
    confirmedBy: v.optional(v.string()),
    confirmedAt: v.optional(v.number()),
    reason: v.optional(v.string()),
  }).index("by_override_key", ["orgId", "patientId", "deviceType", "docType"]),

  cascadeEntries: defineTable({
    patientId: v.id("patients"),
    lineId: v.optional(v.id("coverageLines")),
    docType: v.string(),
    severity: v.string(),         // blocker | alert
    reason: v.string(),
    ruleId: v.string(),           // points to payer_rules.json key
    clientInput: v.string(),
    requirement: v.string(),
    basis: v.string(),
    rulesVersion: v.string(),
    createdAt: v.number(),
  }).index("by_patient", ["patientId"]),

  uploads: defineTable({
    orgId: v.id("organizations"),
    fileName: v.string(),
    uploadedAt: v.number(),
    recordCount: v.number(),
    status: v.string(),           // processing | done | error
    storageId: v.optional(v.id("_storage")),
  }).index("by_org", ["orgId"]),
});

Function Structure

convex/
├── auth.ts                    ← Auth helpers (login, register, session)
├── crons.ts                   ← Cron job definitions
│
├── patient/
│   ├── queries.ts             ← getWorklist, getPatientDetail
│   └── mutations.ts           ← importCSV, deduplicate
│
├── coverageLine/
│   ├── queries.ts             ← getLinesByPatient
│   └── mutations.ts           ← updatePlanType, recomputeLine
│
├── docItem/
│   ├── queries.ts             ← getDocItemsByLine
│   └── mutations.ts           ← setDocItemOverride
│
└── readiness/
    ├── queries.ts             ← gradeLine (read-only grade), rollupPatient
    └── mutations.ts           ← recomputeAll, applyOverride

5. Auth Seam

Recommended Default: Convex auth for the spike

Rationale:

  • One less moving part. The spike is proving Convex on Signal. Adding Clerk adds auth complexity without proving anything new.
  • Convex auth supports the same flows: email/password, Google OAuth, magic link.
  • The seam stays clean — the spike's components call AuthProvider.getUser() and AuthProvider.hasPermission(orgId). The default implementation uses Convex auth. A Clerk implementation would validate Clerk JWT tokens.
  • Swappable later. When/if the spike graduates, the auth backend can change without touching the readiness engine, worklist display, or any business logic.

Seam interface

// convex/auth/AuthProvider.ts
export interface AuthProvider {
  getUserId(ctx: AuthCtx): Promise<string | null>;
  getOrgId(ctx: AuthCtx): Promise<string | null>;
  hasPermission(ctx: AuthCtx, orgId: string, permission: string): Promise<boolean>;
}

Migration path to Clerk

  1. Clerk issues JWT tokens → Convex validates Clerk's JWKS endpoint (already at clerk.sttilsolutions.com/.well-known/jwks.json)
  2. Convex's auth middleware checks the Clerk JWT and extracts user + org
  3. AuthProvider.getUserId() reads from Clerk's session, not Convex's
  4. UI gate swaps from <ConvexAuthGate> to <ClerkAuthGate> — same props, same behavior

6. Readiness Model Reuse Strategy

Copy, not share (for the spike)

The readiness engine + payer_rules.json are copied into signal-convex/convex/readiness/. Rationale:

  • Isolation. The spike is parallel. A shared package creates coupling between a production app and an experiment.
  • Speed. Copying is faster than extracting a package. The spike needs to ship fast.
  • The readiness model is stable. It was locked on 2026-06-23 and won't change during the spike.

If both instances graduate

Extract a shared npm package (@sttil/readiness-engine) that both repos import. The engine is already designed to be portable (Repository interface, no I/O dependency) — this extraction is a mechanical step, not a redesign.

What changes for Convex

The logic is identical. What changes is how it's called:

  • Python → TypeScript. The engine is rewritten from Python to TypeScript (Convex is TS-native). The logic is the same.
  • Convex types for IDs. Instead of string patient_ids, Convex uses typed Id<"patients">.
  • Repository implementation. Instead of Supabase SQL queries, the ConvexRepository uses ctx.db.get/query/patch/insert/delete.
  • No SQL migrations. Convex schema changes are instant (no explicit migrations).

7. Repo Structure + Workflow Integration

Repository

signal-convex/                   ← NEW repo, separate from signal/
├── convex/                      ← Convex server-functions directory
│   ├── schema.ts
│   ├── auth.ts
│   ├── crons.ts
│   ├── data/
│   ├── readiness/
│   ├── storage/
│   ├── patient/
│   ├── coverageLine/
│   ├── docItem/
│   └── notifications/
│
├── src/                         ← React app (Vite + TypeScript)
│   ├── main.tsx
│   ├── convexClient.ts
│   ├── App.tsx
│   └── components/
│
├── package.json
├── convex.json                   ← Convex project config
├── tsconfig.json
├── vite.config.ts
├── CLAUDE.md                    ← Project instructions (like signal/CLAUDE.md)
├── context/
│   └── current-state.md         ← Optional: state-sync for the spike
├── docs/
│   └── convex-signal-design-2026-06-24.md  ← This doc
│
├── .github/
│   └── workflows/               ← CI: typecheck, lint, test on push
│       └── ci.yml
│
└── README.md

Git workflow

  • Source of truth: Forgejo (git.forgejo.sttilsolutions.com:sttil/signal-convex)
  • Mirror: GitHub (via git-push skill convention)
  • Branch strategy: main is the active spike. Feature branches for each phase.
  • Commit discipline: research-versioning patterns apply. The readiness engine and payer_rules.json have an explicit COPY-FROM marker in commit messages.

Operating model

  • Pi designs → Claude builds (same as the current workflow)
  • state-sync applies: signal-convex/context/current-state.md for spike state
  • ship-status-ledger extended: add spike items alongside current-build items
  • research-versioning skill: covers the new repo once the Stop hook is added

8. Compliance Touchpoints + BAA Gate

PHI boundary (copied from the existing model, same rules)

Data Status Reason
patient_id (client UUID) PII-level identifier Same as current build — hashed in logs, only crosswalk key
Patient names, SSNs, DOBs, contacts NEVER INGESTED Architecture contract, same as current build
Device type, DOS, quantity Non-PHI Standard supply chain data
Payer name, plan type Non-PHI Payer policy data

Gate: BAA required BEFORE real PHI

The spike currently runs on mock/synthetic data. Before ANY real patient data touches it:

  1. Convex BAA must be signed by Kisa (email drafted, costs known: $25/mo Professional)
  2. sttil-compliance-check gate: run the compliance check on the Convex setup
  3. Attorney review of the Convex BAA terms vs AWS BAA terms
  4. Data-handling doc update: add Convex to the compliance docs in signal/docs/compliance/

Built-in audit trail

Convex's Professional plan includes an audit log. No extra build needed for basic audit. The cascade entries (citations table) serve as the business-level audit trail.

9. Phased Spike Plan (Timeboxed)

Total estimate: ~40 engineer-hours (Claude with Kisa review). Timeboxed to 5 calendar days of focused work — can be parallelized with the readiness-model build on the current Signal repo.

Phase 1 — Core Readiness Loop (prove the thesis, ~20h)

What ships: CSV in → grade → worklist display. Running on Convex auth.

Step What Hours
1 Create signal-convex repo, scaffold Convex + Vite project 1
2 Define schema (organizations, patients, coverageLines, shipments, docItems) 2
3 Implement Repository interface + ConvexRepository 3
4 Port readiness engine from Python to TypeScript (module 1: gradeLine + rollupPatient) 6
5 Copy payer_rules.json, build rules loader 1
6 Implement CSV upload mutation (parse → store → grade → cascade) 3
7 Build worklist display with reactive queries (patient list + status) 3
8 Implement auth: Convex auth gate, org-aware queries 1

Gate: Kisa reviews Phase 1 before Phase 2 starts.

Phase 2 — Auth Seam + Device-Keyed Override (~12h)

What ships: Override + recompute + re-rollup. Auth seam proven swappable.

Step What Hours
1 Implement Override schema + repository 1
2 Add override mutation + recomputeLine + re-rollupPatient 3
3 Build override UI in patient detail view 2
4 Implement AuthProvider interface + ClerkAuthProvider stub 2
5 Write tests: gradeLine with known inputs, override+recompute, dedup 3
6 Verification: swap to ClerkAuthProvider in config, confirm everything still works 1

Phase 3 — Hardening + Remaining Seams (~8h)

What ships: File storage (Convex), automations (cron reminders), notifications (NavBot), documentation.

Step What Hours
1 File storage seam: upload documents, attach to patients 2
2 Automation: schedule resupply-reminder checks 2
3 Notification: NavBot-compatible alert on new-lead / override-action-needed 2
4 README, CLAUDE.md, compliance notes, spike retrospective 2

10. What Claude Builds First

Claude builds Phase 1, Steps 1-8, in order

The priority is proving the core thesis: CSV-in → grade → worklist-display on Convex. Nothing else matters if this doesn't work.

Explicitly NOT in Phase 1:

  • File storage (Phase 3)
  • Automations (Phase 3)
  • Notifications (Phase 3)
  • Override + recompute (Phase 2 — need the core loop first to verify baseline correctness)
  • Clerk auth (Phase 2 — prove the spike on Convex auth first, swap later)
  • Export, multi-device display, or non-CGM grading

Before Phase 1 starts

  1. Kisa: send the Convex BAA email (item 8 in ship-status ledger)
  2. Claude + Kisa: verify Convex Professional plan includes the BAA (already verified 2026-06-24 — reconfirm at sign-up)
  3. Create signal-convex repo on Forgejo + GitHub

11. Open Questions Resolved

Convex auth vs Clerk for the spike

Decision: Convex auth as the default. The auth seam means Clerk is a config swap, not a rewrite. Reduces spike complexity without compromising future flexibility.

Dedup key (carried from readiness-model decision)

Pending Kisa's decision — same patient + device + DOS under two order numbers → one worklist line or two? The spike design supports both. The shipments table has orderNumber as a displayed field (not a dedup key), and the dedup logic lives in readiness/engine.ts. When Kisa decides, one line changes in the engine.

Data-access interface boundary

Decision: Repository interface between the readiness engine and the database. The reactive query layer (Convex-specific convex/patient/queries.ts) is the cost of Convex and is NOT portable. The tradeoff is documented in Section 3.

12. Effort Estimate Summary

Phase Hours Builds Key Risk
Phase 1 ~20h Claude Porting readiness logic from Python to TS
Phase 2 ~12h Claude Override + recompute complexity
Phase 3 ~8h Claude Notification integration
Total ~40h Claude Timeboxed to 5 calendar days

Key risk: The readiness engine rewrite from Python to TypeScript. The logic is stable and documented (readiness-model-brief-2026-06-23.md), but porting requires attention to edge cases (enum mapping, date handling, the dedup decision). Mitigation: code-auditor review after each Phase-1 step.