Readiness verdicts now ride additively on /api/upload and /api/confirm-visit: - core/worklist_readiness.py: ReadinessIndex — row-to-line membership resolved through the dedup group key (patient_id, device_type, DOS), never re-derived from a row's own plan_type (kills a reviewer-reproduced false green) - RecordOut gains plan_type / readiness_status / readiness_items (additive, default None); UploadResponse gains readiness_stats (reconciles with rows) - Minimal plan_type CSV mapping (client-supplied, lowercased+trimmed, never guessed) per the umbrella reactivation plan; full enforcement lands in P5 - Fixes pre-existing record_lookup miss for order-numbered CSVs; fallback fields now come from the dedup MergedShipment (latest-non-null, collected order numbers) so doc_state and the verdict grade from the same values - confirm-visit normalizes plan_type identically to the CSV path Verification: 132 tests green (108 baseline + 24 wiring/regression); adversarial 3-lens review + refutation pass (7 findings confirmed, all fixed, regression-locked); independent code-auditor verdict SHIP; Pi E2E 33/33. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
592 lines
34 KiB
Markdown
592 lines
34 KiB
Markdown
# Build Plan 01 — Wire the Readiness Model into the Live Pipeline
|
|
|
|
**Date:** 2026-07-06
|
|
**Author:** lead-signal (Claude), grounded against the working tree on this date
|
|
**Status:** PLAN ONLY. No production code has been modified. Execute-from-here document
|
|
for a future session (possibly a less capable model). Every load-bearing claim is
|
|
tagged GROUNDED (verified in a file or by running a command) or GRAFTED (assumption
|
|
or recommendation).
|
|
**Prior design docs this builds on (read them, do not re-derive them):**
|
|
- `docs/readiness-model-brief-2026-06-23.md` (Phase 1 spec, locked decisions)
|
|
- `docs/readiness-model-design-phase2-2026-06-26.md` (Pi's Phase 2 design, esp. section 4 migration strategy)
|
|
- `docs/dedup-design-2026-06-24.md` (locked dedup key and tiebreakers)
|
|
|
|
---
|
|
|
|
## 1. Objective
|
|
|
|
Wire the already-built, already-tested readiness engines (`core/dedup.py` +
|
|
`core/readiness.py`) into the live FastAPI pipeline as an ADDITIVE readiness verdict
|
|
on both scoring paths (CSV batch upload and Confirm Visit recompute), changing no
|
|
existing worklist behavior, per the Phase 2a coexistence strategy.
|
|
|
|
---
|
|
|
|
## 2. Current state (GROUNDED)
|
|
|
|
### The engines exist, tested, and are wired to NOTHING
|
|
|
|
- `python-backend/core/dedup.py` (251 lines). GROUNDED.
|
|
- `dedup(records: list[ShipmentRecord]) -> list[MergedShipment]` groups rows by the
|
|
LOCKED key `(patient_id, device_type, shipment_date-as-date)` (dedup.py:185-215).
|
|
- `assign_to_coverage_lines(merged) -> dict[CoverageLineKey, CoverageLine]` groups
|
|
MergedShipments by `(patient_id, device_type, plan_type or "unknown")`; latest DOS
|
|
becomes `current_shipment`, earlier ones `prior_shipments` (dedup.py:218-250).
|
|
- A fresh `CoverageLine` is intentionally ungraded: `line_status=None`,
|
|
`timing_flag=None` (dedup.py:246-247). Grading is a separate step. GROUNDED.
|
|
- `python-backend/core/readiness.py` (287 lines). GROUNDED.
|
|
- `evaluate_readiness(plan_type, *, csv_swo_status, csv_visit_date,
|
|
confirmed_visit_date, csv_pecos_verified, csv_pa_status, csv_diagnosis_on_file,
|
|
rules) -> ReadinessVerdict` (readiness.py:216-255). Grades ONE line's five doc
|
|
items on two axes (required_state x input_state).
|
|
- `ReadinessVerdict = {plan_type, doc_items: list[DocItem], line_status: LineStatus}`
|
|
(readiness.py:92-96). `DocItem = {doc_type, required_state, input_state, quality,
|
|
value, satisfied}` (readiness.py:82-89).
|
|
- `LineStatus` values are EXACTLY the locked worklist labels plus the sanctioned
|
|
unknown-plan state: "Clear to Ship", "On Track", "Action Needed", "At Risk",
|
|
"Plan Type Needed" (readiness.py:74-79). GROUNDED. Do not invent or rename any.
|
|
- Required-ness is read from `config/payer_rules.json` (readiness.py:197-213, 234-237),
|
|
and `payer_rules.json` carries `pecos_required` / `pa_required` for all four known
|
|
plan types (payer_rules.json:32-72). GROUNDED.
|
|
- A required-and-absent item blocks green (readiness.py:183-185, 278-279); unknown or
|
|
missing plan config yields "Plan Type Needed" (readiness.py:268-274). GROUNDED.
|
|
- **Neither module is imported anywhere in `python-backend/`.** Grep on 2026-07-06 finds
|
|
imports of `core.dedup` / `core.readiness` only in `tests/test_dedup.py`,
|
|
`tests/test_readiness.py`, and `tests/signal_e2e_test.py`. GROUNDED.
|
|
|
|
### Real test status (run 2026-07-06 from repo root, `python -m pytest`)
|
|
|
|
| Suite | Result |
|
|
|---|---|
|
|
| `tests/test_dedup.py` | 30 passed |
|
|
| `tests/test_readiness.py` | 27 passed |
|
|
| `tests/signal_e2e_test.py` (Pi's black-box E2E) | 35 passed |
|
|
| Full `tests/` directory | **108 passed** |
|
|
|
|
GROUNDED (I ran them). Note: `context/current-state.md` (2026-06-27 entry) says
|
|
"73 tests green"; the grounded per-file counts today are the table above. The 57
|
|
engine unit tests (30 + 27) match the 2026-06-26 state entry. Treat the table as
|
|
current truth; do not quote 73.
|
|
|
|
### The live pipeline scores through coverage_calculator only
|
|
|
|
- `python-backend/api/main.py:25` — `from core.coverage_calculator import
|
|
ShipmentRecord, calculate_batch`. GROUNDED.
|
|
- **Batch path (CSV upload -> worklist):** `main.py:443` `POST /api/upload` ->
|
|
`normalize_csv` (main.py:457) -> `calculate_batch(records, as_of, confirmed_visits)`
|
|
(main.py:492-494) -> one `RecordOut` per result via `_to_record_out`
|
|
(main.py:502-521). GROUNDED.
|
|
- **Single-record path (Confirm Visit recompute):** `main.py:670` `POST
|
|
/api/confirm-visit` -> builds one `ShipmentRecord` from echoed request fields
|
|
(main.py:735-750) -> `calculate_coverage(record, confirmed_visit_date=confirmed)`
|
|
(main.py:751, import at main.py:679) -> `_to_record_out` (main.py:766). GROUNDED.
|
|
- `calculate_batch` returns `list[CoverageResult]`, ONE PER INPUT ROW, sorted by
|
|
priority (coverage_calculator.py:368-396). The worklist is row-keyed, not
|
|
patient-keyed (the known bug, Phase 1 brief section 8). GROUNDED.
|
|
- `CoverageResult` carries the TIMING axis only (`CoverageFlag`: SUPPLY_LAPSED,
|
|
VISIT_REQUIRED, RENEWAL_*, RESUPPLY_READY, ACTIVE, NO_RECENT_SHIPMENT,
|
|
TRANSFER_PENDING; coverage_calculator.py:42-51, 86-100). It has no doc-completeness
|
|
verdict. The doc display state comes separately from `compute_doc_state`
|
|
(main.py:321-330), which is driven by `_normalize_payer_type` -> the payer-name
|
|
guesser (`_normalize_payer`, imported at main.py:257). GROUNDED.
|
|
- The live `status_label` comes from `FLAG_LABELS` (main.py:136-146), which is a
|
|
LEGACY label set ("Docs Required", "Escalate", "New Patient", "Confirm Appointment",
|
|
"Begin Outreach", "Clear to Ship", "On Track", "No Recent Shipment"). It is NOT the
|
|
locked four-label verdict set. This plan does not touch it; replacing it is the P6
|
|
frontend-nesting step owned by another agent. GROUNDED.
|
|
|
|
### Two wiring realities discovered by reading the code (both load-bearing)
|
|
|
|
1. **Circular-import constraint.** `dedup.py:35` does `from core.coverage_calculator
|
|
import ShipmentRecord, CoverageFlag`. Therefore `coverage_calculator.py` can never
|
|
import `dedup`/`readiness` (circular). The Phase 2 doc's item 5 sketch ("add a path
|
|
in coverage_calculator.py") cannot be implemented literally as written. The seam
|
|
must live OUTSIDE coverage_calculator: a new orchestration module called from
|
|
`api/main.py`. GROUNDED (import verified; conclusion follows necessarily).
|
|
2. **The normalizer never populates `plan_type`.** `ShipmentRecord` has had a
|
|
`plan_type` field since the 2026-06-23 lock (coverage_calculator.py:79-82), but
|
|
`normalize_csv` constructs records WITHOUT it (normalizer.py:315-331) and
|
|
`HEADER_MAP` has no plan_type aliases (normalizer.py:20-88). Header matching is
|
|
exact-normalized (lowercase, `-`/`_` -> space; normalizer.py:169-189), so a client
|
|
column named "plan_type"/"Plan Type" is currently UNMAPPED and dropped. Beware the
|
|
near-collision: bare "plan", "plan_name", "plan name" already alias to PAYER
|
|
(normalizer.py:46-50). Consequence: on day one of wiring, every record has
|
|
`plan_type=None`, so every readiness verdict reads "Plan Type Needed". That is
|
|
doctrinally correct (never guess) but means no green is reachable end to end until
|
|
plan_type enters via CSV mapping. GROUNDED. See Open Question 1.
|
|
|
|
### Doctrine locks this plan must honor (from the source-of-truth docs)
|
|
|
|
- Dedup key LOCKED: `(patient_id, device_type, date_of_service)`; order_number is
|
|
display-only, never in the key (dedup design doc section 1; implemented at
|
|
dedup.py:189, tested by `test_dedup_missing_order_number` and
|
|
`test_dedup_order_numbers_collected_and_deduped`). GROUNDED.
|
|
- Verdict rules LOCKED: green only by positive evidence on every required item;
|
|
required-and-absent blocks green; unknown plan reads "Plan Type Needed", never a
|
|
guess (Phase 1 brief section 4; implemented and tested:
|
|
`test_required_absent_blocks_green`, `test_plan_type_none_is_plan_type_needed`,
|
|
`test_known_plan_missing_config_does_not_false_green`). GROUNDED.
|
|
- Migration strategy LOCKED by Pi's Phase 2 design section 4: Phase 2a = both paths
|
|
execute, readiness carried as an optional field, frontend ignores it until wired,
|
|
no cut-over date without Kisa. This plan IS Phase 2a. GROUNDED (doc read).
|
|
- Grade the CURRENT shipment only (latest DOS per line); prior shipments are collapsed
|
|
history (Phase 1 brief section 7; dedup.py:226, 238-245). GROUNDED.
|
|
|
|
---
|
|
|
|
## 3. Prerequisites / dependencies
|
|
|
|
1. Repo at or after the 2026-06-27 clean checkpoint; `python -m pytest tests/` green
|
|
(verified 108 passed on 2026-07-06).
|
|
2. Railway CLI authenticated; backend service `signal-api-production-91c2`. Deploy
|
|
command: `railway up --detach` from the project root (forces fresh build).
|
|
3. `SIGNAL_API_KEY` value available for curl-based live verification (X-API-Key header
|
|
path, main.py:117-118).
|
|
4. Decision from Kisa on Open Question 1 (plan_type CSV ingestion now vs with P5).
|
|
The wiring ships either way; the decision changes only whether "Clear to Ship" is
|
|
reachable end to end on day one.
|
|
5. No dependency on the Convex spike, Supabase schema changes, or the frontend.
|
|
Frontend is untouched in this step.
|
|
|
|
Downstream items explicitly NOT in this plan (owned by the P2-P6 agent; reference
|
|
only): rules-to-config citation IDs, synonym vocabularies to config, patient rollup,
|
|
device-keyed override, plan-type enforcement + guesser removal, frontend nesting.
|
|
|
|
---
|
|
|
|
## 4. Implementation approach
|
|
|
|
**Seam decision: the readiness verdict POST-PROCESSES (augments) the
|
|
coverage_calculator output. It does not wrap it and does not replace it.** Grounds:
|
|
|
|
- The two engines grade DIFFERENT axes. coverage_calculator = timing urgency
|
|
(CoverageFlag); readiness = documentation completeness (LineStatus). readiness.py's
|
|
own scope-boundary comment says the timing layer is "combined at the worklist
|
|
level" (readiness.py:139-142). Neither output is a superset of the other. GROUNDED.
|
|
- The Phase 2 design section 4 mandates coexistence (Phase 2a): both paths execute,
|
|
readiness rides as an optional field, old path still serves the worklist. GROUNDED.
|
|
- Wrapping inside coverage_calculator is impossible without a circular import
|
|
(section 2 above). GROUNDED.
|
|
- Replacing would change output cardinality (dedup collapses rows to lines) and break
|
|
the row-keyed frontend, `_compute_stats` (main.py:397-416), and the export echo
|
|
(main.py:559-620) in one step. Cardinality-preserving augmentation keeps this step
|
|
additive and reversible; the cardinality change belongs to P3 rollup + P6 frontend.
|
|
GRAFTED (design judgment, consistent with the locked migration strategy).
|
|
|
|
**How the augment works, concretely:** dedup the batch's ShipmentRecords into
|
|
CoverageLines, grade each line's CURRENT shipment with `evaluate_readiness`, then
|
|
attach each line's verdict to EVERY row-level RecordOut belonging to that line. Row
|
|
count, ordering, stats, legacy labels: all unchanged. Rows from prior shipments of a
|
|
line carry the same line verdict (graded from the latest DOS), which is the locked
|
|
grade-the-current-shipment rule surfacing at row level. GRAFTED (mechanism), built on
|
|
GROUNDED locks.
|
|
|
|
### Ordered steps
|
|
|
|
1. **Create `python-backend/core/worklist_readiness.py`** (new module, the only new
|
|
import surface). Two functions:
|
|
- `readiness_by_line(records, confirmed_visits) -> dict[tuple, ReadinessVerdict]`
|
|
keyed by `(patient_id, device_type, plan_type_id)` where `plan_type_id` is
|
|
`plan_type or "unknown"` (mirrors `CoverageLineKey`, dedup.py:85-90).
|
|
Internally: `dedup()` -> `assign_to_coverage_lines()` -> per line,
|
|
`evaluate_readiness()` on `line.current_shipment`'s csv_* fields, passing
|
|
`confirmed_visit_date` looked up by `sha256(patient_id)` from the
|
|
confirmed_visits dict (same keying as coverage_calculator.py:387-388).
|
|
- `readiness_for_record(record, confirmed_visit_date) -> ReadinessVerdict` for the
|
|
single-record Confirm Visit path (one record is its own line; call
|
|
`evaluate_readiness` directly).
|
|
- Log only hashed patient_ids (reuse the `_hash_pid` pattern, dedup.py:40-42).
|
|
2. **Add additive response fields in `api/main.py`:**
|
|
- New Pydantic model `ReadinessItemOut` (doc_type, required_state, input_state,
|
|
quality, value, satisfied).
|
|
- `RecordOut` gains `plan_type: Optional[str] = None`,
|
|
`readiness_status: Optional[str] = None`,
|
|
`readiness_items: Optional[list[ReadinessItemOut]] = None`. All default None so
|
|
old clients, the export echo (`ExportRequest.records: list[RecordOut]`,
|
|
main.py:559-561), and the frontend are unaffected.
|
|
- `UploadResponse` gains `readiness_stats: dict | None = None` (counts per
|
|
LineStatus value) so live verification is measurable without the UI.
|
|
3. **Wire the batch path** in `upload_csv` immediately after `calculate_batch`
|
|
(main.py:492-494): compute `verdicts = readiness_by_line(records,
|
|
confirmed_visits)`; when building each RecordOut (main.py:502-521), resolve the
|
|
row's original ShipmentRecord via the existing `record_lookup` (main.py:496-500),
|
|
derive its line key, and pass the matching verdict into `_to_record_out` (new
|
|
optional parameter). If the record lookup misses, readiness fields stay None,
|
|
exactly like `doc_state` degrades today (main.py:317-319).
|
|
4. **Wire the single-record path** in `confirm_visit`: add
|
|
`plan_type: str | None = None` to `ConfirmVisitRequest` (main.py:649-667), pass it
|
|
into the `ShipmentRecord` construction (main.py:735-750; the field already exists
|
|
on ShipmentRecord), compute `readiness_for_record(record,
|
|
confirmed_visit_date=confirmed)`, and pass it to `_to_record_out` at main.py:766.
|
|
The frontend already echoes RecordOut fields back on confirm-visit (RecordOut
|
|
comment, main.py:192-194); adding `plan_type` to RecordOut in step 2 makes the
|
|
round trip whole once the frontend echoes it (frontend change itself is P6; until
|
|
then the field arrives None and the recompute honestly reads Plan Type Needed).
|
|
5. **(Decision-gated, Open Question 1) Minimal plan_type ingestion in
|
|
`api/normalizer.py`:** add a `plan_type` entry to HEADER_MAP with aliases such as
|
|
`"plan_type", "plan type", "plan category", "coverage type", "insurance_type",
|
|
"insurance type"`. NEVER include bare `"plan"`, `"plan_name"`, or `"plan name"`
|
|
(those are payer aliases today, normalizer.py:46-50; stealing them would silently
|
|
re-map existing customer CSVs). Carry the value verbatim, lowercased and trimmed
|
|
only; `evaluate_readiness` already normalizes and routes anything outside the four
|
|
known values to Plan Type Needed (readiness.py:231-232, 47-49). This is
|
|
client-supplied mapping, not derivation from the payer name, so it does not
|
|
violate the never-guess lock. GRAFTED (recommendation; Kisa gates).
|
|
6. **Tests** (new file `tests/test_worklist_readiness.py` + API-level tests): see
|
|
section 8. Run the FULL suite; all 108 existing tests must still pass.
|
|
7. **Independent review:** hand the diff to code-auditor before ship (generator and
|
|
reviewer are never the same agent). Then run the `signal-e2e-test` skill so the
|
|
readiness verification is not a closed loop.
|
|
8. **Deploy and live-verify** per section 9. Then update
|
|
`docs/ship-status-ledger.md` and `context/current-state.md` (state-sync) and
|
|
commit. Do not update the pilot-readiness checklist unless an item genuinely
|
|
flips; count PASS items before quoting any percentage.
|
|
|
|
---
|
|
|
|
## 5. Files to touch
|
|
|
|
| File | Change |
|
|
|---|---|
|
|
| `python-backend/core/worklist_readiness.py` | NEW. Orchestration module: `readiness_by_line`, `readiness_for_record`. Imports dedup + readiness. Nothing imports it except api/main.py and tests. |
|
|
| `python-backend/api/main.py` | Add `ReadinessItemOut`; extend `RecordOut` (+3 optional fields) and `UploadResponse` (+1 optional field); extend `ConfirmVisitRequest` (+optional plan_type); new optional `readiness` param on `_to_record_out`; call `readiness_by_line` in `upload_csv` after line 492; call `readiness_for_record` in `confirm_visit` before line 766; small `_compute_readiness_stats` helper. |
|
|
| `python-backend/api/normalizer.py` | ONLY if Open Question 1 is approved: `plan_type` aliases in HEADER_MAP + pass-through into ShipmentRecord construction (normalizer.py:315-331) + it appears in `mapping_summary` automatically via the existing header loop. |
|
|
| `tests/test_worklist_readiness.py` | NEW. Unit tests for the orchestration module + API-shape tests (section 8). |
|
|
| `docs/ship-status-ledger.md` | Append the ship entry after live verification. |
|
|
| `context/current-state.md` | Update ACTIVE/NEXT + one author-stamped decision via state-sync. |
|
|
|
|
Files explicitly NOT touched: `core/coverage_calculator.py` (circular-import
|
|
constraint plus additive-only discipline), `core/dedup.py`, `core/readiness.py`
|
|
(already audited; do not modify while wiring), `core/doc_state_machine.py`,
|
|
`config/payer_rules.json`, anything in `signal-ui/` (P6), persistence/Supabase schema
|
|
(verdicts are recomputed per upload, not persisted, this step).
|
|
|
|
---
|
|
|
|
## 6. Integration points
|
|
|
|
### A. Batch path, `api/main.py:492` (`POST /api/upload`)
|
|
|
|
Today:
|
|
```
|
|
records (normalize_csv) ──> calculate_batch ──> CoverageResult per row ──> RecordOut per row
|
|
```
|
|
After wiring:
|
|
```
|
|
records ──┬─> calculate_batch ──────────────> CoverageResult per row ──┐
|
|
└─> readiness_by_line (dedup ──> lines ──> evaluate_readiness)┴─> RecordOut per row
|
|
+ readiness_status
|
|
+ readiness_items
|
|
+ plan_type
|
|
```
|
|
- Inputs already in scope at the call-site: `records` (main.py:457),
|
|
`confirmed_visits` (main.py:486, dict of `sha256(patient_id) -> date`).
|
|
- Row-to-line resolution: the existing `record_lookup` (main.py:496-500) maps a
|
|
CoverageResult back to its ShipmentRecord; the line key is
|
|
`(record.patient_id, record.device_type, record.plan_type or "unknown")`.
|
|
- Legacy outputs (`flag`, `status_label`, `priority_score`, sort order, `stats`,
|
|
row count) are byte-identical to before. The verdict rides alongside.
|
|
|
|
### B. Single-record path, `api/main.py:751` (`POST /api/confirm-visit`)
|
|
|
|
- `calculate_coverage(record, confirmed_visit_date=confirmed)` stays exactly as is
|
|
(timing axis).
|
|
- Add `verdict = readiness_for_record(record, confirmed_visit_date=confirmed)` and
|
|
pass it to `_to_record_out(result, record=record, confirmed_visit_date=confirmed,
|
|
readiness=verdict)` at main.py:766.
|
|
- `evaluate_readiness` counts a confirmed visit as positive evidence
|
|
(readiness.py:135-144), so confirming a visit flips the visit item to satisfied in
|
|
the same response, no second upload needed.
|
|
|
|
### C. What plugs into what (function-level contract)
|
|
|
|
- `evaluate_readiness` consumes exactly the fields `MergedShipment` carries
|
|
(csv_swo_status, csv_visit_date, csv_pecos_verified, csv_pa_status,
|
|
csv_diagnosis_on_file, plan_type) plus `confirmed_visit_date`. Verified
|
|
field-by-field against dedup.py:64-83 and readiness.py:216-226. GROUNDED.
|
|
- `CoverageLine.plan_type == "unknown"` feeds straight into `evaluate_readiness`,
|
|
which routes it to Plan Type Needed because "unknown" is not in KNOWN_PLAN_TYPES
|
|
(readiness.py:47-49, 231-232; tested by
|
|
`test_plan_type_unknown_string_is_plan_type_needed`). GROUNDED.
|
|
|
|
---
|
|
|
|
## 7. Skeleton snippets (illustrative only, do NOT paste blindly; re-verify line numbers at execution time)
|
|
|
|
> **EXECUTION CORRECTION (2026-07-07, found in adversarial review):** the sketches
|
|
> below key row-to-verdict resolution off `rec.plan_type` (the row's OWN value).
|
|
> That is WRONG when a row's plan_type differs from its dedup group's MERGED
|
|
> plan_type (conflicting groups merge to None) — it can attach another line's
|
|
> verdict, a reachable false green. The shipped implementation resolves rows
|
|
> through the dedup group key `(patient_id, device_type, DOS-as-date)` via
|
|
> `ReadinessIndex` in `core/worklist_readiness.py`. Follow the shipped code,
|
|
> not these sketches.
|
|
|
|
### `core/worklist_readiness.py` (new)
|
|
|
|
```python
|
|
"""Orchestrates dedup + readiness for the live pipeline (Phase 2a coexistence).
|
|
PHI CONTRACT: patient_id only; hash it in any log line."""
|
|
import hashlib
|
|
from datetime import date
|
|
from typing import Optional
|
|
|
|
from core.coverage_calculator import ShipmentRecord
|
|
from core.dedup import dedup, assign_to_coverage_lines, UNKNOWN_PLAN_TYPE
|
|
from core.readiness import evaluate_readiness, ReadinessVerdict
|
|
|
|
LineKey = tuple[str, str, str] # (patient_id, device_type, plan_type_id)
|
|
|
|
|
|
def _confirmed_for(patient_id: str, confirmed_visits: dict[str, date]) -> Optional[date]:
|
|
return confirmed_visits.get(hashlib.sha256(patient_id.encode()).hexdigest())
|
|
|
|
|
|
def readiness_for_record(record: ShipmentRecord,
|
|
confirmed_visit_date: Optional[date] = None) -> ReadinessVerdict:
|
|
return evaluate_readiness(
|
|
record.plan_type,
|
|
csv_swo_status=record.csv_swo_status,
|
|
csv_visit_date=record.csv_visit_date,
|
|
confirmed_visit_date=confirmed_visit_date,
|
|
csv_pecos_verified=record.csv_pecos_verified,
|
|
csv_pa_status=record.csv_pa_status,
|
|
csv_diagnosis_on_file=record.csv_diagnosis_on_file,
|
|
)
|
|
|
|
|
|
def readiness_by_line(records: list[ShipmentRecord],
|
|
confirmed_visits: Optional[dict[str, date]] = None,
|
|
) -> dict[LineKey, ReadinessVerdict]:
|
|
confirmed_visits = confirmed_visits or {}
|
|
lines = assign_to_coverage_lines(dedup(records))
|
|
verdicts: dict[LineKey, ReadinessVerdict] = {}
|
|
for key, line in lines.items():
|
|
cur = line.current_shipment # grade the CURRENT shipment only (locked)
|
|
verdicts[(key.patient_id, key.device_type, key.plan_type_id)] = evaluate_readiness(
|
|
line.plan_type if line.plan_type != UNKNOWN_PLAN_TYPE else None,
|
|
csv_swo_status=cur.csv_swo_status,
|
|
csv_visit_date=cur.csv_visit_date,
|
|
confirmed_visit_date=_confirmed_for(key.patient_id, confirmed_visits),
|
|
csv_pecos_verified=cur.csv_pecos_verified,
|
|
csv_pa_status=cur.csv_pa_status,
|
|
csv_diagnosis_on_file=cur.csv_diagnosis_on_file,
|
|
)
|
|
return verdicts
|
|
```
|
|
|
|
### `api/main.py` additions (sketch)
|
|
|
|
```python
|
|
class ReadinessItemOut(BaseModel):
|
|
doc_type: str
|
|
required_state: str # REQUIRED | NOT_REQUIRED | NOT_EVALUATED
|
|
input_state: str # SUPPLIED | ABSENT
|
|
quality: str # GOOD | PENDING | BAD | NONE
|
|
value: Optional[str] = None
|
|
satisfied: bool
|
|
|
|
# RecordOut gains (all additive, default None):
|
|
# plan_type: Optional[str] = None
|
|
# readiness_status: Optional[str] = None # one of the 5 LineStatus values, verbatim
|
|
# readiness_items: Optional[list[ReadinessItemOut]] = None
|
|
# UploadResponse gains:
|
|
# readiness_stats: dict | None = None
|
|
```
|
|
|
|
```python
|
|
# upload_csv, right after main.py:492-494:
|
|
from core.worklist_readiness import readiness_by_line
|
|
verdicts = readiness_by_line(records, confirmed_visits)
|
|
|
|
# inside the RecordOut build loop: resolve rec = record_lookup.get(...) as today, then
|
|
line_verdict = None
|
|
if rec is not None:
|
|
line_verdict = verdicts.get(
|
|
(rec.patient_id, rec.device_type, rec.plan_type or "unknown"))
|
|
# pass readiness=line_verdict into _to_record_out; inside _to_record_out:
|
|
# readiness_status=verdict.line_status.value if verdict else None,
|
|
# plan_type=(getattr(record, "plan_type", None) if record else None),
|
|
# readiness_items=[ReadinessItemOut(..., required_state=i.required_state.value, ...)
|
|
# for i in verdict.doc_items] if verdict else None,
|
|
```
|
|
|
|
```python
|
|
# confirm_visit, before main.py:766:
|
|
from core.worklist_readiness import readiness_for_record
|
|
verdict = readiness_for_record(record, confirmed_visit_date=confirmed)
|
|
return _to_record_out(result, record=record, confirmed_visit_date=confirmed,
|
|
readiness=verdict)
|
|
# and ConfirmVisitRequest gains: plan_type: str | None = None
|
|
# passed into the ShipmentRecord(... plan_type=body.plan_type) construction.
|
|
```
|
|
|
|
---
|
|
|
|
## 8. Test strategy & acceptance criteria
|
|
|
|
New file `tests/test_worklist_readiness.py`. The engine-level behaviors are already
|
|
covered (57 unit tests + 35 E2E); the new tests target the WIRING, i.e. the seams
|
|
this plan adds. GROUNDED engine coverage, GRAFTED wiring tests to write:
|
|
|
|
1. **False green is blocked at the API layer (the headline acceptance case).**
|
|
One record, `plan_type="medicare"`, SWO "On File", csv_visit_date present,
|
|
diagnosis "Yes", **PECOS absent (None)**, shipment recent enough that the legacy
|
|
flag reads RESUPPLY_READY (legacy `status_label` "Clear to Ship" per FLAG_LABELS,
|
|
main.py:143). Assert: `readiness_status == "Action Needed"` on the same RecordOut.
|
|
This is the exact false-green the model exists to kill, visible in one response:
|
|
legacy label green-ish, readiness verdict honest. (Engine-level twin:
|
|
`test_required_absent_blocks_green`, tests/test_readiness.py:67.)
|
|
2. **Plan Type Needed.** Same record with `plan_type=None` (today's default for every
|
|
normalized record). Assert `readiness_status == "Plan Type Needed"` and PECOS/PA
|
|
items are NOT_EVALUATED while universal items still grade. (Engine twins:
|
|
readiness tests at lines 128-146.)
|
|
3. **Clear to Ship is reachable.** `plan_type="medicare"`, all five items with
|
|
positive evidence (PA not required for Medicare FFS per payer_rules.json:36).
|
|
Assert `readiness_status == "Clear to Ship"`. If Open Question 1 is approved, also
|
|
prove it END TO END through `normalize_csv` with a plan_type column; if not
|
|
approved, prove it at the `readiness_by_line` level with constructed records and
|
|
note the end-to-end gap explicitly.
|
|
4. **Dedup wiring.** Two CSV rows, same (patient, device, DOS), different
|
|
order_numbers -> one line verdict; both output rows (if both survive to output)
|
|
carry the identical verdict. Two rows same patient+device with DIFFERENT DOS
|
|
(a resupply history) -> still ONE line; assert the verdict is graded from the
|
|
LATEST DOS row's csv fields (change a doc field between the two rows to prove
|
|
which one drove the verdict).
|
|
5. **Confirm Visit recompute.** Record with plan_type medicare, everything good
|
|
except no visit evidence -> upload path gives "Action Needed"; POST
|
|
/api/confirm-visit with a valid confirmed date -> response
|
|
`readiness_status == "Clear to Ship"` and the visit item satisfied.
|
|
6. **Back-compat invariants.** For a fixture CSV, the wired response equals the
|
|
pre-wiring response on every legacy field: same row count, same order, same
|
|
`flag`/`status_label`/`priority_score`/`stats`. Readiness fields are the only
|
|
additions. Also: `readiness_stats` totals equal the number of distinct graded
|
|
lines' rows accounted for (define precisely in the test).
|
|
7. **PHI check.** Assert no raw patient_id appears in log output captured from
|
|
`readiness_by_line` (caplog); hashed only.
|
|
8. **Full suite.** `python -m pytest tests/` must show 108 + new tests passed, zero
|
|
failures, before deploy.
|
|
|
|
**Acceptance criteria (all must hold):** cases 1 and 2 pass exactly as stated (these
|
|
prove the false-green kill and the never-guess doctrine in the live pipeline); case 6
|
|
proves the step was truly additive; full suite green; code-auditor sign-off recorded.
|
|
|
|
---
|
|
|
|
## 9. Live-verify steps (drive the real app, measure, never eyeball)
|
|
|
|
Backend-only change, so verification is API-level against the live Railway service.
|
|
The UI renders nothing new by design (Phase 2a: frontend ignores the new fields).
|
|
|
|
1. Deploy: `railway up --detach` from `/Users/sttil-solutions/projects/signal`.
|
|
Watch build logs (`railway logs`) until healthy.
|
|
2. `curl -s https://signal-api-production-91c2.up.railway.app/health` -> `status: ok`.
|
|
3. Upload a synthetic fixture CSV (use the signal-e2e-test skill's generator; NEVER
|
|
real data):
|
|
`curl -s -X POST .../api/upload -H "X-API-Key: $SIGNAL_API_KEY" -F "file=@fixture.csv"`.
|
|
Measure, do not skim:
|
|
- every record has a `readiness_status` key;
|
|
- with no plan_type column: 100% of graded records read "Plan Type Needed"
|
|
(count them with jq, compare to total);
|
|
- `readiness_stats` present and its counts sum correctly.
|
|
4. If plan_type ingestion shipped (Open Question 1): upload the fixture WITH a
|
|
plan_type column containing one engineered Clear to Ship row (medicare, all
|
|
evidence positive) and one engineered false-green row (medicare, PECOS blank,
|
|
timing green). Assert with jq: the first reads "Clear to Ship", the second reads
|
|
"Action Needed" while its legacy `status_label` still shows the legacy green
|
|
label. This is the false-green kill, observed live.
|
|
5. Confirm Visit round trip: POST `/api/confirm-visit` for the Action Needed
|
|
patient's visit case (test 5 shape) and assert the returned record's
|
|
`readiness_status` flips as specified.
|
|
6. Regression: re-run the exact pre-wiring fixture and diff the response against a
|
|
saved pre-deploy response, ignoring only the new keys. Zero legacy diffs allowed.
|
|
7. `railway logs` spot check: any new log lines show hashed patient_ids only.
|
|
8. Record results (counts, not adjectives) in `docs/ship-status-ledger.md`.
|
|
|
|
---
|
|
|
|
## 10. Constraints & gotchas
|
|
|
|
- **PHI (non-negotiable):** patient_id is the sole crosswalk key. `DocItem.value`
|
|
carries only client status strings or ISO dates, never identity fields. Never add
|
|
name/DOB/SSN/contact anywhere, including test fixtures. Hash patient_id in every
|
|
new log line (`_hash_pid` pattern, dedup.py:40-42).
|
|
- **Locked labels:** `readiness_status` emits the `LineStatus` values verbatim: At
|
|
Risk, Action Needed, On Track, Clear to Ship, plus Plan Type Needed for unknown
|
|
plans. Never invent a sixth, never rename, and never mix in the resupply-lifecycle
|
|
set (Supply Lapsed, Renewal Due, Resupply Ready, Active, Outreach Worklist), which
|
|
is a separate locked vocabulary for the pilot guide and copy. The legacy
|
|
FLAG_LABELS map (main.py:136-146) stays untouched until P6.
|
|
- **Locked dedup key:** `(patient_id, device_type, date_of_service)`. order_number is
|
|
display-only. Do not "improve" the key while wiring.
|
|
- **Circular import:** `coverage_calculator.py` must never import
|
|
`dedup`/`readiness`/`worklist_readiness` (dedup.py:35 imports FROM it). The new
|
|
module is only imported by `api/main.py` and tests.
|
|
- **`_load_rules` cache hazard:** `@lru_cache` returns the live mutable dict
|
|
(readiness.py:99-102). Tests must never mutate it (documented hazard, Phase 2
|
|
design section 2B).
|
|
- **Normalizer alias collision:** "plan", "plan_name", "plan name" mean PAYER today
|
|
(normalizer.py:46-50). plan_type aliases must not include them.
|
|
- **Language:** "resupply", never "refill", in any new copy, field names, or test
|
|
names. (`refill_window_days` in payer_rules.json is an existing code key; leave it,
|
|
renaming config keys is not this step.)
|
|
- **Prior-shipment rows carry the current shipment's verdict.** A six-month history
|
|
upload shows N legacy rows for one line, all with the same line verdict graded from
|
|
the latest DOS. By design (grade the current shipment only). Expect it; do not
|
|
"fix" it here; the row collapse is P3/P6.
|
|
- **Verdicts are not persisted.** `persist_upload` (main.py:538-546) stores shipment
|
|
records and coverage results only. Readiness is recomputed per request this phase.
|
|
Persisting verdicts is a schema decision that goes through lead-sttil's data
|
|
handling check first.
|
|
- **Deploy mechanics:** backend `railway up --detach` (fresh build; `railway
|
|
redeploy` only reuses the old image and will NOT pick up code changes). Frontend is
|
|
untouched this step; if it ever needs a deploy, Vercel is manual (`vercel --prod`,
|
|
the GitHub webhook is broken).
|
|
- **Auth on verify:** use the X-API-Key header path; Clerk JWT is the frontend path.
|
|
|
|
---
|
|
|
|
## 11. Definition of Done
|
|
|
|
All of the following, in order, with evidence recorded:
|
|
|
|
1. `core/worklist_readiness.py` exists; imports of dedup/readiness in
|
|
`python-backend/` are exactly: worklist_readiness (plus tests).
|
|
2. Both call-sites wired: `/api/upload` (batch, main.py:492 region) and
|
|
`/api/confirm-visit` (single, main.py:751 region) return readiness fields.
|
|
3. Full test suite green (108 existing + new wiring tests), run and count recorded,
|
|
including acceptance cases 1 (false green blocked) and 2 (Plan Type Needed).
|
|
4. Back-compat invariant proven (section 8 case 6): legacy response fields
|
|
byte-identical for a fixture CSV.
|
|
5. code-auditor review passed (independent agent, findings resolved or accepted).
|
|
6. signal-e2e-test skill run against the wired pipeline, results recorded.
|
|
7. Deployed via `railway up --detach`; all live-verify steps in section 9 pass with
|
|
measured counts, not impressions.
|
|
8. `docs/ship-status-ledger.md` entry appended; `context/current-state.md` updated
|
|
via state-sync with one author-stamped decision line.
|
|
9. No production change outside the files listed in section 5; no PHI fields added;
|
|
no label invented; dedup key untouched.
|
|
|
|
---
|
|
|
|
## 12. Open questions / decisions needed from Kisa
|
|
|
|
1. **Plan-type CSV ingestion now, or wait for P5?** (The single biggest call.)
|
|
Without it, every live verdict reads "Plan Type Needed" on day one: honest and
|
|
doctrine-correct, but no green is reachable end to end and the new field looks
|
|
inert. My recommendation: include the minimal verbatim column mapping (section 4
|
|
step 5) in this wiring. It is a client-supplied mapped field, not a guess, so it
|
|
honors the lock; the full enforcement, guesser removal, and mapping-review UX stay
|
|
with P5's owner. GRAFTED recommendation; Kisa gates.
|
|
2. **Confirm the seam:** readiness AUGMENTS the coverage result (Phase 2a
|
|
coexistence), it does not replace the worklist scoring yet. This follows Pi's
|
|
locked migration strategy, but the wiring makes it real, so I want it said out
|
|
loud and confirmed.
|
|
3. **`readiness_stats` in the upload response now?** Cheap, additive, makes live
|
|
verification measurable. Recommendation: yes. GRAFTED.
|
|
4. **Cut-over timing** (Phase 2b/2c: frontend switches, old path retired) remains
|
|
entirely Kisa's decision per the Phase 2 design, section 4. Nothing in this plan
|
|
pre-commits it.
|
|
5. **State-file correction:** current-state.md says "73 tests green" (2026-06-27
|
|
entry); today's grounded counts are 30 + 27 unit, 35 E2E, 108 total. Approve
|
|
recording the corrected counts at the next state-sync so no one re-quotes 73.
|