fix: engine-dropped rows surface in the skipped list (failed-data honesty)

Kisa ruling 2026-07-07: failed CSV info must be visible and Signal never acts
on rows whose required information was not provided. Rows that survive
normalization but die in the scoring engine (no coverage rule for the
component) previously vanished from the response; they now append to
skipped_reasons before persistence so the stored count matches what the
supplier saw. Auditor SHIP; its MEDIUM (persist-order) fixed in-commit.
Known bound (auditor LOW, tracked): a dropped component inside an otherwise
scored same-DOS group is covered by the scored line and not separately
reported. 169 tests green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Kisa 2026-07-07 06:15:27 -04:00
parent f317bb5042
commit b4f5707937
2 changed files with 27 additions and 0 deletions

View file

@ -858,6 +858,27 @@ async def upload_csv(
) )
) )
# Failed-data honesty (Kisa 2026-07-07): a row the scoring engine drops
# (e.g. a component with no coverage rule) must appear in the skipped
# list, never vanish — and BEFORE persistence so the stored batch record
# matches the count the supplier saw. Signal never acts on rows whose
# required information was not provided; it also never hides them.
from core.dedup import _as_date as _asd
processed_groups = {
(r.patient_id, r.device_type, _asd(r.last_shipment_date))
for r in results
}
_reported_drops = set()
for rec in records:
g = (rec.patient_id, rec.device_type, _asd(rec.shipment_date))
if g not in processed_groups and g not in _reported_drops:
_reported_drops.add(g)
skipped_reasons.append(
f"{rec.patient_id}: {rec.device_type} ({rec.component}) has "
"no coverage rule for this component — row not scored"
)
# Use real user ID and db_conn where possible if we implement it, hardcoding # Use real user ID and db_conn where possible if we implement it, hardcoding
# IP as 0.0.0.0 for now unless req object available # IP as 0.0.0.0 for now unless req object available
from core.supabase_client import get_client from core.supabase_client import get_client

View file

@ -513,6 +513,12 @@ def test_upload_skipped_record_leaves_no_phantom_line_in_stats():
pids = {r["patient_id"] for r in body["records"]} pids = {r["patient_id"] for r in body["records"]}
assert pids == {"PT-OK-1"} # transmitter row skipped by the engine assert pids == {"PT-OK-1"} # transmitter row skipped by the engine
assert body["readiness_stats"]["lines_total"] == 1 assert body["readiness_stats"]["lines_total"] == 1
# Failed-data honesty: the dropped row is visible in the skipped list,
# never silently vanished (Kisa ruling 2026-07-07).
assert body["skipped"] >= 1
assert any(
"PT-TX-1" in r and "not scored" in r for r in body["skipped_reasons"]
)
def test_upload_non_gradeable_rows_carry_no_verdict(): def test_upload_non_gradeable_rows_carry_no_verdict():