Executive Summary
Healthcare claims and remittance data move between payers and providers as X12 837 (claim) and X12 835 (remittance advice) EDI files - a decades-old standard that carries member-identifying information in cleartext by design. JIL Sovereign's ingestion pipeline processes both transaction sets through a single path that tokenizes member identifiers the instant a file is parsed, before any field is written to a data store, and treats reconciliation of the 835's own declared payment control total against its line-level detail as a hard precondition to promoting that remittance into a usable compliance finding.
Only claims that survive both gates - tokenized at ingest and control-total reconciled - are promoted into sealed findings that the downstream detection engine can consume. A file that fails reconciliation is quarantined rather than silently ingested with a corrupted or truncated payment total.
Problem Statement
Conventional EDI ingestion pipelines typically store the parsed claim and remittance data first, with de-identification or tokenization applied downstream as a separate batch process - leaving a window in which cleartext member identifiers sit at rest in a database. Separately, remittance reconciliation against the payer's own declared control total is often a manual or spot-check activity rather than a systematic gate: a truncated or partially-delivered 835 file can be ingested as if it were complete, silently distorting every downstream recovery or denial calculation built on top of it.
Why Conventional EDI Ingestion Falls Short
- Store-then-de-identify pipelines: cleartext PHI is written first and scrubbed in a later batch job, leaving an at-rest exposure window.
- Manual remittance reconciliation: control totals are checked by a human, periodically, rather than enforced as a per-file precondition before any data is usable downstream.
- Generic ETL tooling: parses and loads EDI files without any domain-specific attestation gate tied to the 835's own BPR control total.
- Non-idempotent ingestion: re-delivery of the same interchange produces duplicate findings rather than being deduplicated at ingest.
Technical Architecture
Pipeline Stages
| Stage | What Happens |
|---|---|
| 1. Ingest | X12 837/835 file received and parsed into structured records |
| 2. Tokenize | Member-identifying fields replaced with tokens at parse time, before any write to storage; no cleartext PHI persists |
| 3. Deduplicate | Composite key of interchange control number + claim control number checked; re-delivered interchanges are dropped, not re-ingested |
| 4. Reconcile (835 only) | Declared BPR control total checked against the sum of line-level claim payment amounts in the same file |
| 5. Promote | Only tokenized, deduplicated, reconciled records are promoted into sealed findings consumable by the compliance-detection engine |
The Control-Total Precondition
A remittance file that fails reconciliation - its declared control total does not match the sum of its own line items - is not silently ingested with a flagged discrepancy; it is blocked from promotion entirely. This treats the payer's own declared total as an attestation the pipeline is entitled to hold the file to, rather than a figure to be trusted and reconciled later, after downstream calculations have already been built on top of it.
Idempotency
Because deduplication runs on the composite interchange-control-number-plus-claim-control-number key at ingest time, the same file delivered twice - a common occurrence in clearinghouse retry behavior - produces exactly one set of findings rather than duplicated recovery or denial calculations.
Downstream Consumption by the Detection Engine
A sealed finding produced by this pipeline is the unit of work the compliance-detection engine actually consumes - tokenized, reconciled, and free of cleartext PHI. This means the boundary between raw EDI ingestion and the detection/attestation layer is structural: the detection engine has no code path that ever receives an unreconciled remittance or an untokenized member identifier, because neither can reach promotion without first clearing both gates.
Prior Art Differentiation
| Approach | Tokenization Timing | Control-Total Check | Idempotent? |
|---|---|---|---|
| Store-then-de-identify batch pipeline | Downstream, after storage | Manual or absent | Depends on implementation |
| Generic EDI clearinghouse ETL | Not domain-specific | Not enforced as a precondition | Often not |
| Payer-side proprietary ingestion (e.g. large claims-processing platforms) | Varies, typically not ingest-time | Internal reconciliation, not a promotion gate | Varies |
| JIL Sovereign EDI-to-sealed-finding pipeline | At ingest, before any storage write | Hard precondition to promotion | Yes - composite key dedup |