Executive Summary
A language model is a probabilistic system wearing a deterministic-looking interface. Successive invocations with identical input can still produce different numeric extractions, different formatting, or in the worst case a different verdict - which is fatal for any compliance output meant to be independently reproduced by a third party. JIL Sovereign's deterministic verdict module does not try to make the model itself perfectly deterministic; instead, it bounds whatever the model produces against a floor computed entirely without the model.
The system invokes the language model under a fully-hashed fixed configuration - temperature zero, a fixed seed, a fixed system-prompt hash, a fixed model-identifier hash. Its response is parsed against a strict structured schema; every numeric field in that schema is compared against a deterministic floor independently computed by a non-language-model process (typically a direct database aggregation) over the same input data. If a field falls outside a configured tolerance of its floor, or parsing fails outright, the invocation is retried under the identical configuration up to a bounded count. If every retry fails, the verdict falls back to the deterministic floor with the language-model contribution explicitly marked as suppressed - a fallback that is itself a fully valid attestation product.
Problem Statement
Structured-output features (temperature zero, JSON-schema-constrained generation, fixed seeds) reduce a language model's stochasticity but do not eliminate the specific failure mode that matters for compliance verdicts: numeric drift. A model might correctly recognize a value but format it differently across runs, or - more seriously - emit a number that flatly contradicts ground truth already established by a deterministic query over the same data. Retry-on-parse-failure patterns common in LLM application frameworks catch schema violations but do nothing to catch a syntactically valid number that is simply wrong relative to an independently-known fact.
Self-consistency decoding (sample many times, take the majority) and critic-model verification (a second model checks the first) both attempt to improve reliability, but neither anchors the result to a deterministic, non-probabilistic ground truth, and neither produces a signed, third-party-replayable manifest of the invocation. For a verdict intended to survive adversarial replay under Claim 26's reproducibility manifest, the numeric portion needs a bound that does not itself depend on the model behaving consistently.
Why Existing Approaches Fall Short
- Structured output / schema validation alone: catches parse failures, not numerically-wrong-but-well-formed values.
- LangChain-style retry parsers: retry on parse failure only; no drift guard against an independent ground truth.
- Self-consistency decoding: majority-vote across N samples is still purely probabilistic and not bound to deterministic ground truth.
- Critic / verify-then-generate models: the verifier is itself an LLM, not a deterministic non-LLM check, and produces no signed replay manifest.
Technical Architecture
Pipeline Components
| Component | Function |
|---|---|
| Configuration loader | Reads provider, model, version, endpoint, temperature (required zero), fixed seed, and system-prompt text; hashes the full configuration. |
| Invocation engine | Emits the request under the loaded configuration; captures the verbatim response. |
| Schema parser | Strictly parses the response against a structured schema with named numeric, enumerated, and string fields; any absent or mistyped field is a parse failure. |
| Deterministic-floor computer | Computes, per numeric field, a floor via a non-LLM process over the same input data - typically a structured-query aggregation. |
| Tolerance evaluator | Compares each parsed numeric field to its floor against a configured absolute or relative tolerance. |
| Retry controller | Repeats the invocation, under identical configuration, up to a bounded count on parse failure or tolerance violation. |
| Fallback module | Produces a verdict from the deterministic floor with a suppressed-LLM-contribution annotation if the retry bound is exhausted. |
| Manifest writer | Records configuration hash, schema hash, floor results, tolerance spec, retry log, and outputs into the verdict-replay manifest. |
Retry and Fallback Logic
Each numeric field's schema declaration specifies whether the deterministic floor acts as a lower bound (model output must be at or above) or an upper bound (model output must be at or below), and the tolerance may be expressed as an absolute bound, a relative fraction, or a combination. A bounded retry count - typically three, capped at five in the preferred embodiment - exists specifically to catch transient stochastic anomalies that survive even a temperature-zero, fixed-seed configuration.
Composability
The configuration hash, schema hash, floor results, and retry log all feed directly into the seven-hash-class reproducibility manifest of Claim 26, and the model identity used in each invocation is separately sealed under Claim 32's model-card mechanism - together the three claims give a language-model-mediated verdict the same replay guarantees as a fully deterministic one.
Prior Art Differentiation
| Approach | Bounds numeric output vs. ground truth | Bounded retry | Documented fallback | Replayable manifest |
|---|---|---|---|---|
| Structured output (temp=0, schema) | No | No | No | No |
| LangChain retry parsers | No | Yes (parse only) | No | No |
| Self-consistency decoding | No (majority vote only) | No | No | No |
| Critic / verify-then-generate | Verifier is itself an LLM | Varies | No | No |
| JIL Sovereign | Yes, non-LLM floor | Yes | Yes, suppressed-contribution flag | Yes |
The defensible combination is the deterministic-floor-plus-tolerance-evaluator pairing: no prior art bounds language-model numeric output against an independently-computed, non-LLM floor with a documented, still-valid fallback verdict when the bound cannot be met.