Layer 5

Forensics Lab

Layer 5: mechanistic root cause analysis for governed inference failures.

When a response is blocked, unstable, or unsupported, Forensics Lab explains why at the mechanism level and validates remediations before they are promoted.

Without this

Without mechanistic root-cause analysis, governance failures repeat. You know something broke but not why, so fixes are guesses. The same failure mode surfaces next week with a different prompt.

Position in the platform

System	Layer	Function
Oracle Foundry	Foundation	Produces the authoritative, tamper-evident source base
SIRE Crosswalk	Post-Foundry	Provides routing graph and gap map used in boundary-violation diagnosis
Prompt Compiler	L0	Prevents structural prompt failures before generation
Claim Ledger	L1-L4	Per-claim confidence, entailment, stability, and uncertainty scoring
Process Control System	Cross-layer	Detects statistical degradation and special-cause variation
Forensics Lab	L5	Diagnoses causal failure mechanisms and validates remediation

Diagnostic methods

Representation ablation

Model neurosurgery

Dampens specific directions to test whether a claim was driven by parametric memory or retrieved evidence.

Operates on: Hidden-state directions

Self-hosted required: Yes

Mechanism competition tracing

Model neurosurgery

Finds the crossover layer where parametric signal starts to dominate retrieval signal.

Operates on: Layer-by-layer activations

Self-hosted required: Yes

Context ablation

Input manipulation

Runs leave-one-out and chunk-swap diagnostics to separate retrieval failure from source-gap failure.

Operates on: Retrieved chunks

Self-hosted required: No

Counterfactual arbitration

Diagnosis validation

Applies targeted dampening, regenerates, and verifies whether the corrected output now passes entailment.

Operates on: Modified forward pass

Self-hosted required: Yes

Investigation flow

Phase 1

Triage

Classifies the trigger and selects an investigation path from claim-level failure to population-level drift.

Phase 2A

Context ablation

Tests chunk causality, retrieval ranking quality, and source coverage gaps.

Phase 2B

Oracle routing analysis

Resolves authority-boundary failures through cross-oracle routing or source expansion.

Phase 2C

Neurosurgery + arbitration

Runs ablation and tracing, then validates diagnosis through counterfactual regeneration.

Phase 2D

Population analysis

Samples affected envelopes after SPC signals to isolate systematic failure mechanisms.

Outputs and feedback loops

• Forensics report with complete evidence chain from trigger through validated fix
• Compiler feedback with query-class tags and crossover-layer guidance
• Oracle gap report with source-ingestion recommendations
• Calibration feedback for miscalibrated claim and token classes
• SPC baseline update after remediation

Availability by runtime mode

Full governance (Clean Room profile)

Full forensics

Context ablation with regeneration, hidden-state neurosurgery, and counterfactual arbitration.

Standard governance (Refinery profile)

Partial forensics

Context ablation with regeneration, without hidden-state neurosurgery.

Lite governance (Studio profile)

Diagnostics only

Rescore-only diagnostics. Regeneration and hidden-state methods are intentionally excluded.

Live forensics report

Actual ForensicsReportV1 from a HIPAA evidence-gap investigation with remediation targets.

{  "schema_version": "forensics_report_v1",  "job_id": "fj_8a3c1e92-d4f7-4b01-a6e8-7f2c09b31d54",  "envelope_id": "env_20250228_0742_hipaa_q",  "mode": "refinery",  "triggered_by": ["gate_failure"],  "trigger_reasons": [    {      "trigger_id": "tr_gate_fail",      "matched": true,      "reason": "emission gate outcome: any_fail",      "signal_value": "any_fail",      "threshold": "all_pass"    }  ],  "triage": {    "failure_signals": [      "grounding_required",      "no_evidence"    ],    "probable_root_cause": "evidence_gap",    "recommended_phase": "2A",    "confidence": 0.84,    "failed_claim_summary": {      "total_claims": 6,      "failed_ids": ["clm_03", "clm_05"],      "failure_labels": ["unsupported", "unsupported"]    }  },  "envelope_snapshot": {    "gate_outcome": "any_fail",    "gate_blocked": false,    "gate_failed_rules": ["grounding_required", "no_evidence"],    "oracle_chunk_count": 3,    "web_source_count": 0,    "model_id": "gpt-4o-2024-08-06",    "compiler_version": "cfpo_v2.1"  },  "investigation_status": "completed",  "remediation_targets": [    {      "target": "oracle",      "severity": "high",      "owner_lane": "oracle_foundry",      "action": "Ingest HIPAA §164.524 amendment-rights guidance to close coverage gap for clm_03"    },    {      "target": "compiler",      "severity": "medium",      "owner_lane": "prompt_compiler",      "action": "Add evidence-gap acknowledgement directive for queries outside oracle coverage"    }  ]}

The signals behind the investigation

Forensics Lab doesn’t guess what went wrong. It reads inference-time observables — token confidence, entailment scores, regeneration stability, hidden-state volatility — to trace exactly where and why a claim failed.

See AI Neurosurgery →

Close the loop from failure to fix

Forensics Lab turns governance events into structural improvements across compiler, oracle, calibration, and process control.

See Architecture Oracle Foundry SIRE Crosswalk Prompt Compiler Claim Ledger Process Control System The Clean Room Check Your Risk

Who uses this

Operator

Governance engineers

Incident responders who investigate failures, run ablation diagnostics, and validate remediations.

Consumer

Four feedback systems

Compiler (prompt feedback), Oracle Foundry (gap reports), calibration systems (miscalibration fixes), and Process Control System (baseline updates).