Report
Trace every structural recovery event.
Bible Bench does not penalize the model for structural failures. Instead, it records the full recovery timeline for each chapter evaluation. This accounts for some of the runtime execution spread. If the structured output requested is not structurally correct, it will automatically retry the chapter with a second attempt. If that fails, we fall back to the verse-level fallback tier. Every effort is made to fetch the chapter from the LLM in its entirety.
The Retry Traces Report documents the full structural recovery timeline for every chapter evaluation that required repair or verse-level fallback. Understand why structural failures occurred, how the retry system responded, and whether recovery produced reliable verse results or introduced new scoring anomalies.
What the Retry Traces Report surfaces
Structural recovery analysis across all three retry tiers — success rates, error classification, latency impact, and unrecovered failure counts.
Retry Success Rate
Percentage of structurally failed chapter evaluations that achieved successful recovery through chapter repair or verse-level fallback tiers.
Error Classification Breakdown
Distribution of structural failure types across all retried chapters: ALIGNMENT_DRIFT, VERSE_SET_INVALID, VERSE_MERGE_DETECTED, and VERSE_SPLIT_DETECTED.
Average Recovery Latency
Mean additional time required for chapters needing repair or verse-level fallback — quantifying the performance cost of structural recovery.
Unrecovered Failures
Count of chapter evaluations that exhausted all retry tiers without successful recovery, requiring manual review or re-evaluation.
Retry traces in action
Three views covering the full retry lifecycle — from chapter-level timeline to error distribution to individual verse recovery detail.
See a sample retry traces report
Download a sample PDF to preview the recovery timeline format and error classification breakdowns this report provides.
Understand every structural recovery in your evaluations.
Join the waitlist for early access and trace retry events across all your scripture evaluation runs with full timeline visibility.