Failure-Aware Long-Form Translation: Design and Implementation of a Recoverable LLM Translation System

Technical Summary: Failure-Aware Long-Form Translation

Problem Statement
The paper addresses a critical gap in long-form machine translation systems: the distinction between API-level success and application-level success. A translation request may return a syntactically valid, non-empty stream from a provider yet produce an unusable result. Common failure modes include outputs that are empty, truncated, dominated by source text or prompt leakage, or interrupted after generating only partial content. Conventional "failed-request" abstractions fail to capture these nuances; they either expose unusable or sensitive internal text to the user or discard expensive, partially valid work. In long documents, these errors amplify costs (via full restarts) and introduce alignment risks (via blind retention of corrupted suffixes).

Methodology
The author designed and implemented a recoverable translation system for heterogeneous inputs (manual text, URLs, files, PDFs, OCR) and diverse provider APIs. The core methodology relies on a guarded streaming architecture and a bounded recovery protocol.

1. Guarded Streaming and Containment:

   • Leading-Window Guard: Before any text is visible to the user, the system buffers the first 64 characters. During this window, it checks for prompt markers and source-language residue. If a failure is detected, the attempt is aborted before release.
   • State Distinction: The system maintains three distinct states for text: Buffered (not visible), Visible (passed the leading guard but subject to replacement if full-output checks fail), and Committed (passed all checks and recorded).
   • Boundary-Safe Retention: If a stream is interrupted after release, the system does not simply restart. Instead, it derives a "boundary-safe" prefix from the source and received output. This prefix retains only complete paragraphs and sentence-like units that can be structurally aligned to the source, discarding unfinished fragments.

2. Operational Failure Matrix:
   The system utilizes a matrix of failure conditions (transport errors, quota limits, policy filters, empty outputs, source residue, and catastrophic omissions). Detection combines protocol checks with implementation-specific heuristics (e.g., language-specific residue detection for Japanese and Korean).

   • Omission Guard: Specifically targets "silent omissions" where a model returns a fluent ending while dropping most input paragraphs. This is triggered only when paragraph retention is below 0.70 and the output-to-source character ratio is below 0.15.

3. Routing and Fallback:

   • Stable Model Order: Recovery follows a deterministic, session-specific order of models. Each model is attempted at most once per chunk to prevent infinite loops.
   • Layered Fallback: Once the LLM model pool is exhausted or a shared deadline (240 seconds total, with specific caps for streaming and fallback layers) is reached, the system enters a fallback path. This includes compatibility models, external machine translation, and an emergency machine translation path.
   • Provenance: All fallbacks are marked with typed Server-Sent Events (SSE) and client labels, ensuring the user knows when a machine translation (non-LLM) is used versus an LLM attempt.

Key Contributions
The paper presents three primary contributions:

1. Streaming Architecture: A design that distinguishes between buffered, visible, boundary-safe, and committed text during active recovery.
2. Operational Failure Matrix: A bounded routing policy derived from a deployed implementation, categorizing failures by detection confidence and action safety.
3. Sanitized Executable Artifact: A provider-neutral companion artifact implementing the protocol, passing 38 public tests, and including fixed cases for release, retention, attempts, events, and provenance.

Results
The evaluation focuses on the system's ability to execute control flow decisions rather than translation quality or failure prevalence.

• Fixed Case Testing: The artifact successfully reproduced all 14 configured completion labels.
• Containment: In four early-invalid prefix cases, the leading window prevented 235 characters of invalid content from becoming visible.
• Recovery: In four interrupted stream scenarios, the boundary-safe mechanism retained 31 characters that a full restart would have discarded, while correctly aligning them to the source.
• Protocol Compliance: Two end-to-end scenarios confirmed zero violations of attempt bounds, resume events, or provenance rules.
• Limitations of Results: The author explicitly states these are executable checks of the published control flow. The study does not estimate detector recall, false-positive rates, or translation quality on naturally occurring outputs.

Significance and Claims
The paper claims that for long-form translation, "generating text cannot be the same as committing it." Its significance lies in treating release, recovery, and fallback as separate, observable decisions rather than a monolithic success/failure binary.

• Operational Safety: The system ensures that recovery remains visible (via typed events) rather than silently changing the origin of a translation.
• Cost and Latency Management: By retaining boundary-safe prefixes, the system avoids the latency and cost of full document restarts while preventing the persistence of partial corruption.
• Modest Scope: The author is explicit that the study does not claim to optimize translation quality, nor does it evaluate the routing order's optimality. The work is a structural and operational contribution to handling the "failure" of successful API responses in long-form contexts. The evaluation is confined to the logic of the recovery protocol, not the semantic quality of the output.