A free or anonymous preview can return a rate-limit error, timeout, incomplete stream, empty answer, malformed body, or unavailable route. Blind retries can multiply token use, repeat tool actions, overwrite newer state, or switch to a fallback whose privacy and behavior were never approved.
Use this runbook for an Ox Alpha evaluation, coding assistant, content pipeline, or agent workflow that must stop cleanly and resume without duplicating WordPress, deployment, CRM, or analytics actions.
Quick answer
Classify the failure before retrying. Save request ID, model ID, provider route, status, headers, response shape, last valid stream chunk, tool-call ledger, and checkpoint. Honor Retry-After when present, use bounded exponential backoff with jitter for retryable failures, and cap attempts. Never replay a write unless its idempotency key proves the first action did not apply. Route to a pretested fallback only with the same privacy class, tool contract, acceptance tests, and output validation.
What to check first
- Capture status, error code, request and generation IDs, provider attempts, Retry-After, latency, token usage, and incomplete details.
- Classify authentication, permission, schema, safety, rate-limit, provider-capacity, timeout, partial-stream, route-removed, and client-parse failures separately.
- Read the tool ledger to determine whether a side effect was requested, accepted, completed, rejected, or left unknown.
- Resume from the last durable checkpoint with the same fixture and idempotency keys rather than resending the entire conversation blindly.
- Verify the fallback model, route, terms, context limit, tool support, schema behavior, and data approval before automatic use.
Why this usually happens
- Preview routes can have limited capacity and changing per-request limits.
- A network break can hide whether the provider or tool completed work.
- Client parsers may call an incomplete stream empty even when earlier chunks contained state.
- Fallback routing can change model behavior, context, price, privacy, and tool support.
Useful command or data shape
Adapt paths, IDs, and privacy handling to the site before running commands or storing data on production.
retry_policy:
retryable: [429, 502, 503, 504]
max_attempts: 3
backoff_seconds: [2, 8, 20]
honor_retry_after: true
resume_from: durable_checkpoint
write_replay: idempotency_required
fallback: pretested_only
Diagnostic table
Use this table to connect the observed behavior to evidence and a verification step.
| Action | Evidence to collect | How to verify |
|---|
| Capture the failure envelope | Capture status, error code, request and generation IDs, provider attempts, Retry-After, latency, token usage, and incomplete details. | The incident record explains why each retry was allowed or denied. |
| Classify retryability | Classify authentication, permission, schema, safety, rate-limit, provider-capacity, timeout, partial-stream, route-removed, and client-parse failures separately. | No side effect occurs more than once across original, retry, and fallback attempts. |
| Reconcile tool state | Read the tool ledger to determine whether a side effect was requested, accepted, completed, rejected, or left unknown. | The final output identifies the model and route that actually completed the task. |
| Resume with bounded backoff | Resume from the last durable checkpoint with the same fixture and idempotency keys rather than resending the entire conversation blindly. | A controlled failure leaves a resumable checkpoint and clear human owner. |
Decision rule
Stop automatically when state is ambiguous, a write lacks an idempotency record, the retry budget is exhausted, the privacy class cannot follow the fallback route, or the replacement model has not passed the same tool and output fixtures.
Test scenarios to run
Run the same controlled fixture across these branches. Write down the expected result before testing so a surprising response is easy to identify.
| Scenario | Fixture | Expected result |
|---|
| 429 | Retry-After present | Wait, jitter, bounded attempt |
| Empty 200 | No usable output | Record and retry once from checkpoint |
| Partial tool stream | Arguments incomplete | Do not execute |
| Route removed | Model unavailable | Approved fallback or controlled stop |
Safe fix order
Use a sequence that makes each result easy to prove. Stop when new evidence changes the scope or owner of the problem.
- Capture the failure envelope
- Classify retryability
- Reconcile tool state
- Resume with bounded backoff
- Use only an approved fallback
Production verification checklist
- The incident record explains why each retry was allowed or denied.
- No side effect occurs more than once across original, retry, and fallback attempts.
- The final output identifies the model and route that actually completed the task.
- A controlled failure leaves a resumable checkpoint and clear human owner.
What to tell the client or owner
Give the owner a short evidence packet with the checked time, exact source, test fixture, expected and actual result, privacy class, cost, affected records, rollback path, decision, and next review date. Do not include secrets, customer data, account tokens, or private source code.
Mistakes to avoid
- Treating a public counter, model claim, domain suffix, or paid rank as proof of business value without checking the underlying event and source.
- Copying a changing price, model limit, leaderboard rule, or provider name into evergreen copy without a timestamp and a verification link.
- Running a test with production credentials, customer records, private repositories, irreversible tools, or an unlimited retry loop.
- Publishing several broad pages for the same query instead of assigning one owner and giving every follow-up a distinct decision or implementation task.
Questions teams ask during testing
How often should this be reviewed?
Review volatile model metadata, prices, limits, leaderboard rules, bids, redirects, and domain terms before each decision. Keep the checked time next to the observation so later readers can tell durable guidance from a dated snapshot.
What evidence should the test keep?
Keep the exact URL or model ID, UTC timestamp, fixture, input settings, expected result, actual result, response or event ID, cost, latency, downstream record, reviewer, and final decision. Remove credentials and personal data before sharing the record.
Can this be used on a client production site?
Start with public or synthetic fixtures in an isolated environment. Move toward production only after privacy, security, reliability, rollback, ownership, and measurement gates pass and a responsible person approves the remaining risk.
How does this connect to HandL WP work?
The practical value appears where a trend touches a real website: DNS, redirects, WordPress permissions, forms, checkout, webhooks, analytics, CRM records, security review, search visibility, and recovery when the experiment fails.
When HandL WP should help
Bring in help when this affects leads, checkout, search visibility, security, paid media reporting, or a client production site. HandL WP can trace the issue through WordPress, hosting, cache, tracking, and Search Console, then verify the workflow after the technical fix.
If this is active on a production site, stabilize an automated WordPress workflow.
Related HandL WP guides
Use these related guides when the same issue touches tracking, security, checkout, or crawler visibility.
Helpful references