Making AI Agent Plans Resilient: Implementation Patterns for Replanning Under Uncertain Tool Results

by

An agent plan is a bet.
It bets that a search will return useful results, an API will respond with a success code, a file will exist and be available, a payment will settle, and a human will approve the expected option. In demonstrations, these bets almost always win because the environment is carefully controlled. In production, they lose constantly.
An API returns 429 after three retries. A web scraper encounters a CAPTCHA instead of product data. A database query times out during a migration. A file exists but is locked by another process. A tool technically returns a response, but the response is noise rather than signal.
The difference between a reliable agent and a fragile one is rarely the model alone. It is the code surrounding the model: how it observes tool results, detects deviation, preserves useful work, and decides whether to retry, replan, ask for help, or stop.

Plans Fail Because the World Changes

Consider this ordinary plan:

  1. Fetch a user profile from the database.
  2. Query the user’s latest orders.
  3. Summarize the top three products.
  4. Generate an email draft.
  5. Send the email.
    The plan looks reasonable. But between steps 1 and 2, the database may become unavailable. The query may return no rows because the user has no recent orders, or because the filter is wrong. Between steps 3 and 4, the product data may be incomplete. Before step 5, the user’s preferences may have changed.
    The plan is not necessarily bad. Its assumptions have become invalid.
    This is a state-drift problem: planning happens at one point in time, while execution happens in a changing environment. A resilient agent treats the plan as a provisional hypothesis, not a fixed script.

Tool Results Are Not Binary

Traditional software often models an operation as either success or exception. Tool-using agents need a richer result model. At minimum, a tool call can produce five distinct outcomes:

  1. Verified success: The response is valid, complete, and sufficient for the next step.
  2. Recoverable failure: The operation failed temporarily, such as a timeout or rate limit.
  3. Valid empty result: The tool worked, but there is genuinely no matching data.
  4. Partial success: Some requested information was returned, while other parts are missing.
  5. Ambiguous or invalid success: The tool returned data with a success status, but the content is stale, malformed, blocked, or otherwise untrustworthy.
    These categories require different actions. A 429 may justify exponential backoff. An empty search may require a broader query. Partial data may support a reduced plan. An ambiguous response should trigger validation or an alternate source, not confident generation.
    Treating every non-exception response as success is particularly dangerous for language models. They can transform incomplete or incorrect inputs into fluent, plausible-looking wrong answers.

What Failure Looks Like in Production

Operational failures are often mundane:

  • An API responds with 429 Too Many Requests.
  • A scraper receives a CAPTCHA page with HTTP status 200.
  • A database query times out because a migration is running.
  • A file is present but locked by another process.
  • A search returns zero results because the query is too specific.
  • A payment provider accepts a request but delays final settlement.
  • A human approves a different option than the plan predicted.
  • A service returns cached data that is valid syntactically but too old to use.
    The common pattern is that the tool boundary hides uncertainty. A status code alone is not enough. The agent needs observations about freshness, completeness, schema validity, provenance, and confidence.

The Closed Loop: Plan, Execute, Verify, Replan

The most dependable structure is a closed loop:
Plan -> Execute with observation -> Verify -> Replan if necessary
For example, instead of directly passing database output to the summarizer, an agent can execute a step with an explicit contract:

Expected:
- response status is successful
- result contains an orders array
- data is no older than 24 hours
- at least one required field exists per order

The executor records what actually happened:

{
  "status": "partial",
  "rows": 2,
  "expected_rows": 3,
  "freshness_hours": 6,
  "missing_fields": ["product_category"],
  "retryable": false
}

The verifier then decides whether the original next step remains valid. If product categories are required for the email, the agent might replan to fetch them from a catalog service. If only one order exists, it might generate a summary of one product rather than inventing two more. If the result is empty, it might ask whether the user wants a broader date range.
The important point is that replanning happens after observation, not only after an exception.

Make Every Step Cheap to Repeat

Replanning is useful only when repeating work is affordable. Each step should therefore be designed as a small, observable, independently retryable unit.
Several implementation patterns help:

Use Explicit Preconditions and Postconditions

A step should define what must be true before it runs and what evidence proves it succeeded.
For a “send email” step, the postcondition should not be “the provider returned 200.” It might be:

  • a provider message ID was returned;
  • the recipient and subject match the approved draft;
  • the request has an idempotency key;
  • the provider status is accepted for delivery.
    This prevents downstream steps from treating weak evidence as completion.

Separate Retryable and Non-Retryable Failures

Retrying everything increases latency and can amplify an outage. Retry transient network errors, rate limits, and temporary service unavailability with exponential backoff and jitter. Do not blindly retry validation failures, authentication errors, malformed requests, or CAPTCHA responses.
A practical policy might allow three attempts over 20 seconds for a transient API failure. After that, the agent should switch strategy or replan rather than repeating the same call indefinitely.

Make Side Effects Idempotent

Replanning can accidentally repeat side effects. If an agent retries a payment or sends an email twice, the result may be worse than the original failure.
Use idempotency keys for operations such as payments, ticket creation, and message sending. Store durable step state so the executor can distinguish “not started,” “in progress,” “completed,” and “unknown.” When the network fails after a request is sent, the agent should query the operation status before issuing a second request.

Preserve Successful Work

A replan should not discard everything. Store outputs as artifacts with timestamps, source information, and validation status. If steps 1 through 3 completed successfully and step 4 failed, the new plan should reuse those artifacts unless they are stale or invalidated.
This is how replanning minimizes work lost. It is also why each step should have a narrow responsibility rather than combining retrieval, transformation, and side effects into one opaque tool call.

Use Fallbacks Deliberately

A fallback is not automatically better than failure. If a primary product API is unavailable, a cached response may be acceptable for a recommendation but unacceptable for a financial transaction.
Define fallback quality explicitly:

  • Strong fallback: equivalent source with comparable freshness.
  • Degraded fallback: older or less complete data, clearly labeled.
  • Unsafe fallback: data unsuitable for the current decision.
    The planner should know which fallback classes are permitted for each goal.

Prevent Infinite Loops and Cost Explosions

A resilient agent must also know when to stop. Without limits, a loop of “tool fails, model replans, tool fails again” can consume tokens, money, and time while producing no progress.
Useful guardrails include:

  • a maximum number of replans, such as three;
  • a total time budget, such as 60 seconds;
  • a tool-call budget, such as 12 calls;
  • a token or monetary budget;
  • a progress requirement, where each replan must change the query, source, or strategy;
  • a confidence threshold below which the agent must ask a human.
    The system should log why a replan occurred and what changed. If three plans make the same request against the same unavailable service, that is not resilience. It is repetition with better prose.

Reliability Is a Systems Property

After 18 months of running agent systems in production, the most effective pattern has been simple: make uncertainty visible and make recovery ordinary.
The model can propose alternatives, but application code should enforce contracts, validate outputs, manage retries, protect side effects, and enforce budgets. A tool response is not trustworthy merely because it is syntactically valid, and a plan is not correct merely because it is coherent.
Reliable agents do not assume that the world will follow the plan. They observe the world, verify what changed, preserve what remains useful, and adapt with bounded effort.
The goal is not perfect prediction. It is graceful recovery when the prediction is wrong.