Agent Loops and Tool Reliability

A model-directed application needs an explicit operational loop. Once a model can take more than one step, the core design questions are concrete: how does the system choose its next action, which tools may it call, what may it infer from the result, and when must it stop?

Moving from architecture diagrams to an operational agent requires a bounded cycle rather than a grand theory of autonomy: inspect the current state, choose the next action, observe the outcome, and either continue, escalate, or stop. When that cycle is left vague, failures are erratic and hard to reproduce; when it is explicit, its transitions become testable.

This post focuses on that runtime loop using a fictional OptiVerse Travel copilot with simulated flights, hotels, rates, and dates. We stay close to production mechanics: tool contracts, observation handling, retry policy, escalation rules, and failure modes, leaving broader system-wide governance and auditability to later lessons.

Post 5 established the multidimensional control-and-authority matrix and introduced approval boundaries. Here we turn that framing into runtime behavior: once a system grants bounded model-directed control, how does the loop execute, and where does it break?

What This Post Is Not

Post 5 covered when to choose an agent over a fixed workflow, Post 7 covers hybrid grounding and working memory, and the capstone in Post 10 addresses end-to-end production governance and auditability. This post owns the runtime mechanics in between: loop structure, tool contracts, observation handling, retry policy, and stop conditions.

The Minimal Operational Loop

Most agentic systems share a five-step cycle:

1. inspect the current task state

2. choose the next action

3. execute that action through a tool or controlled generation step

4. observe the result

5. revise the plan or stop

In a reason-act-observe loop, the model participates in a controlled cycle with an external environment that includes tools, retrieved documents, validation checks, and explicit system state.

Each step reshapes the next. Whereas a single-shot assistant can only answer from its initial context, a looped system can gather evidence, detect gaps, revise its search path, or decide that it should not continue.

Even so, many production tasks are better served by deterministic pipelines. Fixed workflows can branch on intermediate observations too. Model-directed control earns its complexity when predefined routes cannot adequately express the choice of next investigation and testing shows a measured benefit.

Worked Example: Building a Complete Itinerary

Consider a concrete task for the OptiVerse Travel copilot:

Build a 10-day accessible Japan itinerary for cherry-blossom season: family of four, wheelchair user, $13,000 to $15,000 budget (JPN-2026-0417).

Fulfilling this request may require checking flight inventory, inspecting hotel accessibility details, verifying rail-pass coverage, and judging whether the plan is complete enough to present to the client. That makes the task a reasonable candidate for an agent loop, though a branching workflow can also query changing availability and should be compared on representative tasks before committing to model-directed control.

A bounded version of the loop looks like this:

1. Search flight availability for the preferred departure date to Tokyo.

2. Review the returned options to see whether the preferred date is sold out and whether required assistance is confirmed, unavailable, or still unknown. Generic seat availability does not confirm that an individual traveler's assistance requirements will be met.

3. If the preferred date is sold out and the user allowed date flexibility, broaden the search to adjacent days and check price and accessibility fit.

4. Compare viable flight dates against hotel availability for wheelchair-accessible rooms in the target cities.

5. If hotel dates conflict with the adjusted flights, shift the hotel block and re-verify availability.

6. Check rail coverage from a current authoritative source carrying an effective date, applicable channel, and source ID, adjusting the route only from validated results.

7. Recheck total cost against the $13,000 to $15,000 budget. Present the revised itinerary with explicit trade-off notes once the evidence is complete; if the plan still exceeds budget or leaves unresolved conflicts after the allowed search attempts, stop and escalate.

This cycle is model-directed only if the model selects the next investigation from the allowed action set. If every branch is hardcoded, the system remains a fixed workflow even when it reacts to environment feedback.

ReAct as a Reference Pattern, Not a Religion

The ReAct pattern remains a useful historical reference because it interleaves model-generated reasoning traces, task-specific actions, and environment observations. Its empirical results were task-dependent: on HotpotQA, standalone ReAct scored below chain-of-thought prompting, the strongest prompting variant in that comparison combined ReAct with chain-of-thought self-consistency, and a separately reported supervised baseline scored higher still, whereas interactive decision-making benchmarks showed clear gains over their tested baselines (Yao et al., 2023).

What carries over to production is that interleaving of reasoning and action, not a verbose chain of thought or a research-demo prompt format. A production loop needs four structural requirements:

• a bounded set of allowed actions for the model

• a structured observation returned for every action

• explicit task-state updates driven by those observations

• explicit stop conditions enforced by the runner

Some systems keep planning as bounded internal state; others use an explicit planner-executor split where one component proposes a step and another executes it. In neither architecture does model-generated rationale text count as a faithful audit record, because generated explanations can omit or rationalize the factors that drove an output (Turpin et al., 2023). Audit the tool calls, observations, state transitions, policy decisions, and environment outcomes instead.

Tools Are Interface Contracts

Tool use is often described loosely as giving a model "access" to a search API, database, or code runner. In engineering terms, tools are typed interface contracts.

For an agent loop, a tool contract usually specifies:

• the tool name

• when the tool should and should not be used

• an input schema and valid argument constraints

• a structured result format

• a typed error format

• iteration or permission limits

Every one of those fields shapes how the runtime handles model proposals. Ambiguous descriptions muddy tool selection, permissive schemas let malformed arguments slip past validation, and ad hoc result strings make observation handling brittle. Test each boundary directly rather than treating schemas as passive documentation.

The provider-neutral teaching contract below specifies a simulated read-only flight-search tool. It is not a provider API payload; adapt input_schema to your model provider's tool format and validate result_schema separately in application code:

{
  "name": "search_flights",
  "description": "Search available flights by origin, destination, date range, and accessibility requirements.",
  "input_schema": {
    "type": "object",
    "properties": {
      "origin": {
        "type": "string"
      },
      "destination": {
        "type": "string"
      },
      "date_start": {
        "type": "string",
        "format": "date"
      },
      "date_end": {
        "type": "string",
        "format": "date"
      },
      "accessibility": {
        "type": "string",
        "enum": [
          "wheelchair",
          "none"
        ]
      },
      "max_results": {
        "type": "integer",
        "minimum": 1,
        "maximum": 10
      }
    },
    "required": [
      "origin",
      "destination",
      "date_start",
      "date_end",
      "accessibility",
      "max_results"
    ],
    "additionalProperties": false
  },
  "result_schema": {
    "oneOf": [
      {
        "type": "object",
        "properties": {
          "status": {
            "const": "ok"
          },
          "options": {
            "type": "array",
            "maxItems": 10,
            "items": {
              "type": "object",
              "properties": {
                "flight_id": {
                  "type": "string",
                  "minLength": 1
                },
                "assistance_status": {
                  "type": "string",
                  "enum": [
                    "unknown",
                    "reported_available",
                    "reported_unavailable"
                  ]
                }
              },
              "required": [
                "flight_id",
                "assistance_status"
              ],
              "additionalProperties": false
            }
          },
          "source_timestamp": {
            "type": "string",
            "format": "date-time"
          }
        },
        "required": [
          "status",
          "options",
          "source_timestamp"
        ],
        "additionalProperties": false
      },
      {
        "type": "object",
        "properties": {
          "status": {
            "const": "error"
          },
          "error_code": {
            "type": "string",
            "enum": [
              "invalid_arguments",
              "access_denied",
              "rate_limited",
              "service_unavailable"
            ]
          },
          "retryable": {
            "type": "boolean"
          }
        },
        "required": [
          "status",
          "error_code",
          "retryable"
        ],
        "additionalProperties": false
      }
    ]
  }
}

Even this compact schema encodes concrete operational boundaries: it requires an explicit assistance-request tag (which flags a request rather than completing a full accessibility assessment), separates start and end dates, rejects extra fields, caps result counts, and distinguishes success from error payloads via a status discriminator. Because JSON Schema makes format assertions optional (many validators skip them by default) and strict-schema enforcement varies across providers, validate arguments again on the server with date-format assertions enabled, and separately enforce date_start <= date_end, permitted date flexibility, valid airport codes, and source freshness. Inject authentication, tenant identity, and credentials from the runtime rather than accepting them in model-generated arguments.

Output structure matters just as much. When a tool mixes prose snippets, ad hoc JSON objects, and untyped error strings, recovery logic becomes guesswork. Keep tool outputs as strictly typed as tool inputs.

Observation Is Environment Feedback

Explanations of agent loops often get fuzzy at the observe step, treating it like model introspection when it is simply environment feedback.

In an operational system, an observation may be:

• a validated tool result

• a validation failure

• a missing required field

• a call timeout

• an access-control denial

• an unconfirmed or missing assistance field

• an empty search result for the requested flights or hotels

Each outcome calls for a distinct transition rule.

When flight search returns viable options on adjacent dates, the loop can move on to hotel verification. When a target date range returns zero flights, policy may permit broadening the date window once. Incomplete room-accessibility records may warrant checking an alternate property or escalating to a human specialist, whereas an access denial on a partner booking API must not be retried with slight variations of the forbidden request.

Observation handling therefore belongs in application logic rather than prompt wording alone. Without explicit rules for each result class, the model is left to improvise recovery behavior from ambiguous failures.

Treat every tool result as untrusted data. Web pages, partner documents, and compromised tools can embed instructions designed to hijack the loop, as demonstrated in indirect prompt-injection attacks against LLM-integrated applications (Greshake et al., 2023). Validate and size-limit observations, record their provenance, and re-authorize every proposed action after the model reads them.

Planner-Executor Splits and When They Help

A single loop handles compact tasks well, while larger workflows sometimes benefit from separating planning and execution. In a planner-executor architecture, the planner selects the next subtask and the executor performs that bounded action under tighter tool and schema constraints.

For the travel copilot, a planner might lay out four subtasks:

1. search Tokyo flights for the target week, retaining the traveler's assistance requirements and recording whether the airline has confirmed them

2. verify hotel availability and wheelchair accessibility in Kyoto and Tokyo across the matching date range

3. check current rail coverage for the planned segments and record the source version used

4. assemble a total-budget claim only once flights, hotels, rail, meals, taxes, fees, and all other required costs are accounted for; otherwise report a qualified subtotal

The executor runs each subtask through typed tools and records the resulting observations. The split can improve clarity: because the executor completes one bounded step at a time instead of rethinking the overall strategy on every turn, its prompts and tool scopes can stay narrower.

That separation trades simplicity for extra coordination: another interface contract, another state handoff, and another place for errors to accumulate. Start with a single bounded loop and introduce a planner-executor split only when task complexity justifies the added surface area.

Retry Is a Policy, Not a Reflex

Once external tools enter the loop, partial failures are routine: a booking API times out, a property lookup errors out, or the model emits an invalid field. The engineering question is which failure classes warrant a retry and under what bounds.

For the travel copilot, a disciplined retry policy might look like this:

• retry once when a call fails input validation and an obvious, schema-valid fix preserves user intent

• retry once with a wider date range when the preferred departure date returns zero flights and the user authorized date flexibility

• do not retry indefinitely when a valid query reports no availability across the permitted window

• do not retry access_denied under unchanged authority; legitimate permission changes require a fresh authorization check, and human approval alone cannot grant missing access rights

• stop retrying whenever consecutive attempts add no new information

Unbounded retries can create the illusion of diligence while compounding error and cost. Record what new evidence each additional search expects to obtain, and keep date adjustments within user-authorized flexibility rather than relaxing hard constraints just to force a match.

Reflexion retains verbal self-reflections on task feedback across trials without weight updates (Shinn et al., 2023), and some loops similarly revise after a failed step. Revision can help when the failure carries actionable feedback, provided strict attempt caps prevent "just one more revision" from drifting indefinitely.

Classify every failure before deciding whether to retry:

• transient: a retry-safe read timeout, rate limit, or temporary upstream error (whereas a write timeout must first be treated as a potentially unknown outcome);

• correctable input: a schema or parameter error fixable without altering user intent;

• permanent for the current request: an invalid resource or unsupported operation (a valid search that returns no availability is a domain result rather than a transport error or proof that inventory will never change);

• policy: an access denial or missing required approval;

• unknown outcome: a lost response on a request that may already have mutated remote state.

For retry-safe transient failures, combine per-call timeouts with bounded exponential backoff and jitter. Honor Retry-After, whose value RFC 9110, §10.2.3 defines as either delay-seconds or an HTTP-date; if the required wait cannot fit within the remaining task deadline, stop or defer instead of retrying early. Cap both total attempts and elapsed wall-clock time, and retry at a single architectural layer so SDK, tool-wrapper, and agent-loop retries do not multiply one another.

Automatic retries on externally visible writes require an idempotency or equally strong duplicate-suppression contract supported by the destination service; without one, retries can duplicate side effects (Featonby, AWS). Generating a local UUID does nothing if the receiving endpoint ignores it. Reuse a single stable operation key only for identical intent and parameters, and reconcile remote state after any write timeout. Whenever reconciliation cannot confirm whether the write took effect, terminate with unknown and escalate; never let the model guess that the booking succeeded.

Escalation and Stop Conditions Are Part of the Design

A reliable loop defines its exits as carefully as its happy path.

Give the runner four explicit terminal states:

• complete: verified evidence satisfies every task requirement

• abstain: available evidence and permissions cannot answer the request reliably

• escalate: completing the step requires human review, approval, or judgment

• unknown: a potentially side-effecting call ended inconclusively; preserve the operation key and hand off without claiming success or failure

Specify these exits upfront in the state machine rather than improvising them in prompt instructions.

For the travel copilot, concrete stop conditions include:

• stop and synthesize once flights, accessible rooms, and rail coverage align within the authorized dates and budget

• stop and abstain once the search budget is exhausted without a viable flight-and-hotel combination

• escalate when hotel accessibility records are ambiguous or contradictory and tool queries cannot resolve them

• escalate whenever progressing would issue a reservation, charge a payment card, or override a travel advisory

• stop when the loop hits its tool-call cap or wall-clock latency budget

These conditions bind evidence sufficiency, authority limits, and operational cost together: a loop can also fail by overrunning latency, burning budget, or overstepping its permissions, even when the itinerary it returns is factually correct.

Common Failure Modes in Agent Loops

In a multi-step loop, failures compound sequentially: one misread observation can skew the next tool call, and one permissive retry rule can trigger a costly spiral. Watch for seven recurring failure modes:

Invalid Tool Arguments

The model supplies nonexistent field names, wrong types, or contradictory parameters. Strict JSON Schemas catch structural errors but not every invalid combination, so pair server-side validation with a deterministic error payload that tells the runner whether a single repair attempt is permitted.

Incorrect Tool Choice

The loop calls flight search when the fixture requires hotel availability, or checks rail passes before confirming inbound flights. Root causes range from overlapping tool descriptions and prompts that push premature action to incorrect task state, schema problems, or model selection errors; inspect the recorded request and tool trace before changing prompts or schemas.

Tool-Result Parsing Errors

Even a valid tool call fails downstream when the runner misinterprets its output, such as treating an empty options array as a transport crash or treating unconfirmed assistance, such as an "unknown" status, as confirmed availability.

Premature Synthesis

After gathering partial evidence, the loop generates a polished itinerary before verifying critical constraints. Smooth prose hides thin or conflicting underlying records unless a completion gate checks required evidence IDs first.

Repeated Loops Without New Information

The runner repeats equivalent searches or nudges dates back and forth without changing task state, burning tokens and latency while giving the appearance of thoroughness.

Missing Escalation

When the loop hits ambiguous or contradictory data or an authorization boundary without a modeled escalate transition, it keeps probing tools on a problem that requires human judgment.

Compounding Error

An early mistake in flight dates or room accessibility carries forward into rail checks and budget totals, turning one localized misread into a plausible-looking but unusable itinerary.

None of these failure modes rules out agent loops; together they call for explicit contracts, state tracking, and hard budgets.

Practical Guidance for Operational Design

Six design rules make operational agent loops more predictable as you move from prototype to production.

First, keep the action space minimal. Expose only the tools required for the current task; extra tools widen selection ambiguity, the error surface, and the attack surface.

Second, enforce strict input and output schemas so every call and observation is validated on both sides of the tool boundary.

Third, persist observations in structured task state instead of forcing the model to reconstruct facts from a growing chat transcript; explicit state keeps the loop inspectable and reduces accidental drift.

Fourth, separate recoverable errors from terminal ones: a fixable malformed argument may earn one retry, whereas an access_denied response earns none while authority is unchanged.

Fifth, wire stop conditions (evidence requirements, attempt caps, wall-clock deadlines, and approval gates) into the runner from day one.

Sixth, grade the trajectory alongside the final response by inspecting which tools ran, what observations returned, why retries fired, and whether the runner stopped on time.

Hands-On Exercise

Implement the loop runner with seven explicit states: plan, call, observe, complete, abstain, escalate, and unknown. Persist every transition with the tool name, validated arguments, result class, attempt count, policy decision, call latency, and operation key.

Enforce a hard cap of 8 admitted tool attempts (counting rejected arguments, retries, and reconciliation lookups) and a 30-second total wall-clock budget covering planning, waits, and tool execution. Cancel or time-box in-flight calls where supported, keeping in mind that halting the local runner does not roll back a remote write. Exercise these five fixtures:

• successful read-only flight and hotel search;

• rate limit returning Retry-After: 2;

• invalid arguments correctable on a single retry;

• access denied, which must not retry;

• write timeout after a simulated hotel hold may already have succeeded.

Verify deterministic behavior with scripted observations first, then run each fixture three times from a clean state with live model-directed planning if available (treating three runs as a smoke check rather than a statistical reliability bound). Grade both trajectory policy compliance and final environment state. While tau-bench combines a final-database check with required-response checks and reports consistency across repeated trials, its authors note that outcome rewards alone can miss procedural violations such as acting without user confirmation, so add an explicit trajectory authorization check (Yao et al., 2024).

Drive deadline and Retry-After checks with a fake clock so tests run deterministically without live bookings. Increment the attempt counter upon admission before schema validation, verify remaining budget before sleeping or dispatching, and treat retryable as an untrusted adapter hint subordinate to runner policy:

Also test three malformed tool outputs (numeric options, mixed ok and error fields, and an invalid source_timestamp), verifying that all three fail result validation with format assertions enabled, plus an inconclusive reconciliation lookup that terminates in unknown while preserving the hold operation key even when the attempt budget is exhausted. Implement atomic key-and-hold persistence in your simulated hold tool (distinct from the read-only search_flights schema above).

Deliberately Broken Case

Suppose the booking sandbox commits hold H-447 but drops the HTTP response. A naive runner sees timeout, mints a fresh request key, and retries, creating H-448. Its final answer reports a single room hold while two active holds sit in the database.

Fix the runner by binding human approval and canonicalized parameters to a single operation key, querying hold state before any retry after a lost write response, and retrying with that same key only under the destination's documented duplicate-suppression contract. Scope keys by tenant and operation, reject payload mismatches against existing keys, and persist operation records across process restarts for the full retry horizon. The test passes only if exactly one hold exists. If reconciliation remains inconclusive, the runner must terminate as unknown and escalate without issuing a second intent.

Why This Matters More Than Agent Branding

Teams often spend more time debating whether a feature "counts" as an agent than specifying its loop contract.

From an engineering standpoint, seven operational questions matter:

• which decisions the model may make autonomously

• which tools are in scope

• what typed observations come back

• what task state persists across steps

• which failures trigger bounded retries

• which conditions require human escalation

• which conditions force a stop

Clear answers to those questions determine system behavior far more than architectural labels. Narrow action spaces, validated contracts, and explicit stop rules make failures easier to reproduce and test. Establish comparative performance with task measurements instead of assuming that either a simpler or a more ambitious loop wins.

Exit Criteria

Your loop runner is complete only if:

• every action and observation matches a typed contract;

• tenant identity and credentials come from the runtime, not model arguments;

• access denial and permanent failures do not retry under unchanged authority and inputs;

• transient retries honor delay policy and remain within both budgets;

• externally visible writes use idempotency and outcome reconciliation;

• injected tool content cannot expand permissions or select a prohibited action;

• all fixtures terminate in a declared state across repeated trials;

• the timeout-after-side-effect fixture produces exactly one hold;

• the trace is sufficient to grade policy and final environment state without using rationale text as proof.

Knowledge Check

Answer without looking back.

1. Why is generated reasoning text not an audit record?

2. Which failure class requires reconciliation before retrying?

3. Why should retries occur at only one layer?

4. What is the correct terminal state when an external outcome cannot be determined?

5. What does the broken hold test verify?

Answer Key

1. Model rationales can omit or rationalize influences; auditable facts are calls, observations, policy decisions, state transitions, and outcomes.

2. An unknown outcome after a potentially side-effecting call.

3. Nested retry layers multiply attempts and can create retry storms or duplicate work.

4. unknown is terminal for this runner; a separately tracked reconciliation or human-review process may resolve the operation later.

5. The simulated destination's duplicate-suppression contract, stable key, and reconciliation produce one hold despite a lost response. This does not establish the behavior of an arbitrary real booking endpoint.

Bridge: When Basic RAG Stops Being Enough

A compact reason-act-observe loop is often enough for straightforward evidence lookups, but some multi-step investigations outgrow isolated single-turn retrieval. They may need to reason across a growing bundle of documents, preserve verified intermediate findings, or balance live lookups against a stable working set assembled for the active case.

Once an investigative workflow (whether inside an agent loop or a staged pipeline) needs a coherent case packet instead of a fresh top-k query on every turn, retrieval architecture and working memory become central design problems.

Continue with hybrid retrieval and evidence packets after completing its RAG and memory prerequisites. That lesson evaluates sparse and dense retrieval, reranking, and bounded working context as independent engineering choices. Repeated reasoning over evidence is a reason to evaluate reuse and freshness, not a requirement to adopt every one of those mechanisms.

Continue the Shared Local Lab

Download the travel-copilot lab and unpack it, or use examples/travel-copilot in the repository. Run commands inside that directory. Python 3.9+ is sufficient; start with README.md and keep one out directory across lessons. Inspect tool_loop and run the local action rehearsal. It reads the prior passing evaluation, deliberately loses a committed response, and performs one authoritative lookup before stopping.

python3 lab.py capstone --out out

Run pipeline, retrieve, and evaluate in that order before the action stage. The lab uses fictional records, a small bilingual lexical dictionary, extractive answers, and a local SQLite supplier. Approval is simulated; no model API, real reservation, payment, OCR, or image analysis is invoked. It implements one inspectable slice of the broader architecture, with explicit extension boundaries in the README.

Sources and Further Reading

This post draws on the following primary and official sources:

• Yao, S., Zhao, J., Yu, D., et al. "ReAct: Synergizing Reasoning and Acting in Language Models." Core reference for interleaving reasoning, actions, and observations. arxiv.org/abs/2210.03629

• Schick, T., Dwivedi-Yu, J., Dessì, R., et al. "Toolformer: Language Models Can Teach Themselves to Use Tools." Reference for learned tool-use decisions. arxiv.org/abs/2302.04761

• Shinn, N., Cassano, F., Berman, E., et al. "Reflexion: Language Agents with Verbal Reinforcement Learning." Reference for bounded verbal feedback and revision loops without weight updates. arxiv.org/abs/2303.11366

• Anthropic. "Building effective agents." Practitioner reference for simple composable agent patterns and the case against unnecessary framework complexity. anthropic.com/research/building-effective-agents

• Anthropic. "Tool use with Claude." Official reference for tool schemas and result handling. platform.claude.com/docs/en/agents-and-tools/tool-use/overview

• National Institute of Standards and Technology. "Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile." Reference for bounded control, testing, and governance language. nist.gov/publications/artificial-intelligence-risk-management-framework-generative-artificial-intelligence

• Turpin, M., Michael, J., Perez, E., and Bowman, S. R. "Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting." 2023. Evidence that generated rationales can be unfaithful. arxiv.org/abs/2305.04388

• Greshake, K., Abdelnabi, S., Mishra, S., et al. "Not What You've Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection." 2023. Evidence that external content can redirect tool-using systems. arxiv.org/abs/2302.12173

• Featonby, M. "Making retries safe with idempotent APIs." Amazon Builders' Library. Guidance for stable operation identifiers, safe retries, and side-effect reconciliation. aws.amazon.com/builders-library/making-retries-safe-with-idempotent-APIs

• Yao, S., Shinn, N., Razavi, P., and Narasimhan, K. "tau-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains." 2024. Repeated-trial final-state and required-response evaluation; a passing reward can still miss policy violations. arxiv.org/abs/2406.12045

• RFC 9110, §10.2.3. Retry-After supports delay-seconds and HTTP-date.