010: Agent/Loop
- Requires:
An iterative sequence of Requests aimed at achieving a goal. The agent continues to invoke Requests, process the resulting Calls, and feed the output back into the context until the LLM generates a Final Output, signaling the task is complete.
The Execution Loop is the engine that orchestrates all the message types and protocols from previous chapters. It enables an agent to perform multi-step tasks by iteratively making Requests. This iterative process of context assembly, tool use, and feedback is what is commonly meant when referring to an "agent."
The Execution Loop
The execution loop is the primary mechanism for autonomous, multi-step execution. It operates via a nested loop structure:
-
Outer Loop (Request Generation): The agent's lifecycle is a sequence of Requests. It starts with an initial context and enters a loop.
-
Request & Call Streaming: Inside the loop, it invokes a single Request. The Request streams back Calls as they are generated, which are collected into a pending queue.
-
Inner Loop (Call Orchestration): For each Request, an inner orchestration loop is responsible for executing its associated Calls. This process is a reactive, event-driven loop that is triggered by two events: a new Call is streamed from the LLM, or a previously running Call completes. This process is highly concurrent:
-
The orchestrator continuously scans the queue of pending Calls to find all that are currently unblocked (i.e., their dependencies are met).
-
All unblocked Calls can be presented for confirmation and then executed in parallel. This allows for high throughput, but requires careful management of state. If multiple concurrent Calls write to the same path in the State, the final value will be determined by the last call to complete, which may lead to non-deterministic outcomes. The system relies on a "last-write-wins" approach for resolving these conflicts.
-
As each Call completes, its output updates the shared context, potentially unblocking other pending Calls.
-
This reactive, parallel execution continues until the stream for the current Request is closed and all of its pending Calls have been drained. This model significantly reduces latency, as the agent can start working on multiple independent steps simultaneously, even before the full plan is known.
-
-
Termination & Continuation: Once the inner loop completes for a given Request, the agent inspects the final Solution. The decision to continue is based on the
outputfield:-
If
outputisnull, the agent determines that its task is not yet complete. It loops back to step 2, invoking a new Request with the enriched context that now contains the results of the executed Calls. -
If
outputis notnull, the agent's goal is considered achieved. The outer loop terminates, and theoutputvalue, which conforms to the user-defined output schema, is returned as the final result. An agent can produce bothcallsand a finaloutputin a single step; the presence of theoutputis the definitive signal to stop.
-
Human-in-the-Loop
The new Execution Loop provides support for human oversight by placing the confirmation step just before execution. This ensures the user is only prompted to act on calls that are ready to run:
- Approval: Before an unblocked Call is executed, the system can present it to a user for approval. This is an efficient approach, as it prevents the user from having to review and confirm calls that might be blocked by dependencies and never run.
- Correction: The user can modify the parameters of a Call or even replace it with a different one
It is important to note that these specific HITL mechanisms are not part of the core protocol. The architecture simply provides the necessary separation between proposing actions and executing them, giving developers the flexibility to implement any kind of intervention, from a simple manual approval to a complex, automated system with timeouts.
This capability is critical for safety and for collaborative tasks where the agent acts as an assistant. User adjustments and feedback can be leveraged by the Plan, allowing the agent to refine its strategy based on human input.
Error Handling and Self-Correction
The Execution Loop is designed to be resilient, providing a reactive safety net that allows the agent to handle and recover from both its own planning mistakes and unexpected runtime failures. This is achieved by catching errors and feeding them back to the LLM as structured, contextual information.
The Error Message
To facilitate this, the system uses a specialized Error Message, which is based on the Data Message. When an error is caught, the loop generates an Error Message and appends it to the context for the next iteration. Its data property contains two critical pieces of information:
call: The full, serializedcallobject that failed.error: An object or string detailing the reason for failure.
The Self-Correction Loop
The loop catches two primary types of unhandled failures:
- Structural Errors: Caught before execution. This includes schema validation failures on a call's parameters, mismatched types, or invalid Variable References.
- Runtime Errors: Caught during execution. This occurs when an explicit Activity throws an unhandled exception (e.g., due to an API failure or internal logic error).
When either error type is caught, the loop generates the Error Message and adds it to a temporary list. At the end of the execution tick (when no more calls are unblocked), all Error Messages collected during that turn are appended to the context.
This provides the LLM with direct, actionable feedback. On its next turn, it sees the failed call and the specific error it produced, allowing it to debug and generate a corrected plan.
The loop's ability to catch errors and report them back to the LLM is a reactive safety net for unplanned failures. However, the most robust workflows are designed with failure in mind from the start.
Using the Plan system with branching Expressions, you can create explicit "happy" and "unhappy" paths for your tools. An Activity can handle its own errors internally and, instead of throwing an exception, return a specific Data Message that directs the workflow down a pre-planned error-handling branch. This proactive approach is detailed further in the Plan chapter.
From Simple Loops to Strategic Plans
The Execution Loop provides a dynamic structure for agent behavior, but its power is in executing tactical, single-shot Requests. To manage complex, multi-step workflows with dependencies, a more advanced system of strategic planning is required.
011: Agent/Expressions introduces the protocol for defining these long-term strategies as a graph of interconnected Tool Calls.