OpenAI Agent Improvement Loop Practical Guide: Why agents need to be continuously modified through trace·eval·Codex handoff after deployment

1. One-line problem definition

Key takeaways: Since agents are meant to be operated repeatedly, not demos, they need a structure to turn failures into learnable units of work again.

The

OpenAI Cookbook's Agent Improvement Loop with Traces, Evals, and Codex example covers “how to make it better” after creating an agent. The problem here is not simply one of poor model performance. The real problem is that when users complain, that feedback doesn't lead to the next prompt, tool, verification condition, or test.

This article is aimed at developers and team leaders who can already create simple agents. Rather than being a tutorial for installing the OpenAI Agents SDK for the first time, it explains practical standards for attaching trace, feedback, evaluation, and Codex modification instructions to running agents. This may be overkill for tasks where the cost of failure is low, such as a one-off chatbot, in-house FAQ, or simple summarizer.

2. First, conclusion

Key takeaways: Agent improvement loops become valuable when they bundle “execution history - feedback - evaluation - code fixes” into one contract.

There are three recommended targets. First, the agent is a team that invokes the tool and creates an answer in several steps. Second, the team must manage the quality of answers through regression testing, not just by feel. Third, this is a team that wants to entrust editing work to a coding agent like Codex, but have a review flow with human approval.

Conversely, if the daily call volume is still low, there is little user feedback, and the failure types are not organized, it is better to start with a manual log review first. The improvement loop is not a device that automatically solves quality problems, but rather an operating method that turns problems into reproducible evidence and corrective action.

3. Decomposition of core structure

Key summary: The key word in this structure is harness. A harness is an execution contract that combines directives, tools, policies, output formats, and validation conditions around a model.

The

OpenAI Cookbook example creates five trace runs for a financial due diligence agent and attaches human and model feedback to those runs. Afterwards, the feedback is converted into a Promptfoo evaluation case, HALO organizes which harness changes take priority, and then passes it to a handoff document that Codex can implement.

Simplifying the flow is as follows:

1. Trace: Records what input the agent received, what tools it called, and what response it gave.
2. Feedback: A person or model writes “what was wrong” in the trace.
3. Eval: Turn that feedback into tests that can be repeated later.
4. Diagnosis: Diagnose whether the failure is a prompt issue, a tool schema issue, or an output validation issue.
5. Codex handoff: Hands off files to be modified, expected actions, and verification commands to the coding task.

From the perspective of a novice developer, trace can be understood as “black box recording,” eval as “retest problem,” and harness as “a bundle of agent execution rules.” The important thing is not just to keep records, but to ensure that records lead to testing and code changes.

4. Description of design intent

Key takeaway: This loop is not about reducing prompt tuning, it's about limiting prompt tuning to evidence-based changes.

Many teams use longer system prompts when the agent is incorrect. It may seem fast at first, but after a few weeks it becomes a prompt where you don't know why that sentence was there. The reason the OpenAI example puts trace and eval first is to leave a basis for modification.

Agents According to the SDK documentation, the SDK provides runtime elements such as tool calls, handoff, guardrail, session, and tracing. This means that agent quality is not determined by a single model call. Actual quality is determined by which tools are called and when, which outputs are blocked, and what reasons are left in case of failure.

Some things are given up by design. Automatically reflecting all feedback is fast but risky. He explains that the example can start with a change set that the developer reviews. In the early days of operation, “Codex making revisions and humans approving diffs” is safer than automatic merging.

5. Evidence and Comparison

Key takeaway: The trace-only approach, eval-only approach, and improvement loop approach solve different problems.

Approach	What you're good at	Weak point	Recommendation status
Manual log review	Quickly identify early failure types	Relies on reviewer memory and has weak regression prevention	When the call volume is low and the failure type is not yet known
Trace-centered operation	Check the location of tool call, handoff, guardrail failure	Revision priority does not appear automatically even when looking at records	When debugging complex agent flow
Eval-centered operation	Good for making regression tests and distribution gates	If actual user failure is not reflected in eval, it becomes a formal test	When important failure cases have already been sorted out
Trace + feedback + eval + Codex loop	Connect failures to evidence, testing, and corrections	Requires initial configuration cost and review time	When the agent needs to be continuously improved during operation

The

OpenAI tracing document explains that the Agents SDK basically traces the entire Runner execution, agent span, LLM generation, function tool call, guardrail, and handoff. Additionally, trace export may be delayed in long-running tasks, so you can call flush_traces() at the end of the unit of work. This detail is important in operations. This is because if the trace is uploaded late, it is easy to mistakenly believe that “there was no failure.”

6. Actual operation flow and step-by-step execution method

Key summary: Rather than fully automating from the beginning, it is better to first fix a small loop with 5-20 traces and 10 evals for one task.

In practice, you can start with the following order:

1. Select a task. Select a task that can inspect results, for example, “Contract Summary Agent,” “Customer Inquiry Triage Agent,” or “Report Inspection Agent.”
2. Place harness in file. Place directives, tool list, output schema, inhibit conditions, verification commands in repository.
3. Turn on trace. Agents SDK has tracing enabled by default, but if there is a sensitive data policy, the storage range is checked first. Organizations with a ZDR policy should also note the documented restriction that tracing cannot be used.
4. Collect more than 5 failure traces. Don't just collect wrong answers, mix cases where the tool was chosen incorrectly, sources were omitted, and answers were correct but formatted incorrectly.
5. Do not leave feedback in one sentence. Instead of “bad”, write an editable rule such as “controlled source should take precedence when sources A and B conflict”
Replace with
6. eval. Doesn't matter whether it's Promptfoo or an internal test, but must have input, expected condition, and failure message.
7. Creates a Codex handoff. Contains the file to be modified, change target, change to suppress, and verification command.

Minimal execution example is as follows:

python -m venv .venv
source .venv/bin/activate
pip install openai openai-agents halo-engine
npx promptfoo@0.121.9 eval -c promptfoo.yaml

In actual operation, the output path is more important than the model name. The key artifacts of the example are the trace file, eval configuration, and codex_handoff.md. Having this file ensures that improvements become “input for the next job” rather than “the next person’s memory”.

7. Mistakes and Pitfalls

Key takeaway: Most improvement loop failures result from a lack of standards, not a lack of automation.

Trap 1: Traces have been collected, but there is no evaluation standard

Just looking at the trace dashboard does not improve quality. To prevent this, add one sentence to each trace: “What condition was violated?” To recover, reread the 10 most recent failures and extract common failure conditions with eval:

Trap 2: Trusting the LLM judge as the answer

Model feedback is fast, but it does not replace the organization's quality standards. To prevent this, mix up a sample of human reviews and specify prohibition conditions and priorities in the judge prompt. To recover, add the failed answer passed by the judge as a separate regression case.

Trap 3: Entrusting Codex with too broad corrections

“Improve agent quality” is not a good handoff. To prevent, narrow down your modifications to things like harness files, tool schemas, and eval files. To recover, do not merge Codex results right away, but review eval failures and diff together.

Trap 4: Sensitive data is left in the trace

Tracking is powerful, but dangerous if turned on without security review. To prevent this, mask input, limit trace metadata, and set retention periods. To recover, delete sensitive traces and place a tracing processor or pre-filter to prevent recurrence.

8. Strengths and Limitations

Key takeaways: The strength of this approach is iterative improvement, while its limitations are initial design and data governance costs.

Strengths are clear. Failures turn into testing and fixes rather than scattered chat logs. Even if team members change, you can track why prompts have changed. You can also check if existing failures have recurred before deployment.

The limit is also large. First, writing a good eval takes time. Second, traces may contain sensitive information, so security standards are required. Third, relying too quickly on automatic improvement loops can lead to incorrect judges spreading throughout the product. Fourth, it is not cost-effective for small chatbots.

This is my judgment. Agents with failure costs such as customer service, document review, code changes, and finance/legal/security require this structure. Conversely, for in-house idea generation, short summaries, and personal productivity tools, traces and manual reviews are enough to get you started.

9. Points to study more deeply

Key summary: The next learning sequence is Agents SDK basic structure, tracing, eval, Codex handoff.

• The basic primitive of the Agents SDK: You must first see what roles agent, tool, handoff, and guardrail play.
• Tracing Structure: Understanding the difference between trace and span will help you locate the failure much faster.
• Eval design: Conditional evaluations such as “include source”, “no prohibited actions”, “comply with priority criteria” are more important than comparing strings of answers.
• Codex handoff How to write: Implementation request must include modification scope, file boundary, verification command, and anti-regression condition.
• Data Security: You must first check whether customer information, contract information, and internal documents are included in the trace.

10. Implementation checklist and author's perspective

Key takeaways: The criteria for adoption is not “nice automation” but “can failure be replicated and prevented?”

• Are the core tasks of the target agent defined in one sentence?
• Is the harness managed as a storage file and is there a change history?
• Are more than 5 failure traces actually listed as improvement candidates?
• Do you store human feedback and model feedback separately?
• Is there at least 10 evals and can they be rerun before deployment?
• Does the codex handoff include modification range, prohibition range, and verification command?
• Is there a masking standard to prevent sensitive data from remaining in trace and eval fixtures?

Definition of Done: The requirements from the recent failure trace are fixed to at least one eval, the harness change made by Codex passes that eval, and the diff has been approved by a human.

I view this approach as “the smallest operating unit of agent quality management.” Changing the model is easy, but preventing failure again requires structure. This is why OpenAI’s examples are important. This is because agent development is treated as an operational system improvement, not as a demo creation.

Reference material

• OpenAI Cookbook: Build an Agent Improvement Loop with Traces, Evals, and Codex (Confirmed 2026-05-19)
• OpenAI Agents SDK official document (confirmed 2026-05-23)
• OpenAI Agents SDK Tracing Document (Confirmed 2026-05-23)
• Promptfoo Official Site: LLM Evaluation and Regression Testing Tool (Verified 2026-05-23)
• HALO GitHub repository: Trace and feedback-based optimization tool (confirmed on 2026-05-23)