Google Genkit Middleware Commentary: Why agent apps must fix model/tool call boundaries in code before prompting

1. One-line problem definition

Key takeaway: As agent apps get smarter, the points of failure expand beyond the prompts.

When creating an AI agent app, I usually write a long list of rules in the prompt at the beginning. This is done by inserting statements such as "If it fails, try again", "Do not run dangerous tools", and "Do not read beyond this file" to the system prompt. The problem is that although these rules can change the model's response habits, they cannot definitively control the actual API failure, tool execution, file access, and approval flow.

Genkit Middlewarereleased by Google on May 14, 2026 addresses this issue at the code layer. We sandwich middleware between Genkit's generate() call and the tool execution loop to handle retries, model fallbacks, tool authorization, skill injection, file access, and logging with a common policy.

The scope of application is clear. It is suitable for teams that need to bundle multiple models and tools to create product features, and manage failures, costs, permissions, and logs on an operational basis. Conversely, if it is a simple summary chatbot, one-time demo, or internal PoC, the middleware design may be excessive.

2. First, conclusion

Key takeaways: Genkit Middleware is a tool that brings down “prompt rules” to “execution contracts”

I would not view Genkit Middleware as a model performance improvement feature. A more accurate name would be Agent execution boundary layer. Rather than what the model says, it is closer to fixing in code how far to retry when a model call fails, which tools to use for human approval, which directory to restrict file access to, and which logs to leave.

It is worth introducing if the agent calls tools that are difficult to reverse, such as writing, deleting, distributing, making payments, or modifying CRM files.
If you need to change multiple models such as Gemini, Claude, OpenAI, and local models depending on the situation, you can commonize the fallback policy.
If you need to track which layer failed when a failure occurred, the Developer UI and middleware logs are helpful.

There are cases where just observation is still necessary. Apps that call only a single prompt on a single model, content creation apps without tool execution, or apps that have low traffic and require human failure recovery are better to start with a simple structure first.

3. Decomposition of core structure

Key takeaways: Genkit Middleware divides the creation loop into three sections, each providing a different control point:

Genkit's generate() call is not a structure that simply asks the model once and ends. When a tool is connected, the loop repeats: the model creates an answer, requests the necessary tool, the application runs the tool, and passes the results back to the model.

Tier	Execution time	Suitable policy	Practical example
Generate Hook	Each iteration of the tool loop	Context injection, message rewriting, per-conversation policy	Change reference document range based on user level
Model Hook	For each model API call	Retries, fallback, caching, latency logging	Switch to another model when quota is exceeded
Tool hook	Each time a tool is run	Audit log by approval gate, sandbox, tool	`deleteFile`, `deployProduction` Request for approval

The reason this structure is important is because it narrows down the scope of failure. Re-running the entire tool loop from the beginning when the model API fails with RESOURCE_EXHAUSTED may result in duplicate calls to tools that have already been run. Genkit's Retry middleware, according to its official description, only retries model calls and does not rerun surrounding tool loops.

4. Description of design intent

Key takeaway: The key to middleware design is to reduce the unpredictability of agents, but not hide everything in the framework.

The design chosen by Genkit Middleware is similar to web server middleware. Just as a request goes through authentication, logging, validation, and routing in that order, model calls also go through a chain of common processes like retries, filtering, tool authorization, and file access restrictions.

What you get is three things. First, we take the policy out of the prompt and stick it in code. Second, the same policy is reused across multiple flows and multiple teams. Third, you can view and debug middleware execution traces in the Developer UI.

An alternative is to give up. If you get the middleware order wrong, it will behave differently than intended. The official announcement explains that the stack is built from left to right, with the first item being the outermost wrapper. In other words, the middleware is not magic automation, but policy code with an execution order

5. Evidence and Comparison

Key takeaways: Genkit Middleware's competition is not with other models, but with scattered exception handling and prompt-based operating rules.

Apply policy close to tool loop with

Approach	Advantages	Limit	Recommendation status
Prompt Rule	Fastest and requires less extra code	The guarantee that the model will be maintained is weak	Read-only chatbot, demo
Direct if statement and try/catch	Less framework-dependent and immediately understandable	As flow increases, policies disperse	Small service, single model
Graphic Orchestration	Strong in complex state transitions and long-term operations	High initial training cost	Multi-step workflow, state machine
Genkit Middleware	`generate()`	Depends on Genkit ecosystem and sequence design	Genkit-based product features, tool call agents

Based on official data, the representative middleware currently provided are Retry, Fallback, Tool approval, Skills, and Filesystem. In JavaScript, it can be used as the @genkit-ai/middleware package, and in Go, the path to the core Genkit module and middleware plugin is provided. As of the announcement, TypeScript, Go, and Dart are supported, and Python support is planned at a later date.

6. Actual operation flow / step-by-step execution method

Key takeaways: To start small, you only need to stick to three things first: retry, tool authorization, and file access restrictions.

mkdir my-genkit-app
cd my-genkit-app
npm init -y
npm pkg set type=module
npm install genkit @genkit-ai/google-genai @genkit-ai/middleware
npm install -D typescript tsx
export GEMINI_API_KEY=<your_key>

Then attach the middleware to the model call. The code below is an example of the concept. In real projects, you will need to change the tool name and approval UI to match your service.

import { genkit } from 'genkit';
import { googleAI } from '@genkit-ai/google-genai';
import { retry, fallback, toolApproval, filesystem } from '@genkit-ai/middleware';

const ai = genkit({
  plugins: [googleAI()],
  model: googleAI.model('gemini-2.5-flash'),
});

const response = await ai.generate({
prompt: 'Create a draft README in the workspace and edit only the necessary files.',
  tools: [writeFileTool],
  use: [
    retry({ maxRetries: 3, initialDelayMs: 1000, backoffFactor: 2 }),
    fallback({
      models: [googleAI.model('gemini-flash-latest')],
      statuses: ['RESOURCE_EXHAUSTED', 'UNAVAILABLE'],
    }),
    toolApproval({ approved: ['readFileTool'] }),
    filesystem({ rootDirectory: './workspace', allowWriteAccess: false })
  ],
});

First, record the model call failure rate and average delay time.
Attach Retry only to requests with a high failure rate.
Attach fallback when quota or supplier failure is a real problem.
Leave Tool approval as the default for write/delete/distribute/payment tools.
File access is limited to the working directory, not the project root.
Check the middleware order and execution trace in the Developer UI.

7. Pitfalls

Key summary: Middleware can be a fail-safe or, if used incorrectly, a layer that hides failures.

Trap 1: Mistaking Retry for universal recovery

Retry is helpful for temporary network errors. However, if the prompt is wrong or the tool input schema is wrong, you will get the same failure even if you repeat it three times. Limit the status codes to be retried, and log the cause if it fails even after retries.

Ptrap 2: Ignoring personality differences in fallback models

Falling back from a larger model to a smaller model may improve cost and availability, but may result in different output format and inference quality. After fallback, it is safer to apply schema validation to the results and mark important operations as temporary response status.

Pitfall 3: Think of Tool approval only as a UI event

Tool approval isn't just a button, it's an audit log. Recovery after an accident is difficult if you do not keep track of who approved what, when, with what input, and with what tools.

Trap 4: Filesystem root is set too wide

If you open the entire project like

rootDirectory: './', the agent can read even unnecessary files. It is better to create a temporary directory for each task and turn off write permission by default.

8. Strengths and Limitations

Key takeaways: Genkit Middleware is practical for teams using Genkit, but it is not the standard answer for all agent systems.

The strengths are clear. You can intervene between model calls and tool calls, providing greater control than prompts. It reduces the need to create common operational functions such as Retry, Fallback, and Tool approval each time. The execution flow can be viewed in the Developer UI, making it easy for even novice developers to trace why this response occurred.

There is also a limit. Teams that have already tightly designed state transitions with LangGraph or their own workflow engine may feel like Genkit Middleware is a redundant layer. The fact that Python support is still in the planned stages is also a realistic limitation for Python-centric teams. Additionally, the presence of middleware does not automatically guarantee security. Secret management, audit log storage, and deployment approval processes must be designed separately.

9. Points to study more deeply

Key takeaways: To properly use middleware, you need to understand the tool call loop and authorization resume flow before Genkit itself.

Tool call loop: Iterative structure where the model makes a tool request, the app runs the tool, and puts the results back into the model.
Interrupt and resume: This is the flow of stopping a dangerous tool, receiving human approval, and then resuming with the same conversation record.
Scheme validation: If you attach a fallback model or custom filter, you will need to validate the output format again.
Observability: Request ID, model name, middleware name, tool name, and approver must be left in the operation log as well as in the Developer UI.

10. Action Checklist + Author's Perspective

Key summary: The quality of an agent app is determined by how quickly failure, permission, and log criteria are coded, rather than model selection.

Have you classified the tool as read-only, writable, destructible, or incurring external costs?
Do destructible tools basically pass Tool approval or a separate approval gate?
Have you checked whether Retry only retries the model call or does not duplicate tool execution?
When using a fallback model, are the output schema and quality degradation criteria verified separately?
Is the file access root restricted to the working directory and default write permissions turned off?
Is the middleware order fixed in the team template, and is a review received when changing the order?
Can you track the model name, tool name, approval status, and failure code in both the Developer UI and the operation log?

Definition of Done: If you can actually block one dangerous tool, resume after approval, and reproduce which middleware was involved from the failure log, the first phase of introduction is complete.

My conclusion is clear. Genkit Middleware is not a feature that makes agents smarter. When an agent has already done enough work, it is an operating layer that bundles the work into a manageable form within the product.