Sakana Fugu Commentary: Why model orchestration should design routing costs, verification logs, and fallback boundaries before performance tables

One-line problem definition: The core of Fugu is not a “single stronger model,” but an operational approach that selects, delegates, verifies, and integrates multiple models behind an API. The issue that development teams need to look at is how to control routing failures, cost explosions, and lack of observability rather than whether or not they are #1 in the benchmark. This article is not an introduction to adopting Fugu right away, but rather a practical guide that outlines what boundaries should be designed before putting model orchestration into a product.

1. Problem Definition: It is not a problem of adding one more model

Key takeaways: Model orchestration is an operational architecture issue, not a performance issue

AI Times reported on June 23, 2026 that Sakana AI unveiled the model orchestration platform Fugu. As of the official announcement, Fugu appears to be one OpenAI compatible API, but under the hood it picks up multiple LLMs, splits the work, and combines the results.

This structure has the advantage of decoupling model selection from application code. Instead, new operational questions arise, such as “Why was this model called?”, “Which model did we fall back to if it failed?”, and “How much of the internal orchestration token was used?”

Scope of application is tasks that do not end with a single model call, such as coding review, research, security analysis, and review of long documents. Conversely, it may be overkill for tasks where latency and cost are more important, such as short FAQs, fixed categories, and simple summaries.

2. First, conclusion: Fugu should be viewed as a ‘high-end work router’ rather than a ‘basic model’

Key takeaway: Sending all requests to Fugu Ultra will likely lead to cost and debugging issues before performance.

If you are looking at

Fugu, your first introduction should be some operations that have high failure costs, not all traffic. For example, detecting security vulnerabilities in PR reviews, writing long research reports, researching patents and papers, and analyzing the causes of complex failures are “tasks that people have to look at again for a long time if they make a mistake once.”

The recommendation method is 3 steps. Basic requests are processed by the existing single model, and only requests with high difficulty scores are sent to Fugu. Among them, only requests where accuracy and depth are truly important are uploaded to Fugu Ultra.

It is also clear that it is not recommended. For teams that have not yet measured the cost of calling a single model, teams without prompts, evaluation sets, and logs, and teams with unclear personal information processing boundaries, it will be more difficult to find the cause of the problem if Fugu is added first.

3. Core Decomposition: Four Layers Behind a Single API

Key summary: Fugu can be understood by dividing it into call interface, orchestrator, agent pool, and verification/synthesis stages.

First, OpenAI compatible API layer

Developers call Fugu like a model. According to the official model document, it supports Responses API, Chat Completions API, and Models API, and model IDs are provided as fugu, fugu-ultra, fugu-ultra-20260615.

Second, orchestrator model layer

Fugu itself is a language model, solving some problems directly and delegating others to other models. The word “router” alone is not enough. This is because it is responsible for planning, delegation, review, and synthesis rather than simple branching.

Third, agent pool layer

The official announcement explains that Fugu calls various LLM pools. However, it has not been disclosed which models are used and how they are distributed. This non-disclosure is both a competitive advantage and an operational risk.

Fourth, verification/synthesis layer

Partial answers created by multiple models should not be sent to users as is. You need to merge them into a final answer, sort out conflicts, and filter out failed inferences. If this step is weak, the quality will be unstable even if many models are used.

4. Explanation of design intent: Strategy to grow coordinators rather than one large model

Key summary: Fugu's design intent is closer to “combining strong models for each situation” rather than “putting all knowledge into one model”.

In its official statement, Sakana AI views single vendor dependence as an operational vulnerability. This is because if access to a specific model is blocked due to regulations, pricing, obstacles, or contractual issues, the entire service may be shaken. Fugu attempts to alleviate this problem with model pool replacement and dynamic routing.

What you gain is flexibility. As new models come out, they can be put into the pool without major changes to the application code. Difficult requests can be handled by multiple expert models.

There is also giving up. If internal routing paths are not fully transparent, reproducibility and auditing become difficult. As the number of model calls increases, latency and cost increase. In particular, Fugu Ultra's official documentation states that internal orchestration tokens are actually subject to charging, so the assumption that “limiting only the final answer token to a short period is cheaper” is not correct.

5. Rationale and Comparison: Fugu, DIY Router, Workflow Orchestrator

Key takeaways: Comparison criteria are control, cost, observability and failure response, not performance tables.

Approach	Advantages	Limit	Correct situation
Fugu / Fugu Ultra	Model selection, delegation, and synthesis can be entrusted with a single API	Internal paths and model pool only visible, Ultra beware of orchestration token cost	Teams that want to quickly improve the quality of complex analysis, coding, and research
DIY Router	Direct control of routing standards, logs, budget, and privacy boundaries	You must create your own evaluation set, operation tool, and response to failures	Regulated industry, own model, team where cost optimization is key
LangGraph·Semantic Kernel type workflow	Status, approval, retry, and human review steps can be explicitly designed	Model selection intelligence requires separate implementation	Teams where work procedures are important and require audit logs

The benchmark should be used as a reference only. Sakana's official model page suggests that the Fugu Ultra scored LiveCodeBench 93.2, GPQA Diamond 95.5, and SWE Bench Pro 73.7. However, these numbers are a mixed comparison of supplier announcements, and actual product quality must be measured against your request distribution and failure criteria.

Related research also points in the same direction. A comparative study of financial document processing multi-agent reported that the reflexive structure yielded the highest F1 but at a cost 2.3 times higher than the sequential basis, while the hierarchical structure showed a better balance with an F1 of 0.921 and 1.4 times the cost. In other words, “more agents” is not always the answer.

6. Actual operation flow: Design the pilot before introduction like this

Key summary: Do not implement it immediately, but compare quality, cost, and delay before and after routing in the same dataset.

Step 1 is to divide the work. For example, if it is a PR review, separate assessment items such as security vulnerabilities, performance issues, type stability, missing tests, and missing documentation.

Step 2 places a difficulty gate. Divide the “General Model”, “Fugu”, and “Fugu Ultra” by rules such as number of file changes, security-sensitive paths, whether external input is handled, and whether migrations are included.

type Route = "single-model" | "fugu" | "fugu-ultra";

function routePullRequest(input: {
  changedFiles: number;
  touchesAuth: boolean;
  hasMigration: boolean;
  estimatedTokens: number;
}): Route {
  const risk =
    input.changedFiles * 1 +
    (input.touchesAuth ? 8 : 0) +
    (input.hasMigration ? 5 : 0) +
    (input.estimatedTokens > 60000 ? 4 : 0);

  if (risk >= 12) return "fugu-ultra";
  if (risk >= 6) return "fugu";
  return "single-model";
}

Step 3 places a cost limit. Fugu Ultra's max_output_tokens only applies to the final response, token usage in the orchestrator model can occur separately. Therefore, “maximum cost per request”, “maximum latency per operation”, and “number of retries” should be disconnected from the application side.

#Example of logs to be kept in pilot before operation
request_id=pr-1842
route=fugu-ultra
task_type=security_review
input_tokens=48210
final_output_tokens=3810
orchestration_tokens=measurements
latency_ms=measurement
human_acceptance=accepted_with_edits

Step 4 adds human evaluation. Just counting the number of issues found by the model can lead to overexploitation. “Issues actually corrected,” “Issues rejected by reviewers,” and “Serious issues missed” must be recorded together.

7. Mistakes and pitfalls: Model routing quietly accumulates failures

Key takeaways: The risks of Fugu-type systems are usually first revealed in logs, costs, and reproducibility rather than in response quality.

Mistake 1: Sending all requests to Ultra

Prevention: Put request difficulty gate and budget cap first. Recovery: Reduce tasks for Ultra by comparing costs and human approval rates for each task type in the last 7 days of logs.

Mistake 2: Estimating internal orchestration costs based only on final tokens

Prevention: Save orchestration token fields in Fugu Ultra as separate indicators. Recovery: Request types where the p95 cost exceeds the budget will be downgraded to a single model or generic Fugu.

Mistake 3: Mistaking benchmark scores for product success criteria

Prevention: Create 50 to 100 self-assessment sets and look at the quality, delay, cost, and rejection rate of the answers. Recovery: Retune routing policy based on actual user operation success rate, not a benchmark.

Mistake 4: Deciding personal/confidential data boundaries later

Prevention: First classify which data can go to which model pool. Recovery: Restrict sensitive paths to opt-out generic Fugu or your own router, and re-establish log retention policy.

8. Strengths and limitations: ‘AI sovereignty’ claims must also be translated into operational reality

Key takeaways: Fugu can reduce vendor dependence, but it doesn't give you complete control.

The strengths are clear. Tying multiple models behind one interface simplifies your application code. Even if a specific model failure or access restriction occurs, there is room to bypass it through routing. In complex tasks, it is possible that review and synthesis are better than single models.

You should also look at the limits. Regulatory response and reproducibility explanations are difficult if the internal model pool is private. Even with better quality, longer latency may not be suitable for real-time products. When orchestration tokens are included in billing, cost prediction becomes difficult for services with high usage.

So, the conclusion from the author's perspective is as follows. Fugu is more likely to view it as a “quality-increasing option for high-risk/difficult requests” rather than “the default for all AI calls.” Teams where auditing and data boundaries are key, such as finance, healthcare, or public, are better off designing explicit workflows and their own routing first rather than Fugu.

9. Points to study more deeply

Key takeaways: To understand Fugu, you need to look at orchestration papers, cost logs, and routing evaluation methods rather than model performance tables.

First, read the Fugu official announcement and model document to check the supported API and token charging structure. Second, we need to look at the Fugu technical report and the Trinity,Conductor paper to see what the “learned orchestrator” is trying to,optimize. Third, we need to refer to multi-agent cost-accuracy research to understand the cost curve that arises when increasing agents.

From a code perspective, we recommend creating the following four files or modules first. Routing rules are placed in router.ts, self-assessment sets are placed in eval-set.jsonl, budgets per request are placed in cost-budget.ts, and route·latency·cost·human_acceptance indicators are placed in review-dashboard.

10. Implementation checklist and completion criteria

Key summary: The criteria for completion of introduction is not “API call success” but “Can explain with numbers when to use and when not to use”.

We measured the quality, average cost, and p95 latency of the existing single model baseline.
Clearly dividing the types of work to be sent to Fugu and Fugu Ultra.
Comparison of single models, Fugu and Fugu Ultra, using more than 50 self-evaluation sets.
Stores the orchestration token, final output token, and total cost as separate fields.
Implemented limits on maximum cost per request, maximum delay time, and number of retries.
The policy documents which model pool sensitive data can go to.
Created a fallback to a single model or human review in case of routing failure.
Instead of benchmark scores, human approval rate and actual revision reflection rate were used as key indicators.

Definition of Done: Fugu The pilot is completed when the human approval rate increases compared to a single model in the applicable task, p95 cost and p95 delay time remain within predetermined limits, and the fallback path in case of failure is confirmed in logs.