Google Virgo Network + Agent Sandbox Commentary: Why agent era infrastructure should separate east-west traffic and untrusted code execution layer rather than GPU

Publication date: 2026-04-28 | Category: Development information

1) One-line problem definition

Key takeaways: Agent systems take precedence over model accuracy East-West traffic explosionand Untrusted codes The operation is unstable due to execution.

Target readers are lead developers and platform engineers responsible for in-house AI platforms, MLOps, and cloud infrastructure. This explanation is based on teams that want to go beyond a single chatbot and run workloads that mix tool calls, code execution, and multi-agent collaboration.

The scope of this article is to explain how to design the agent infrastructure by separating the layers based on the Virgo Network, Axion N4A, and GKE Agent Sandbox released at Google Cloud Next '26. Comparison of individual model quality or detailed calculation of the price list of a specific vendor is excluded from the scope.

2) Conclusion first

Key takeaway: Agent operations team isolates network fabric and executions before buying more GPUs You need to separate the layer first.

The conclusion I draw is simple. In an agent environment where dozens to hundreds of subtasks occur simultaneously when a user requests a single request, a bottleneck will soon occur if model serving, general applications, and untrusted code execution are run on the same network and same node policy.

This is also the point that Google emphasized this time. Virgo Network separates east-west traffic between accelerators into a separate scale-out fabric, and GKE Agent Sandbox places tool calls and code execution in an isolated sandbox. In other words, the key to agent infrastructure is what and where to separate from the larger model.

Conversely, if your team is still focused on a single inference API call and very little code execution, this design is overkill. At that stage, plain Kubernetes + Inference Gateway + Observability Enhancement is sufficient.

3) Core structure decomposition

Key summary: Agent infrastructure should be divided into at least three layers to make it easier to judge.

• Accelerator layer: TPU/GPU is responsible for model learning and inference. Bandwidth, aggregation, KV cache, and RDMA efficiency are key here.
• Scale-out network layer: Handles east-west traffic between accelerators. Google highlighted up to 134,000 TPU 8t chips in the Virgo Network, 47Pb/s non-blocking bandwidth, up to 4x the bandwidth per accelerator compared to the previous generation, and 40% lower no-load latency.
• Agent Execution Layer: Responsible for planning, tool invocation, untrusted code execution, and feedback loop. Google bundled Axion N4A and GKE Agent Sandbox in this area.

The important point is that while all three layers are needed, should not be scaled in the same way. GPU should be focused on throughput, fabric should be focused on delay and collective communication, and sandbox should be focused on isolation and congestion control.

4) Explanation of design intent

Key takeaway: Google's design intent was not to run everything in one cluster, but to separate conflicting loads.

Virgo Network has a separate east-west fabric from Jupiter for north-south traffic. This choice is a declaration that general service networks and AI collective communications will not be treated with the same rules. The reason is simple. This is because AI learning and large-scale inference are so sensitive that the entire goodput can be destroyed by a single momentary burst or straggler.

Agent Sandbox is in the same context. Although agent-generated code or tool calls are functionally part of an application, they are potentially untrusted input from a security perspective. Therefore, it is better for operational stability to separate it into a sandbox rather than run it directly on a regular worker.

There are also things this design gives up. Operational simplicity, vendor neutrality, and upfront learning costs. What you get in return is a smaller failure radius, cost predictability, and faster recovery from agent flooding. I think this exchange is reasonable enough for a team that has entered the agent operation phase.

5) Evidence and comparison

Key takeaways: Comparison is based on isolation level, network scalability, and operational resilience, not GPU numbers.

• Cost: On the surface, having a sandbox may seem like it costs more resources, but reducing the number of situations where runaway agents grab expensive GPU inference paths can lower the total cost.
• Time: Google touted Agent Sandbox as a native service, Axion N4A claimed up to 2x price/performance compared to regular x86 VMs, and the Agent Sandbox combination claimed up to 30% price/performance compared to the next competing hyperscaler. However, this figure must be re-verified by pilot.
• Accuracy: The model accuracy itself does not change, but reducing the straggler/delay deviation increases the overall agent success rate. This is because agents care more about overall chain completion rate than single token speed.
• Operability: AWS EFA is powerful, but as per documentation, Huge Pages, VPC CNI, and instance/availability zone constraints must also be managed. On the other hand, Google's approach is different in that it commercializes the network and sandbox layers together.

6) Actual operation flow / step-by-step execution method

Key summary: Instead of switching from scratch, start with a pilot that splits inference, execution, and observation into three branches.

1. Week 1: Measure call chain
   Measure how many model calls, tool calls, and code executions one user request makes internally. Without this number, separation priorities cannot be determined.
2. Week 2: Separate inference path and execution path
   Leave model serving as existing GPU/TPU workload, and move code execution/tool ​​wrapper to a separate sandbox node pool or isolated runtime.
3. Week 3: Fix network bottleneck metrics
   P95 Cluster delays, job straggler rate, sandbox failure rate, and average number of tool runs per request into a dashboard.
4. Week 4: Add limit policy
   Declaratively fix the number of concurrent executions per agent, CPU upper limit, network egress policy, timeout, and number of retries.
5. After week 5: GPU optimization is followed by
   KV cache, RDMA, and high-performance fabric tuning only after the isolation layer is stabilized. If you reverse the order, the causes of the failure will be mixed.

#Agent Sandbox example preparation variables (based on Google official documentation)
export PROJECT_ID=$(gcloud config get project)
export CLUSTER_NAME="agent-sandbox-cluster"
export LOCATION="us-central1"
export CLUSTER_VERSION="1.35.2-gke.1269000"

#Example of enabling Agent Sandbox when creating a new Autopilot cluster
gcloud beta container clusters create-auto "$CLUSTER_NAME" \
  --project "$PROJECT_ID" \
  --location "$LOCATION" \
  --cluster-version "$CLUSTER_VERSION" \
  --enable-agent-sandbox

#Example of operational metrics to collect first in the agent execution layer
metrics:
  - agent_request_total
  - tool_execution_total
  - sandbox_startup_p95_ms
  - sandbox_timeout_total
  - model_inference_p95_ms
  - straggler_detection_total

7) Mistakes/Pitfalls

Key takeaways: Failure comes more often from lack of separation than from lack of skills.

1. Pitfall: Co-locating agent tool executions on regular app nodes
   Prevention: Default to sandbox-only node pool or isolated runtime Leave
   Recover: Emergency quarantine the tool execution workload and lower the concurrency limit to 1 order.
2. Trap: Judging that the infrastructure is healthy by only looking at GPU utilization
   Prevention: Request completion rate, straggler rate, and sandbox failure rate View together:
   Recovery: First deconstruct in the dashboard whether it is an execution chain bottleneck rather than a model delay.
3. Pitfalls: Reflecting vendor benchmarks as is in budget
   Prevention: Based on actual workload in 2-week pilot, P50/P95 and Remeasure the cost per request.
   Recovery: Recalculate GPU, CPU, and sandbox resource allocation by dividing them into separate cost centers.

8) Strengths and limitations

Key takeaways: This structure is strong for large-scale agent operations, but it is not a one-size-fits-all solution for all teams.

• Strengths: Reduced failure radius, tool execution isolation, east-west traffic optimization, cost separation of inference and execution layers.
• Strengths: The AI ​​Hypercomputer layer and the GKE agent operation layer are connected in the same story, making it easy to explain the architecture.
• Limit: Increases vendor lock-in. In particular, although the design philosophy of Virgo Network can be learned, it cannot be directly ported.
• Limitation: Since many of the disclosed figures are announced by suppliers, performance and price efficiency must be independently verified.
• Limitations: This may only increase operational complexity for teams that have barely yet run code.

9) Points to study more deeply

Key takeaway: This topic is better than GPU: goodput, straggler, sandbox isolation It is better to delve into three concepts first.

• You should first read Google's explanation of the east-west / north-south separation to see why the Virgo Network split from Jupiter.
• From an agent operations perspective, the GKE Agent Sandbox and Code Execution Isolation documents should be viewed together.
• Teams considering AWS alternatives should first check the Availability Zone, Huge Pages, and VPC CNI constraints in the EFA document.
• To see learning/inference productivity, you need to look at goodput, MTTR, and straggler detection automation together rather than raw FLOPS.

10) Execution Checklist + Author’s Perspective

Key takeaways: The criterion for adoption is whether our team's agents execute the code.

• The average number of model calls and tool executions per user request was measured
• Separated untrusted code execution workloads from regular app nodes
• View inference delay and sandbox delay as separate indicators
• The number of simultaneous agent executions and default timeout values ​​are declaratively fixed
• There are plans to re-verify vendor performance figures as a pilot
• In case of a failure, you can determine which of the GPU layer, network layer, and sandbox layer is the bottleneck within 10 minutes

Definition of Done: If the request completion rate, sandbox failure rate, inference P95, and cost per request can be separately measured for two consecutive weeks in the agent pilot, and the cause of the failure can be immediately classified into one of the three tiers, the minimum operating standards required for adoption are met.

Author's perspective: I highly recommend this direction to any team running a multi-agent or code execution product. Especially in teams where GPU utilization is high but user perceived quality is low, the problem is likely not the model but the fabric and execution layer mix. On the other hand, if it is a simple RAG or single chatbot stage, it is more realistic to just refer to design philosophies such as Virgo and start with the observability and inference gateway now.

Reference material

• Google Cloud Blog - What’s next in Google AI infrastructure: Scaling for the agentic era (Confirmation date: 2026-04-28)
• Google Cloud Blog - Introducing Virgo Network, Google’s scale-out AI data center fabric (Confirmation date: 2026-04-28)
• Google Cloud Blog - What’s new with compute: Scaling core and agentic workloads (Confirmation date: 2026-04-28)
• Google Cloud Docs - About GKE Agent Sandbox (Confirmation date: 2026-04-28)
• Google Cloud Docs - Enable Agent Sandbox on GKE (Confirmation date: 2026-04-28)
• AWS Docs - Run machine learning training on Amazon EKS with Elastic Fabric Adapter (Confirmation date: 2026-04-28)