Skip to content
JetBrains Air Public Preview Practical Evaluation Guide: 5 things to check before adopting a multi-agent development environment
← Back to blog

JetBrains Air Public Preview Practical Evaluation Guide: 5 things to check before adopting a multi-agent development environment

Development·8 min read

JetBrains Air has been released to public preview. We have organized what a team that wants to handle Codex, Claude Agent, Gemini CLI, and Junie in one workspace should verify before introduction from the perspective of ACP, isolation, and operation.

1. Problem Definition: Which teams should review JetBrains Air now

JetBrains Air is not simply “a tool with another AI chat window.” As explained by JetBrains, it is close to the agentic development environment, which runs and switches multiple coding agents in parallel within one environment. The problem is that if you immediately adopt this type of tool across the company just because it looks good, it may actually conflict with existing IDE, security rules, and review processes, which can affect productivity.

This article is written for the following teams:

  • Teams that are already using different agents, such as Codex, Claude Agent, and Gemini CLI, but have scattered work contexts
  • Teams that want to experiment with agents while maintaining development standards including Git worktree, Docker isolation, and code review
  • Teams that want to leverage the JetBrains ecosystem and ACP to design an operating model without “specific vendor lock-in”

On the contrary, there are two things excluded from the scope of this article. First, it is still a scenario for direct deployment to operating systems other than macOS. As of the official announcement, the current public preview is macOS first, and Windows and Linux are “coming soon.” Second, it is an approach that completely abandons the IDE and replaces all development with an agent environment. JetBrains also describes Air as an agent orchestration-oriented environment rather than a tool that replaces existing IDEs.

2. Why It's Worth Watching Now: Air's Core Values ​​and Limitations

There were three most important points in the official announcement. First, it provides basic support for Codex·Claude Agent·Gemini CLI·Junie. Second, each task can be isolated into a Docker container or Git worktree and run in parallel. Third, the task context can be specified in units of specific lines, commits, classes, and methods, so structured code context can be delivered to the agent instead of vague text.

However, there are clear limitations. It is currently in the public preview stage focused on individual developer productivity, and the team collaboration feature is heralded as the “next step.” Operating system support is also prioritized for macOS. So the task now is not “adopt company-wide standard tools,” but High-quality pilot evaluation. Verification items must be fixed before introduction or not.

3. Comparison of alternatives: Where should you compare Air to make the right decision?

The most common mistake when evaluating Air is comparing it 1:1 to single agent experiences like Cursor, Claude Code, or Copilot CLI. In practice, the axis of comparison should be changed from “who is smarter” to “does the operating model fit the team?”

AlternativeStrengthsWeaknessThe right team
JetBrains AirMulti-agent switching, isolation per task, code context specification, JetBrains workflow affinityPublic preview, macOS first, team features still limitedTeams that are already using a mix of agents and want to create operating standards
Single agent CLI/editor combinationQuick start, individual tool optimization, low learning costSession distribution, context disconnection, parallel task management relies on human handsTeam that prioritizes individual productivity optimization
Center of ACP integration within IDEACP agent can be expanded while maintaining the IDE, reducing dependence on subscriptionsAgent orchestration UX can be simpler than AirTeam that only needs ACP-based experiments while maintaining the current IDE

Decision-making criteria must be at least the following four:

  • Time: Does parallel work reduce actual lead time or only increase inspection costs
  • Control: Is Docker/worktree isolation, log collection, and easy to attach approval procedures
  • Compatibility: Is the current development environment centered on macOS or highly dependent on WSL
  • Vendor Risk: Is it possible to secure replacement with ACP and BYOK

4. Step-by-step implementation: 5-day pilot evaluation process

The introduction review should be conducted as a pilot operation, not a functional demonstration. Below is the 5-day assessment process:

Day 1. Fixed evaluation range

  1. Select only one target repository. A medium-difficulty project with a mix of front-end and back-end is recommended.
  2. Choose only 3 tasks: 1 bug fix, 1 refactoring, and 1 document/test enhancement.
  3. Write down the success criteria in advance. Example: 30% faster PR creation time, 20% less human rework, 100% test pass rate.

Day 2. Agent combination design

Combine at least two of the agents supported by Air. For example, the roles are divided into Codex for structural exploration, Claude Agent for long context refactoring, and Gemini CLI for scriptability tasks. The key is not to “find the best model” but to create role boundaries which tasks will be assigned to which agents.

Day 3. Isolation strategy verification

Actually uses Docker containers or Git worktree isolation as officially announced. Launch two or more parallel sessions and record the following:

  • Dependency installation reproduction time
  • Task conflict?
  • Difficulty of reviewing diff before main branch reflection

Day 4. ACP scalability verification

Based on JetBrains official documentation, ACP-compatible agents can be installed through registry or manually registered by ~/.jetbrains/acp.json. If possible, try adding an ACP-based custom agent on one day of the pilot. Only then can you check whether protocol-based operation is possible, rather than a specific vendor UI.

{
  "default_mcp_settings": {
    "use_idea_mcp": true,
    "use_custom_mcp": true
  },
  "agent_servers": {
    "example-agent": {
      "command": "/path/to/agent",
      "args": ["acp"],
      "env": {
        "API_KEY": "redacted"
      }
    }
  }
}

This example is an abridged version of the JetBrains official ACP document structure. In actual operation, sensitive information must be separated into a separate secret management system.

Day 5. Write review/control report

On the last day of the pilot, please leave the following items on page 1.

  • Time required for each task and human inspection time
  • Strengths/failure patterns by agent
  • Whether conflicts are reduced compared to not using quarantine
  • Official introduction conditions: macOS pilot maintenance / Linux standby / limited introduction to specific teams, etc.

5. 4 frequently failed patterns and recovery methods

Failure 1. The illusion that “it is faster if all tasks are run in parallel”

Parallel execution increases throughput, but also increases the inspection queue. The solution is to match the number of parallel sessions to the team review capacity. Typically, one reviewer can properly review around two active sessions at the same time.

Failed 2. No agent role boundary

If you blindly throw the same problem at multiple agents, comparisons are possible, but no operational standards remain. The solution is to divide roles first, such as “Exploration/Modification/Verification,” and set a promotion rule to transfer them to another agent in case of failure.

Failed 3. ACP attached but no log left

JetBrains documentation has a separate method for collecting ACP logs. It is almost impossible to reproduce the problem if you do not leave logs while running a public preview. It is safer to turn on extended logging only in the development sandbox and include rules for masking sensitive information.

Failed 4. Ignoring operating system reality

Based on the current official guide, ACP-compatible agents do not support WSL. Additionally, the Air itself will be released first to macOS. If you are a Windows/Linux focused organization, it is not “standardization now” but “limited verification of macOS experimental group”.

6. Action Checklist and Definition of Done

  • One target repository and 3 representative tasks were selected
  • A table of roles for each agent was created
  • At least one of Docker or Git worktree isolation has actually been applied
  • Operated two or more parallel sessions and recorded conflict/review time
  • ACP registry installation or acp.json manual registration has been verified at least once
  • Documented log collection and sensitive information masking principles
  • Concluded as either formal introduction/suspended/conditional introduction

Definition of Done: If all 7 items above are met, the “time saving or context integration” effect is confirmed in documentation in at least 3 actual tasks, and a conclusion report including operating system constraints and security control measures remains, the pilot is considered completed.

7. Author's perspective: Do I recommend it now, or is it too early

My judgment is clear. Recommended for limited pilots by individuals or small macOS-focused teams. That's because Air tackles head-on the more practical problems of multi-agent orchestration and context management rather than competing for single-model performance.

However, it is still too early to introduce a company-wide standard. Currently, it is a public preview, Windows/Linux support is in progress, and the team collaboration function is also difficult to consider complete. Therefore, the most realistic strategy is “2-week pilot in macOS experimental group → Establish quarantine/log/role division standards → Expand OS support and then reevaluate.”

If you summarize it in one line, it looks like this. Air is of greater value to teams that want to bundle multiple agents into an operational systemthan to teams looking for the “smartest single agent”. If there is a team that fits those criteria, there is plenty of reason to look at it now.

Reference material

Share this article

Related articles

Take the AQ test

See your AI capability in three minutes. Assess recognition, utilization, verification, integration, and ethics at once, then receive practical insights.

Start the free AQ test