NVIDIA Nemotron-Cascade 2 complete guide: MoE model introduction strategy that delivers trillion-level performance with 3 billion parameters

Problem Definition: Why is this model important

Target audience: Enterprise AI teams, ML engineers, startup CTOs who need frontier performance on coding/math tasks while reducing inference costs.

Problem solved: 1 trillion+ parameter model has unrealistic inference cost and delay. However, when that level of performance is required for advanced coding (IOI/ICPC) and math (IMO) tasks, the limitations of a small model were clear.

Solution for Nemotron-Cascade 2: 30 billion Only 3 billion active when inferred from parameter MoE structure. Achieved Gold Medal performance equivalent to DeepSeek-V3.2-Speciale (671 billion, A37B) with 20 times fewer parameters.

Scope of application: Production environments requiring coding agents, mathematical reasoning, algorithmic problem solving, and complex multi-step reasoning.
Scale of application: Based on memorization of knowledge. QA (somewhat weak in MMLU-Pro, etc.), simple creative writing, and agentic work (lower than competing models in τ²-Bench and Terminal Bench).

Evidence and comparison: Skills based on benchmarks

Based on NVIDIA official benchmark data released in March 2026. Compare with equivalent size models and larger models.

Benchmark	Nemotron-Cascade 2 (30B, A3B)	Qwen3.5-35B-A3B	Kimi-K2.5-1T	Remarks
AIME 2025 (Mathematics)	92.4 (98.6†)	91.9	-	†TIR mode
HMMT Feb25 (Mathematics)	94.6	89.0	-	+5.6p
LiveCodeBench v6 (coding)	87.2 (88.4†)	74.6	85.0	Exceeds the 1 trillion model
IOI 2025	🏅 439.3	348.6	-	Gold Medal
ArenaHard v2 (sort)	83.5	65.4	-	+18p difference
τ²-Bench (Agentic)	58.9	81.2	-	Weakness area
MMLU-Pro (Knowledge)	79.8	85.3	-	Weakness area

Source: NVIDIA arXiv 2603.19220 (2026-03-20), Hugging Face model card

Key judgment criteria:

Coding/Math Reasoning Focused: Choose Nemotron-Cascade 2 (best performance/cost)
Knowledge-based QA-centric: Consider Qwen3.5 or larger models
Agent task-oriented: Based on τ²-Bench results, Qwen3.5-35B-A3B is more suitable
Cost priority: 3 billion active parameters = H100 262K context inference possible in 1 sheet

Core technology: Cascade RL + MOPD

Cascade Reinforcement Learning

The existing RL method learned all domains (mathematics, coding, sorting, etc.) simultaneously, resulting in a ‘forgetting problem’. Cascade RL solves this with sequential domain-specific learning.

Learning sequence (6 steps):

Instruction-Following RL (IF-RL)
Multi-Domain RL (group of similar formats such as mathematics, reasoning, etc.)
RLHF (human preference sorting)
Long-Context RL (32K input, 49K maximum sequence)
Code RL (competitive programming, binary reward)
SWE RL (Software Engineering, Agent/Non-Agent)

3 advantages:

Domain-specific RL barely reduces previous benchmark performance
Optimized hyperparameter tuning for each domain
Improve computing efficiency with same-domain deployment

Multi-Domain On-Policy Distillation (MOPD)

A technique to readjust the overall balance by using the checkpoint that showed the highest performance in a specific domain in the middle of Cascade RL as a 'teacher model'.

Efficiency compared to GRPO: Based on AIME25, MOPD reaches teacher level (92.0) in 30 steps. GRPO only reached 91.0 in the same step.

How to do it step by step: Deploy vLLM

1. Prerequisites

#vLLM 0.17.1 or higher required
pip install vllm>=0.17.1

#GPU memory: 24GB or more recommended (H100/A100/RTX 4090)
#262K context is 80GB or more

2. Run default server

vllm serve nvidia/Nemotron-Cascade-2-30B-A3B \
  --port 8000 \
  --tensor-parallel-size 1 \
  --gpu-memory-utilization 0.9 \
  --max-model-len 262144 \
  --reasoning-parser nemotron_v3 \
  --mamba-ssm-cache-dtype float32 \
  --trust_remote_code

3. Tool Call Supported Version

vllm serve nvidia/Nemotron-Cascade-2-30B-A3B \
  --port 8000 \
  --tensor-parallel-size 1 \
  --gpu-memory-utilization 0.9 \
  --max-model-len 262144 \
  --reasoning-parser nemotron_v3 \
  --mamba-ssm-cache-dtype float32 \
  --trust_remote_code \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder

4. Switch Thinking/Instruct mode

from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained('nvidia/Nemotron-Cascade-2-30B-A3B')

messages = [
  {"role": "system", "content": "You are a helpful assistant."},
  {"role": "user", "content": "Solve: What is the sum of first 100 prime numbers?"}
]

#Thinking mode (deep reasoning, using  tag)
prompt_thinking = tokenizer.apply_chat_template(
  messages, tokenize=False, 
  add_generation_prompt=True, 
  enable_thinking=True
)

#Instruct mode (quick response, skips the thought process)
prompt_instruct = tokenizer.apply_chat_template(
  messages, tokenize=False, 
  add_generation_prompt=True, 
  enable_thinking=False
)

5. API call example

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "nvidia/Nemotron-Cascade-2-30B-A3B",
    "messages": [{"role": "user", "content": "Write a Python function to check if a number is prime"}],
    "temperature": 1.0,
    "top_p": 0.95
  }'

Recommended sampling settings: temperature=1.0, top_p=0.95 (NVIDIA official recommendation)

Mistakes/Pitfalls

1. Overconfidence in performance in agentic tasks

Problem: Significantly higher in τ²-Bench (58.9) and Terminal Bench 2.0 (21.1) compared to Qwen3.5-35B-A3B (81.2, 40.5) Low.

Prevention:Use autonomous agents, multi-step tools. Use different models in parallel for workflows. In SWE-Verified, it is good at 50.2 in the OpenHands framework, but lower than the competing model (69.2).

2. Limitations in knowledge-based QA

Problem: MMLU-Pro 79.8, GPQA-Diamond 76.1, lower than Qwen3.5 (85.3, 84.2).

Prevention: Use RAG pipelines or larger models for tasks where fact retrieval is important. This model is optimized for ‘reasoning’ rather than ‘knowledge memorization’.

3. Mistake in setting context length

Problem: Setting --max-model-len larger than GPU memory causes OOM.

Prevention: 24GB GPU → 32K~64K, 80GB GPU → up to 262K. NIAH@1M test is 99.0, but memory calculation is required for actual deployment.

4. Missing license check

Problem: NVIDIA Open Model License, but conditions need to be checked for commercial distribution.

Prevention: Be sure to review the Commercial Use Terms in the License section of the Hugging Face model card.

5. Thinking mode context management

Problem: In a multi-turn conversation, if content continues to accumulate, the context explodes.

Prevention: According to the official guide, only the final summary of the previous turn's Thinking content is added to the history. Utilize tokenizer basic operation.

Execution Checklist

☐ Verify vLLM version 0.17.1 or higher
☐ Set max-model-len based on GPU memory (24GB→64K, 80GB→262K)
☐ Add required --reasoning-parser nemotron_v3 flag
☐ --mamba-ssm-cache-dtype set float32 (stability)
☐ temperature=1.0, top_p=0.95 Apply sampling parameters
☐ Use OpenHands framework when working with agents (OpenCode not supported)
☐ NVIDIA Open Model License commercial terms reviewed

Definition of Done: vLLM server starts up properly, returns correct answers to sample math problems, and P99 delay is less than 5 seconds in production traffic.

Reference material

Author's perspective

Recommended: Highly recommended for teams whose core workload is solving coding agents or algorithm problems. LiveCodeBench 87.2, IOI Gold Medal performance is equivalent to a 1 trillion parameter model, while the inference cost is 1/20th. In particular, H100 1 can handle 262K contexts, resulting in significant infrastructure cost savings.

Not recommended for: In productions where fact-based QA, agentic autonomous work, and knowledge discovery are important, consider Qwen3.5-35B-A3B or larger models. τ²-Bench 58.9 vs 81.2 The difference is felt in actual agent performance.

Ultimate judgment: The combination of Cascade RL + MOPD is a sign of changing the ‘model size competition’ paradigm. If a 20 times smaller model achieves equivalent performance using only post-learning techniques, future corporate AI strategies should invest in ‘better learning methods’ rather than ‘bigger models’.