Can NVIDIA Nemotron 3.5 Super VL Reduce Agentic AI Costs on AWS?
AWS details deployment of a 120B-parameter mixture-of-experts model that activates only 12B parameters per token on EKS and SageMaker.
Key points
- NVIDIA Nemotron 3.5 Super VL is a 120B-parameter MoE model that activates 12B parameters per token.
- The model supports multimodal tasks including code validation, web research and document intelligence.
- AWS provides reference implementations for deployment on EKS, EKS Auto Mode and SageMaker HyperPod EKS.
AWS published an advisory detailing how to deploy NVIDIA Nemotron 3.5 Super VL on Amazon Elastic Kubernetes Service and SageMaker HyperPod EKS.confirmed The post highlights the model’s architecture and provides technical guidance for running agentic AI workflows on AWS infrastructure.
Evidence: confirmed
Accelerate agentic AI with NVIDIA Nemotron models on Amazon EKS and SageMaker HyperPod EKSaifleet-aws
The announcement
NVIDIA Nemotron 3.5 Super VL is a 120B-parameter mixture-of-experts model that activates only 12B parameters per token.reported It combines reasoning capabilities with tool use and multimodal understanding for text, images, documents and video. AWS states that this single model can serve both text and visual tasks efficiently.
Evidence: reported
Accelerate agentic AI with NVIDIA Nemotron models on Amazon EKS and SageMaker HyperPod EKSaifleet-aws
The context
The model is distilled from Nemotron 3 Ultra and uses a hybrid Mamba-Transformer MoE architecture.reported NVIDIA publishes weights, training datasets and post-training recipes, allowing customization for domain-specific workloads. The model integrates with open agent frameworks through NVIDIA reference implementations and the NVIDIA Agent Toolkit.
Evidence: reported
Accelerate agentic AI with NVIDIA Nemotron models on Amazon EKS and SageMaker HyperPod EKSaifleet-aws
What the evidence means
NVIDIA reports performance scores of 63.44 percent on Hermes Bench and 87.05 percent on LangChain agent evaluations.reported The model also achieves 80.8 percent on the CharXiv chart-reasoning benchmark.reported Sparse routing allows compute per token to match smaller models while GPU memory holds full weights.
Evidence: reported
Accelerate agentic AI with NVIDIA Nemotron models on Amazon EKS and SageMaker HyperPod EKSaifleet-aws
Evidence: reported
Accelerate agentic AI with NVIDIA Nemotron models on Amazon EKS and SageMaker HyperPod EKSaifleet-aws
What business readers should know
Architects can deploy the model on standard Amazon EKS, EKS Auto Mode or SageMaker HyperPod EKS. With EKS Auto Mode, AWS manages scaling, GPU resources and Elastic Fabric Adapter support. Reference implementations run on p6-b200.48xlarge and p5en.48xlarge instances, using NVIDIA Dynamo for aggregated or disaggregated serving.reported
Evidence: reported
Accelerate agentic AI with NVIDIA Nemotron models on Amazon EKS and SageMaker HyperPod EKSaifleet-aws
Why it matters
- Mamba-2 layers maintain constant-size state during generation, preventing memory growth with sequence length and keeping long contexts affordable.
- Sparse routing activates only 12B of 120B parameters per token, reducing compute costs compared to dense models of similar size.
- Disaggregated serving options allow prefill and decode stages to run on different nodes, optimizing resource utilization for vision-language tasks.
- Published weights and recipes enable customization for specific domains without relying on proprietary API access or black-box models.
Topics: NVIDIA, AWS, Nemotron, agentic AI, mixture-of-experts
Updated
Ask the article
Local keyword search of this article; no generated answers.Which of these is confirmed?
Check the answer
AWS published an advisory detailing how to deploy NVIDIA Nemotron 3.5 Super VL on Amazon Elastic Kubernetes Service and SageMaker HyperPod EKS.
The matched primary source and announcement wording support the confirmed label.
Did this article give you the evidence you needed?
Your choice stays on this device. No public tally.Questions readers ask
What hardware instances are validated for running Nemotron 3.5 Super VL?
The architecture has been validated on p6-b200.48xlarge instances with eight NVIDIA B200 GPUs and p5en.48xlarge instances with eight NVIDIA H200 GPUs.
Does the model support video understanding tasks?
Yes, the model supports video understanding tasks including search, content review and incident analysis, alongside text, image and document processing.
How does the model handle long context windows efficiently?
Mamba-2 layers carry a constant-size state during generation, so memory does not grow with sequence length like traditional KV cache implementations.
Connected stories
Reading receipt
- NVIDIA Nemotron 3.5 Super VL is a 120B-parameter MoE model that activates 12B parameters per token.
- The model supports multimodal tasks including code validation, web research and document intelligence.
- AWS provides reference implementations for deployment on EKS, EKS Auto Mode and SageMaker HyperPod EKS.