Skip to content

strands-cosmos

NVIDIA Cosmos Reason VLM provider for Strands Agents — physical AI reasoning, video understanding, and embodied intelligence.

PyPI GitHub

Overview

Enables Strands agents to use Cosmos-Reason2 models for video captioning, driving analysis, robot planning, temporal reasoning, and physics understanding.

Three Ways to Use

  1. Strands Model — CosmosVisionModel drop-in for any agent
  2. Agent Tool — cosmos_vision_invoke inside Bedrock/OpenAI agents
  3. Text-only — CosmosModel for physics reasoning without vision

Install

pip install strands-cosmos strands-agents

Requirements: Python ≥3.10, NVIDIA GPU (24GB+ for 2B, 32GB+ for 8B)

Quick Start

from strands import Agent
from strands_cosmos import CosmosVisionModel

model = CosmosVisionModel(model_id="nvidia/Cosmos-Reason2-2B")
agent = Agent(model=model)
agent("Caption: <video>dashcam.mp4</video>")

from strands import Agent
from strands_cosmos import cosmos_vision_invoke

agent = Agent(tools=[cosmos_vision_invoke])
agent("Analyze this dashcam for safety: /path/to/video.mp4")

model = CosmosVisionModel(reasoning=True)
agent = Agent(model=model)
agent("<video>scene.mp4</video> Is this physically plausible?")

Capabilities

Capability Description
🚗 Driving Analysis Traffic, hazards, navigation from dashcam
🤖 Robot Planning Next-action prediction, 2D trajectory
🎬 Video Captioning Temporal-spatial descriptions
⚛️ Physics Reasoning Object permanence, causality
🔍 2D Grounding Bounding box localization
📍 Temporal Localization Event timestamps in video
🧠 Chain-of-Thought <think> reasoning before answers

Models

Model GPU Memory Architecture
Cosmos-Reason2-2B 24GB Qwen3-VL
Cosmos-Reason2-8B 32GB Qwen3-VL

Jetson Support

Battle-tested on Jetson AGX Thor. Ships a CUBLAS fix for compatibility:

strands-cosmos-fix-cublas    # auto-detect and fix
strands-cosmos-fix-cublas --revert  # undo