Strands Robots: Natural Language Robot Control 🤖

Control robots with natural language through Strands Agents.

Integrates NVIDIA Isaac GR00T for vision-language-action policies and LeRobot for universal robot support.

How It Works

graph LR
    A[Natural Language<br/>'Pick up the red block'] --> B[Strands Agent]
    B --> C[Robot Tool]
    C --> D[Policy Provider<br/>GR00T/Mock]
    C --> E[LeRobot<br/>Hardware Abstraction]
    D --> F[Action Chunk<br/>16 timesteps]
    F --> E
    E --> G[Robot Hardware<br/>SO-101/GR-1/G1]

    classDef input fill:#2ea44f,stroke:#1b7735,color:#fff
    classDef agent fill:#0969da,stroke:#044289,color:#fff
    classDef policy fill:#8250df,stroke:#5a32a3,color:#fff
    classDef hardware fill:#bf8700,stroke:#875e00,color:#fff

    class A input
    class B,C agent
    class D,F policy
    class E,G hardware

Architecture

flowchart TB
    subgraph Agent["🤖 Strands Agent"]
        NL[Natural Language Input]
        Tools[Tool Registry]
    end

    subgraph RobotTool["🦾 Robot Tool"]
        RT[Robot Class]
        TM[Task Manager]
        AS[Async Executor]
    end

    subgraph Policy["🧠 Policy Layer"]
        PA[Policy Abstraction]
        GP[GR00T Policy]
        MP[Mock Policy]
    end

    subgraph Inference["⚡ Inference Service"]
        DC[Docker Container]
        ZMQ[ZMQ Server :5555]
        TRT[TensorRT Engine]
    end

    subgraph Hardware["🔧 Hardware Layer"]
        LR[LeRobot]
        CAM[Cameras]
        SERVO[Feetech Servos]
    end

    NL --> Tools --> RT --> TM --> AS --> PA
    PA --> GP & MP
    GP --> ZMQ --> TRT --> DC
    AS --> LR --> CAM & SERVO

Quick Start

from strands import Agent
from strands_robots import Robot, gr00t_inference

# Create robot with cameras
robot = Robot(
    tool_name="my_arm",
    robot="so101_follower",
    cameras={
        "front": {"type": "opencv", "index_or_path": "/dev/video0", "fps": 30},
        "wrist": {"type": "opencv", "index_or_path": "/dev/video2", "fps": 30}
    },
    port="/dev/ttyACM0",
    data_config="so100_dualcam"
)

# Create agent with robot tool
agent = Agent(tools=[robot, gr00t_inference])

# Start GR00T inference service
agent.tool.gr00t_inference(
    action="start",
    checkpoint_path="/data/checkpoints/model",
    port=8000,
    data_config="so100_dualcam"
)

# Natural language control!
agent("Pick up the red block and place it in the box")

Supported Robots

Robot	Type	Integration
SO-100/SO-101	Robot arm	LeRobot + Feetech servos
Fourier GR-1	Humanoid	LeRobot
Unitree G1	Humanoid	LeRobot
Bimanual Panda	Dual arm	LeRobot
Custom	Any	LeRobot compatible

Policy Providers

NVIDIA GR00T

Vision-language-action policy:

agent.tool.gr00t_inference(
    action="start",
    checkpoint_path="/path/to/model",
    data_config="so100_dualcam"
)

Mock Policy (Testing)

robot = Robot(
    tool_name="test_arm",
    robot="so101_follower",
    policy="mock"  # For testing without hardware
)

Camera Configuration

cameras = {
    "front": {
        "type": "opencv",
        "index_or_path": "/dev/video0",
        "fps": 30,
        "width": 640,
        "height": 480
    },
    "wrist": {
        "type": "opencv", 
        "index_or_path": "/dev/video2",
        "fps": 30
    }
}

---

Why Strands Robots?

• Natural Language: "Pick up the red block" instead of code
• GR00T Integration: State-of-the-art vision-language-action
• LeRobot: Universal hardware abstraction
• Async Execution: Non-blocking robot control
• Jetson Ready: Runs on edge devices

---

Control robots with words, not code. 🤖