Skip to content

Give Your Agent a Phone 📱

Published: May 2, 2026

So I did a thing. I gave my AI agent a phone — and now it texts my mom for me.

Meet strands-adb: an @tool-decorated Android control library for Strands Agents and DevDuck. One tool, 90+ actions, full Android automation from an LLM.

The Problem

LLMs are smart, but they're trapped. They can reason, plan, write beautiful code — but when you ask "reply to the last WhatsApp message from mom", they hit a wall. Phones are a different universe. Closed APIs. OAuth hell. Rate limits.

But there's a backdoor the industry already standardized: adb — Android Debug Bridge. Every Android device since 2008 speaks it.

So why not just… let the agent drive adb?

graph LR
    A[LLM] -->|@tool adb| B[strands-adb]
    B -->|shell commands| C[adb CLI]
    C -->|USB/TCP| D[📱 Android Device]
    D -->|screenshots, logs, UI tree| B
    B -->|Converse image blocks| A

That's it. That's the whole idea.

The "Aha" Moment: Vision Blocks

Here's where it gets spicy. When the agent takes a screenshot, it doesn't get a file path — it gets the actual pixels as a Converse API image block:

from strands import Agent
from strands_adb import adb

agent = Agent(tools=[adb])
agent("take a screenshot and tell me what app is open")

The vision model reads the image directly and responds:

"You're on the WhatsApp chat with Mom. Last message: 'Nasılsın oğlum?' sent 2 hours ago."

No OCR hacks. No file passing. Just pixels → reasoning. This is the same pattern as strands_tools.image_reader, so the agent treats the phone screen like any other image input.

Why this matters

Most "LLM + phone" projects dump base64 blobs into text prompts. strands-adb uses proper vision blocks, which means the agent can actually see the UI, reason about pixel positions, and tap coordinates intelligently.

One Tool, Many Domains

I didn't want 50 separate tools cluttering the agent's tool registry. Just one tool — adb — with an action parameter that dispatches to the right domain:

adb(action="screenshot")
adb(action="tap", x=540, y=1200)
adb(action="type", text="hello world")
adb(action="launch", package="com.whatsapp")
adb(action="sensors", sensor="accelerometer")

90+ actions across 13 domains:

mindmap
  root((adb tool))
    Device
      list / select
      battery / thermals
      wake / unlock
    UI
      tap / swipe / type
      smart_tap
      gestures
    Screen
      screenshot
      screen_record
      frames extraction
      ui_dump / ui_find
    Apps
      launch / kill
      install / uninstall
    Camera
      camera_photo
      camera_video
    Sensors
      accelerometer
      gyro / light
      pressure / steps
    Comms
      dial / sms_compose
      media_control
    Logs
      logcat
      log_stream
      notifications_parsed
    A11y
      services
      captions
      font_scale
    Settings
      brightness
      ringer
      any setting_put

Non-Blocking Screen Recording

The killer feature for debugging: record what the agent does while it's doing it.

adb(action="screen_record_start", output_path="/tmp/agent_run.mp4")

# Agent works — recording runs in background
agent("open whatsapp, find mom's chat, reply with 'geldim oğlum'")

result = adb(action="screen_record_stop")
print(result["merged_path"])  # single mp4, auto-stitched

Android's built-in screenrecord has a 180-second limit. strands-adb chains multiple segments and stitches them together with ffmpeg so you get one continuous video. Review it later, tweet it, debug with it. 🎬

The DevDuck One-Liner

If you're a DevDuck user, this is all you need:

export DEVDUCK_TOOLS="strands_adb:adb;strands_tools:shell"
devduck "open whatsapp and read the last message from mom"

DevDuck loads strands-adb, spins up the agent, and you're driving your phone from natural language. No wrappers, no config files, no OAuth.

Real Use Cases I Actually Use

Daily driver examples

Read notifications while coding: 

devduck "what notifications do I have right now?"

Send myself a WhatsApp reminder: 

devduck "text myself on whatsapp: remember to push strands-wfm branch"

Take a photo with the back camera: 

devduck "take a photo with the back camera and describe it"

Check battery before long drives: 

devduck "battery %? thermal state? any apps hogging power?"

Debug a crash: 

devduck "tail logcat for com.myapp, find the last ANR"

The Accessibility Angle

This part I didn't expect to care about as much as I do.

strands-adb exposes full accessibility service control:

  • Enable/disable TalkBack
  • Toggle system captions
  • Adjust font scale
  • Turn on magnification

Which means an agent can set up an elderly parent's phone over a remote adb connection. Large font, high contrast, captions on, voice narration enabled. All via natural language.

devduck "make this phone easier to use for someone with low vision"

And it just… does it. That's wild to me.

Event Bus Integration

When log_stream is active, adb events flow into DevDuck's unified event bus:

sequenceDiagram
    participant Phone as 📱 Phone
    participant ADB as strands-adb
    participant Bus as Event Bus
    participant Agent as DevDuck

    Phone->>ADB: logcat line
    ADB->>Bus: parsed event
    Bus->>Agent: context injection
    Agent->>Agent: reasons with phone state

Now the agent has ambient awareness of what's happening on the phone — new WhatsApp message? Calendar alert? Incoming call? It knows.

What's Next

I'm working on:

  • Multi-device orchestration — one agent, multiple phones
  • iOS sibling — same @tool interface, idevice under the hood (see strands-ios)
  • Recording → training data — turn agent sessions into VLA fine-tuning datasets for NEI

Try It

pip install strands-adb
brew install android-platform-tools  # or apt / pacman / winget

# Enable USB debugging, plug phone in, accept trust dialog
adb devices

Then:

from strands import Agent
from strands_adb import adb

agent = Agent(tools=[adb])
agent("what's on my phone right now?")

Full docs: cagataycali.github.io/strands-adb GitHub: github.com/cagataycali/strands-adb

DevDuck Users

strands-adb is pre-integrated. Just export DEVDUCK_TOOLS="strands_adb:adb" and you're running.

Give your agent a phone. See what happens. 📱🤖