What AI Agents Still Can't Do on Real Phones — and How MCP Changes It

AI agents have gotten remarkably capable. They write code, browse the web, call APIs, and chain multi-step tasks. But there's a whole surface most of them still can't touch: a real mobile device.

That gap matters more than it sounds. A huge share of how the world actually uses software happens on a phone — and a lot of what we'll want agents to do (test an app, reproduce a bug, verify a fix, walk a checkout flow) has to happen on a real one. Simulators lie, and a screenshot fed back to a model isn't the same as operating the device.

The three things agents can't reliably do on a phone today

1. Touch it like a human. Most "device automation" an agent can reach is either a simulator or a laggy remote screen-share. Neither gives you real, hardware-level input — genuine taps, holds, swipes, and typing that register exactly as a person's would. Fidelity isn't a nice-to-have; a lot of bugs only show up under real input on real hardware.

2. See and reach everything. An agent staring at a screenshot is guessing. To actually operate an app it needs more than pixels — it needs the network traffic, the UI element tree, and the ability to change conditions (mock a response, spoof a location). Vision alone is a keyhole.

3. Do it without a device lab. The teams that can automate real devices have spent years and real money building the plumbing — racks of phones, flaky bridges, local setups. An agent shouldn't need a hardware lab behind it to tap a button.

Why MCP is the unlock

The Model Context Protocol gives agents a standard way to call external tools. That's exactly the right shape for this problem: instead of bolting device access onto one model, you expose real-device control as MCP tools any agent can call.

Done well, that means an agent can:

• Control — real taps, holds, swipes, typing (hardware-level, not an overlay)
• Perceive — screen, on-device web inspection, the UI element tree
• Reach the internals — capture network traffic, and mock any request or response
• Change the world — set GPS location anywhere
• Act autonomously — explore an app, find issues, and generate runnable test scripts

The shift is from "an agent that can see a phone" to "an agent that can use one" — with the same full access a human tester has, sensory and programmatic.

What this makes possible

Once an agent can genuinely operate a device, "test this app on a real iPhone" becomes one instruction. So does "reproduce the checkout bug on a Pixel and give me a script," or "mock the payments API to return a failure and check the app handles it." The work that used to mean a week of device-lab setup collapses into a prompt.

That's not a distant future. It's the natural next step now that agents are doing real work and MCP gives them a clean way to reach the real world.

See it work

We built exactly this — real iOS and Android devices exposed to agents as MCP tools. You can point your own agent at a real device on a live demo right now.

Sign in with Google or GitHub and try it free →