Mobile QA control plane for AI-built apps
Your AI writes mobile tests. RunnerQA proves they work.
RunnerQA turns agent-generated test plans into verified local runs, runtime evidence, approved fixes, and a persistent verification history — powered by Maestro.
The agent understands the code. RunnerQA verifies the reality.
Free during beta · Talk to the Founder
“Cover the premium paywall with a smoke test”
FAILED — “premium_banner” not found
→ UI hierarchy + step timings attached
- tapOn: "Scan" + tapOn: + id: "scan_tab"
$ mqa run
Run #2 — PASSED (28s, id selectors)
Proof, not promises
An AI agent drove this full cycle through the RunnerQA MCP — create, run, fail, analyze, approve, pass. Product proof from a local dogfood run of our own app, not a benchmark.
- 1AI agent creates tests
- 2Local run fails
- 3RunnerQA captures evidence
- 4AI proposes fix
- 5You approve
- 6Rerun passes
Both tests were created, analyzed, and fixed by an AI agent over MCP — each fix was applied only after explicit approval, and both re-runs passed on the first try.
How it works
From project context to release confidence in three steps.
Give your agent project context
Create a project, then connect Claude Code or any compatible agent through hosted MCP — one URL, sign in with RunnerQA OAuth.
$ claude mcp add --transport http \
runnerqa https://www.runnerqa.dev/api/mcp/mcpThe agent proposes coverage
It knows your code, so it plans the flows worth testing — and files them into a persistent test catalog as Maestro-compatible tests.
Run locally, keep the evidence
The CLI executes on your simulator or device. Every run stores status and logs, with timings and UI evidence attached when available — a persistent verification history for your release decisions.
$ mqa run-all
Premium Paywall: PASSED · 29.3s
Failure → evidence → approved fix
When a run fails, RunnerQA attaches what the device actually showed — real accessibility ids and per-step timings. The fix is proposed from evidence and applied only when you approve it.
- tapOn: "Scan"
- waitForAnimationToEnd
- tapOn:
id: "premium_banner"The text selector matched screen copy instead of the tab — and the blanket wait burned seconds on every run.
✗ "premium_banner" not found
UI hierarchy snapshot:
"Scan" id: "scan_tab"
"Scan Pet" id: "scan_pet_button"
Step timings:
3. tapOn "Scan" — 3.9s
5. tapOn premium_banner — 17.1s ✗On failure, the CLI attaches the screen's real accessibility ids and per-step timings to the report.
- tapOn:
id: "scan_tab"
waitToSettleTimeoutMs: 500
- assertVisible:
id: "premium_banner"
- tapOn:
id: "premium_banner"The approved rerun passed: 49.2s → 29.3s in this local dogfood example.
Why RunnerQA
Agents can write a test in any chat window. What they can't give you is a control plane: a persistent catalog, runtime evidence, history, and an approval loop your release can stand on.
Persistent test catalog
Agent-proposed and hand-written tests live in one catalog — named, versioned, and tied to your app's real flows. Not throwaway scripts in a chat window.
Runtime evidence, not vibes
Every run records status and logs, with per-step timings and — on failure — the screen's UI hierarchy attached when the run produces them. Debuggable and reproducible.
Persistent test history
Runs, failures, fixes, and YAML versions accumulate into a verification history. See what regressed, what got fixed, and when.
Human-approved fixes
Failure analysis proposes a fix backed by evidence. Nothing changes until you approve it — every version stays in history.
Vision
Every AI-built mobile app ships with verifiable runtime confidence.
Mission
Turn agent-generated tests and real-device evidence into a trustworthy, human-approved QA loop.
Local-first execution
Built for real mobile apps: the run happens where your app actually lives.
- Tests execute on your own simulators, emulators, and devices through the RunnerQA CLI
- Your app binary and your data stay on your machine — no device farm, no upload
- The agent prepares and reads; it never drives your simulator
- Results report back to one dashboard your whole loop can read
Connect your AI agent in under a minute
One URL, OAuth sign-in, no API token to copy. Your agent gets your project context, the test catalog, run evidence, and the approval loop — while execution stays local through the CLI.
claude mcp add --transport http runnerqa https://www.runnerqa.dev/api/mcp/mcpThen sign in with RunnerQA OAuth and approve the scopes. Powered by Maestro for execution — RunnerQA adds project context, evidence, history, approvals, and release confidence.
Product context
codebase · specs
AI Agent
Claude, Cursor, …
RunnerQA
hosted MCP · OAuth
Test Catalog
Maestro-compatible
Local Runner
simulator · device
Evidence
timings · history
Prefer a token-based setup? The local stdio MCP server and the raw Agent API are documented under Integrations.
Security and approval model
Agents get exactly the access you approve — and no fix lands without you.
OAuth with explicit scopes
Agents connect through RunnerQA OAuth and get only the permissions you approve on the consent screen. Access is limited to the account and scopes you approve.
Per-account isolation
An agent sees only the projects of the account that signed in. Tokens are hashed; secrets are shown once.
Approval before any fix
Failure analysis proposes; you dispose. A fix is applied only after you explicitly approve it, and prior versions stay in history.
Local-first execution
Hosted MCP manages the catalog and evidence. Execution stays on your machine — RunnerQA never remote-controls your device.
More detail on the Security page.
Frequently asked questions
How does my AI agent connect?
Add the hosted MCP URL — https://www.runnerqa.dev/api/mcp/mcp — to Claude Code or any compatible MCP client and sign in with RunnerQA OAuth. No API token to copy. Your agent can then inspect projects, create tests, read run evidence, and propose fixes that you approve. A local stdio MCP server and a raw Agent API remain available as advanced paths.
Is RunnerQA a Maestro competitor?
No — RunnerQA is powered by Maestro for execution. Maestro runs the flows; RunnerQA adds what a QA loop needs around it: project context for your agent, a persistent test catalog, runtime evidence and history, failure classification, and human-approved fixes.
Does RunnerQA run tests in the cloud?
No. Execution is local-first: the RunnerQA CLI runs tests on your own simulators, emulators, or physical devices, and reports the evidence back. Hosted MCP manages the loop — it never drives your device.
Does the AI change my tests on its own?
Never silently. Failure analysis returns a proposed fix backed by the run's captured evidence — logs, timings, and the UI hierarchy when available. It is applied only after you explicitly approve it, and every previous version stays in history.
Does RunnerQA replace QA engineers?
No. It gives engineers and agents a shared, evidence-backed loop so small teams can hold a QA bar without a QA department.
What does it cost?
RunnerQA is free during beta. The hosted MCP endpoint is available now for every account.