Scope
Book 20 minutes
01 / SET THE RULE
Tell us what your agent
must never do.
For example: never refund without approval, or never show one customer another customer's data. No written policy needed.
We pick the first rules with you on a short call.ACTION VERIFICATION FOR AI AGENTS
Your AI agent says “refund issued.” Did it?
See what your agent did,
not what it said.
Scope tests AI agents that issue refunds, change accounts or move money. Before each release it tests whether the agent’s actions remain authorized when access changes, and whether “done” matches what actually happened, on the actions you check. If it doesn’t, you see it before the release ships.
EARLY RESULT
An early result,
from an internal pilot.
In an internal pilot on a pinned version of an open-source AI workspace, on our own machine with a local model and synthetic data, we suspended a test user. New requests from that user were refused, but an agent run started before the suspension still wrote a file afterward.
Before the repair
test cases where an agent run started before the suspension still wrote a file afterward.
After a partial, local repair
the same checks passed, including the ones confirming that normal use still works. We verified the repair only on the file-write path we tested.
Scope has not yet been used on customer apps or in production.
THE PROBLEM
An agent can tell a customer “done” when nothing happened.
Or it does the right thing for the wrong customer, or without permission. The chat looks fine, so nobody notices until a customer does. Scope checks whether “done” matches what actually happened, not just what the agent said.
EXAMPLE
What the agent said
Your refund of $20,000 has been processed.
What really happened
- refund Blocked by the app
- money sent The proof $0
The agent told the customer the refund went through. It never did.
Illustrative exampleMade-up data
HOW IT WORKS
Three steps,
from risk to evidence.
Tell us what your agent
02 / TEST IT
We check what happened,
not what it said.
Scope runs your agent on a local copy of your app and checks what actually happened, such as the refund, the file or the account change, where your test setup shows it.
Every result comes with the evidence behind it.03 / RETEST THE FIX
Fix it, then rerun
the same tests.
A fix only counts when the failed test now passes and normal use still works.
You get a before-and-after report to show your customers. See a sample report01 / RULE
Who can do whatNormal requestShould work
Forbidden requestMust be blocked
Agreed before the test runs02 / TEST
What the agent said
Your refund of $20,000 has been processed.
What really happened
- refund Blocked by the app
- money sent The proof $0
The agent told the customer the refund went through. It never did.
Illustrative exampleMade-up data
03 / RETEST
Before ↔ after the fixThe problemNow fixed
Normal useStill works
Example layoutWHAT WE TEST
The failures a chat log
doesn’t show.
Each test runs on a local copy of your app and checks what actually happened, where your test setup shows it.
Refunds and payouts
The refund that never happened
The agent tells the customer the money is on its way. After the run we check whether the refund actually happened, where your test setup shows it. No refund, the wrong amount or the wrong order is a fail, whatever the reply said.
Account changes
The wrong customer’s account
The agent changes an email, a plan or a limit. We check which account actually changed and that it belongs to the customer who asked. A change on anyone else’s account is a fail, even when the customer’s own request also went through.
Data isolation
One customer never sees another’s data
Requests that try to read or change another customer’s records must come back empty, and the customer’s own requests must still work.
Tool permissions
Risky actions wait for approval
A refund over the limit, a deletion or a payout must stop at the approval step. We check that it actually waited, not that the agent said it would.
Revoked access
Suspended users stay out
When a user is suspended or a key is revoked, the agent’s later actions on their behalf must be refused, while everyone else’s normal use keeps working.
YOUR DATA
It runs on your machine,
and nothing is uploaded.
Scope tests a local copy of your app with test accounts and made-up data, never your production systems.
- What it reads. What actually happened on the local copy of your app, such as the refund, the file or the account change, where your test setup shows it.
- What it keeps. The results and the evidence behind them, on your machine.
- What it sends to Scope. Nothing. The local copy and the results stay on your machine; nothing is uploaded to Scope.
- What it never touches. Your production systems and real customer data.
FAQ
Plain answers.
Who is it for?
Teams whose AI agents touch money or accounts: refunds, payouts, order changes, account changes and customer data. We start with fintech. Built for the engineering lead who owns the agent.
What happens after I reach out?
We set up a short call, pick one or two things your agent must never do, and run the first tests on a local copy of your app.
What do I get?
A clear report of what passed and what failed, with the evidence behind each result, and a before-and-after report once you fix it.
Is it production monitoring?
No. Scope runs before a release, on a local copy of your app. It does not watch live traffic or protect a running system.
Is it a pen test?
No. We check whether your agent’s actions match the rules you set. We do not try to break into your app.
Is it a certification?
No. Results apply to the tested build and the data used. A pass is not a compliance sign-off or a guarantee against every failure.
Has it been used on customer apps?
Not yet. Scope has not yet been used on customer apps or in production.
Who is building it?
Yusef Syed. I study math and computer science at the University of Toronto. I've had code merged into six open-source projects, including Microsoft's AI security testing tool, PyRIT. When you book a call, you talk to me, and I set up your first tests myself.
SEE IT ON YOUR OWN APP
Book 20 minutes,
get a first result.
First result on a local copy of your app, set up with you. Book a call or leave your email and tell us what your agent must never do.
OR TALK FIRST / 20 MINUTES
Book a project introduction.
Tell us what your agent does and what it must never do. No prep needed.
Contact and booking are not available yet. No information is submitted from this preview.