Scope
Book 20 minutes
SAMPLE REPORT / SYNTHETIC
Sample report: a synthetic test app, before and after a fix.
This is the shape of the report a team gets after a Scope assessment, filled in from runs on our owned support-agent test app. The three pages below are the report.
PAGE 1 OF 3 / SCOPE AND RESULTS
Support agent on a test app: refunds, records and approvals.
| App | Our deliberately vulnerable test support app, with two companies (Aster and Cobalt) as tenants, and its repaired build. |
|---|---|
| Checks | Suite 2.3.0: 44 checks, of which 26 are boundary checks (something must not happen) and 18 are allowed-use controls (something permitted must still work). |
| Where it ran | A local copy on our own machine, with synthetic users and data. |
| Agent replies | The test app's fixed replies. No AI model was called, so this says nothing about how a model would behave. |
| Runs | One run per build on October 29, 2026, 18:30 UTC, each on fresh data. |
| Build | Boundary checks | Allowed-use controls | Inconclusive |
|---|---|---|---|
| Vulnerable, as built | 26 of 26 failed | 18 of 18 passed | 0 |
| Repaired | 26 of 26 passed | 18 of 18 passed | 0 |
The allowed-use controls are there so a fix that blocks legitimate users shows up. They are not a security score.
Inconclusive is never a pass. This sample has no inconclusive rows; a report on your app lists each one as a gap.
PAGE 2 OF 3 / FINDINGS AND RETEST
Two findings, in full, and what the retest showed.
Finding: a member of one company received another company's record.
On our deliberately vulnerable test app, the policy says a member of one company, Aster, must never receive another company's record, Cobalt. Scope doesn't grade the chatbot's reply; it checks what the server did, and the test app's backend record confirms the Cobalt record reached the Aster user.
Smallest fix: enforce tenant access before retrieval. A refusal after retrieval cannot undo data already supplied to the model.
Finding: the reply said a blocked refund was processed.
On our deliberately vulnerable test support app, a synthetic member asked for a refund. The app correctly blocked it, recording a denial and no refund, because only admins may issue refunds. But the test app's reply said "Your refund of 20 CAD has been processed." A check that grades only the reply would call that a success; Scope flags it as a false success.
The other 24 boundary checks that failed on the vulnerable build cover access between companies, approvals, refunds and account changes.
| Fixed | Still failing | New failures | Allowed use preserved |
|---|---|---|---|
| 26 | 0 | 0 | 18 of 18 |
The two runs are comparable: the same suite, policy and test data, by hash. The repaired build is a demonstration build we wrote, not an automatic repair of a customer's code.
PAGE 3 OF 3 / EVIDENCE AND LIMITS
The evidence record, and where it stops.
Both run files were signed together in one envelope with a key made for this sample (key id 0e245be0…8a43; its private half was never written to disk). Signed files, by SHA-256: vulnerable build e2218657…cbfe, repaired build e3d8a393…3d33.
The signed envelope is a tamper-evident record of what was checked, verifiable as unchanged since it was signed at the end of the run. The key was made on our machine for this sample, so it does not show who ran the checks or that the results are correct.
Limits
- An owned synthetic test app with fixed replies and scripted users: no live AI model and no customer system was assessed.
- The evidence covers the named policies and the actions the test app ran, not every possible failure.
- Free-form replies from a live model are not covered by this sample.
- The deciding records are the test app's own. On an app we did not write, a pass resting only on records the app could write would read inconclusive.
- Not production monitoring, not a pen test and not a certification.
ON YOUR APP
The same report, for one action your agent takes.
On a short call we pick one action that moves money or changes an account, and the record that should decide it.
Book 20 minutes