AI pentest example: a worked assessment of a demo e-commerce API

An AI-assisted pentest is an authorized security assessment where an agent does much of the testing work under rules the system owner approves. This example follows the process in how AI pentesting works on an invented target, from the first brief to the final report.

Excession Labs builds Incursion, a terminal agent for authorized security testing. In the delivery model described here, Incursion does the testing work under the tool rules. Excession Labs agrees the scope with you in writing, checks the evidence for each finding, and writes the report.

The brief: what Pinecart asked for#

Pinecart is a small e-commerce SaaS. Merchants run online shops on it. Shoppers place orders through a public API. The Pinecart team wanted to know if one shopper could read another shopper's data before a large customer's security review.

The system owner approved this written scope.

The brief also lists Pinecart's worries. The coordinator turned them into tasks:

1. Can a signed-in shopper read or change another shopper's orders or profile?
2. Can an anonymous caller reach any signed-in endpoint?
3. Are session tokens and password reset safe against simple abuse?
4. Do the search and filter parameters accept input that changes the query?
5. Does the API leak internal details in errors?

The safety page lists the general rules.

Tool rules: what ran on its own, what waited, what was blocked#

Each tool has one of three rules. A tool with the allow rule runs on its own. A tool with the ask rule waits for a human to approve each use. A tool with the deny rule is blocked. The operator sets the rules before the run.

For Pinecart, the rules were these.

A bounded test is possible under the ask rule. For example, a rate-limit check can send a small, fixed number of requests after a human approves the number. The report states that number.

The run: coordinator, workers and status#

The coordinator is the planner. It reads the brief, splits the work into tasks and hands each task to a worker. Workers run in parallel. Each worker keeps its own notes and findings.

For Pinecart the coordinator made four tasks.

The numbers in this table are invented. A task is done only when each finding links to its evidence. W2 reached that point only after the validation steps below.

This fictional log excerpt shows part of the session.

09:02:11 coordinator brief loaded. scope: api.pinecart.example. 5 objectives, 4 tasks.
09:02:12 coordinator rules: GET/HEAD/OPTIONS allow | POST/PUT/PATCH/DELETE ask | password guessing deny
09:02:40 W1 GET /v1/openapi.json 200 (41 paths)
09:03:05 W1 bundle app.4f1c.js: found /v1/orders/{id}, /v1/users/{id}/addresses
09:03:31 W1 task done: 6 endpoints take an object ID
09:04:02 W2 POST /v1/orders as shopper-a ASK create test order, note "EXCESSION TEST DATA"
09:04:20 operator approved (record is marked as test data)
09:04:21 W2 POST /v1/orders 201 id=48213 owner=shopper-a
09:04:48 W2 GET /v1/orders/48213 as shopper-b 200 owner=shopper-a
09:04:49 W2 candidate C-01: cross-account order read. needs controls
09:05:30 W2 GET /v1/users/me/orders?customer_id=A as shopper-b 200
09:05:31 W2 candidate C-02: filter by other customer. needs controls
09:06:10 W3 POST /v1/password-reset ASK one request, test inbox only
09:06:12 W3 waiting for operator...
09:06:44 W4 GET /v1/search?q=%27 200 (empty list, no error text)
09:07:02 coordinator W2 finding list is not closed: C-01, C-02 have no evidence links yet
09:07:03 coordinator W2 task stays open until controls and read-back are recorded

Order creation waited for a human because it changes state. The coordinator kept W2 open because a single 200 response is only a lead.

How does a candidate become a finding?#

A candidate becomes a finding when we can show a real effect and the controls behave as expected. The method follows the evidence standard used across our work. A bare "success" response is not proof. Proof for a read is a record that only the other account should see, returned to the wrong account. Proof for a write is the changed record, read back through another session.

Each exploit request runs next to controls. Controls are requests that should fail or return different data:

• A nonexistent identifier.
• A wrong record type.
• A request with no token, or a token for another account.
• A comparison of anonymous and signed-in callers.

If the controls do not behave as expected, the candidate is not a finding. The two W2 candidates below show both outcomes. For the method in more depth, see false positives and validation and how agents test authorization.

Candidate C-01: validated end to end#

C-01 claimed that a shopper can read another shopper's order by changing the number in the URL. Pinecart order IDs are sequential integers, so the next ID is easy to guess. This is the pattern OWASP lists as API1:2023 Broken Object Level Authorization.

The worker took these steps. Each step is saved with a timestamp.

1. Shopper A created order 48213 with the note EXCESSION TEST DATA. The response named shopper A as owner.
2. Shopper B requested GET /v1/orders/48213 with B's own token.
3. The response was 200 and contained A's test email and shipping address.
4. The worker ran four controls.

The exploit request and response:

GET /v1/orders/48213 HTTP/1.1
Host: api.pinecart.example
Authorization: Bearer <token-of-shopper-b>

HTTP/1.1 200 OK
Content-Type: application/json

{"id":48213,"owner":"shopper-a@example.com","note":"EXCESSION TEST DATA",
 "ship_to":"1 Test Lane, Exampleville","total_cents":4200}

The response names shopper A, so shopper B received A's data. The four controls then checked the result.

The last step is the read-back from the second account. The worker asked shopper A to read order 48213 again with A's token. The note and total matched what shopper B saw. So B's response came from A's real record, not from a cached or generic reply.

The evidence index for C-01 lists five saved request and response pairs, each with a timestamp. The main pairs carry a SHA-256 hash in the manifest.

The coordinator then closed the C-01 part of W2, because the finding links to all of its evidence.

Candidate C-02: rejected as a false positive#

C-02 looked like a second access control flaw. Shopper B called the order list with another customer's ID in a query parameter:

GET /v1/users/me/orders?customer_id=cus_A91 HTTP/1.1
Authorization: Bearer <token-of-shopper-b>

HTTP/1.1 200 OK

{"orders":[{"id":48220,"owner":"shopper-b@example.com"}, ...]}

The status was 200. A scanner or a weak check would stop here and report a leak. The worker ran controls instead.

The 200 response returned the caller's own data. The server takes the account from the token and discards the extra parameter. No other shopper's record was exposed, so there is no access control effect to report.

We did not put C-02 in the report as a finding. A finding needs evidence of a real effect, and this one has none. Reporting it would tell Pinecart's engineers to fix something that already works. It would also bury PIN-01 among noise.

The report keeps the rejection in the "Controls that held" section as a short entry: the list endpoint ignores a customer_id parameter and scopes results by token. If a later release changes that behavior, a retest will show it. The entry also protects the fix: a developer who patches C-01 should not break this working protection.

What does the human review check?#

Excession Labs checks the evidence for each finding and writes the report. For this assessment, that review covered these points before anything reached Pinecart:

• Each finding has linked request and response evidence with timestamps.
• The effect is shown by a read-back or a second-account view, and a status code alone is not accepted.
• The controls ran and behaved as expected. If they did not, the finding was dropped or relabeled.
• Every state-changing proof used test-owned accounts and records marked as test data.
• The severity matches the written definition for that level.
• Any inferred path is labeled as inferred, and open questions sit under "Limitations and open questions".
• The tool rules were followed, and the report states any bounded test and its cost.

The agent proposes findings and runs tests. The review decides what the buyer reads. Human in the loop pentesting covers the review in more depth.

The allow, ask and deny rules give people control during the run, and the review follows it. The report lists what was not tested, because the agent does not find all vulnerabilities.

Report excerpt: the validated finding#

The final report writes C-01 as a finding section with description, impact, verified results, reproduction and remediation. The full report holds the executive summary, controls that held, limitations and evidence index in other sections.

The severity is High because the exposed data is personal, the attack needs only a free account, and the effect is repeatable at scale. The report gives each level a written definition. It does not add CVSS vectors by default. If your program needs CVSS, say so during scoping.

How a retest marks it fixed#

Pinecart fixes the check and ships. A retest reruns the original request set against the fixed system. It uses the same accounts, the same IDs and the same controls.

A finding is marked fixed only when all these are true:

1. Shopper B's request for order 48213 now fails. The response is 404 or 403, with no order data in the body.
2. Shopper A can still read order 48213. This shows the fix did not break normal access.
3. The controls behave as before: the nonexistent ID gives 404, the anonymous call gives 401.
4. The neighboring routes (update, cancel) fail for B in the same way.
5. The "Controls that held" entry for C-02 still holds.

A retest report keeps the old and new evidence side by side.

For Pinecart, the retest of PIN-01 would show this change:

GET /v1/orders/48213 HTTP/1.1
Authorization: Bearer <token-of-shopper-b>

HTTP/1.1 404 Not Found

{"error":"order_not_found"}

The same request with A's token still returns 200. PIN-01 changes from Open to Fixed, with the date of the retest and the new evidence IDs. Excession Labs can retest fixed findings. We do not publish terms for retests on this page. Ask during scoping.

What the example leaves out#

A real run has many more leads than the two shown, and most close with a control and a short note. Business logic findings need a person to reason about how the product should behave, and none appear here (where AI pentesting falls short). The counts are invented, no result comes from a real system, and we make no claim about past engagements. Real counts, tools and rules come from the written scope.

Run this on your own application#

The path to a real assessment starts with a short call and a written scope. We test only with written authorization from the system owner. A small written scope can start as a free pentest pilot. Places are limited and reviewed by hand.

You approve rules like these before testing starts: see how it works and the safety rules. The sample report from our pentest service shows a full web and API report. Incursion describes the agent itself.