Is AI pentesting safe in production? Rules, limits and a checklist

AI pentesting is safe on a production system only when you set limits before the test starts. The limits cover which actions the agent may take, which data it may touch, how fast it may send requests, and when it must stop. No test removes all risk.

The controls are written scope, tool rules, test-owned data, bounded costs and stop conditions. Use staging for anything risky. The safety page states our position.

What can go wrong in a production pentest?#

Four things can go wrong: destructive actions, data exposure, load, and third-party cost. Each has a matching control.

An agent works fast, and a mistake that a human makes once can repeat many times in parallel. For this reason the controls sit in the harness and in the instructions to the model.

How do ask and deny rules work?#

Each tool in Incursion has a rule: allow, ask or deny. Allow runs on its own. Ask waits for approval. Deny is blocked. The operator sets the rules.

Use this mapping as a start:

• Allow: read-only requests to in-scope hosts, reading client-side JavaScript, listing public pages.
• Ask: any request that changes state, account creation, file uploads, password-guess runs, email-sending flows, calls to paid third-party APIs.
• Deny: hosts outside scope, tools that change infrastructure, any action on real customer accounts, anything on your forbidden list.

Approval slows the test. Move a tool from ask to allow only after you trust its behavior in that scope. See human-in-the-loop pentesting for where approvals fit and how agentic pentesting works for the harness.

How do test-owned accounts and data keep production safe?#

Test-owned data lets the tester prove a destructive effect without harming a real user. Create the accounts and records for the test. Mark them as test data. Prove writes and deletes only against them.

A cross-account proof needs two test accounts. The tester acts as account A against a record that belongs to account B, then reads the result back from B's session. A bare "success" response is not proof; the read-back is.

Reproduction scripts should be read-only where possible. Anyone who reruns them must have authorization and use test-owned identifiers. Never run a script against identifiers that belong to real users.

How do you bound rate and third-party costs?#

Set a request budget for each test class and write it into the scope. OWASP lists unrestricted resource consumption as API4:2023 and names the cost of SMS, email and paid services as a risk. A test can trigger the same cost.

Bound these four classes of test:

1. Uploads: a fixed number of files with a size cap.
2. Password guessing: a small list against a test account, with a lockout plan.
3. Email and SMS flows: a few messages to inboxes you own.
4. Paid third-party APIs: a call budget in currency or count.

We bound these tests and state their cost in the report.

What are good stop conditions?#

A stop condition is a rule that halts the test without debate. Write them in advance. Name who can call a stop and how it is confirmed.

Good stop conditions:

• Error rate or latency passes the threshold you set.
• The agent reaches a record that is not test-owned.
• An alert from your monitoring that you marked as a stop alert.
• A request would leave the agreed scope or hit a denied host.
• Your named contact asks for a stop.

After a stop, the team reviews the saved requests before the test resumes. Saved requests carry timestamps, so the review can follow the order of events.

Production safety checklist#

When should you test staging instead?#

Choose staging when the check could change or delete data, send many messages, or stress a fragile service. Staging carries less risk to customers, but it only helps if its code, configuration and roles match production.

When staging differs, say where. Then agree a bounded production check for the gap, using test-owned data. The CI/CD guide shows how to set up a staging copy and send a scope delta.

NIST SP 800-115, published in 2008 and still listed as final, also advises planning and rules of engagement before technical testing. That advice is old and still applies to an agent. The full service flow is on the how it works page.