AI pentest tools: tool, platform or service?

AI pentest tools are products that use AI to find security flaws in web apps and APIs. They come in four categories, and the label "AI" tells you little about which one you are looking at. Pick by three facts: who runs the test, who checks the results, and what evidence each finding carries. This page gives a category table, the questions to ask a vendor and a short list of cases where a service fits better than a tool.

What is the difference between tools that use AI to test and tools that test AI?#

Search results for "AI pentest tools" mix two different jobs. The first job is using AI to test a normal web app or API. The second is testing an AI application, such as a chatbot, a retrieval system or an agent with tool access.

The rest of this page covers the first column. If your product ships an LLM feature, you probably need both kinds of testing, and they are separate scopes.

Which four categories of AI pentest tools exist?#

Four categories cover the products we see described. The table says who runs the tool, who checks the output, what a finding looks like and how that category usually fails. The failure column is a design observation, not a claim about any one product.

A fifth item sometimes appears on lists: general-purpose coding assistants that someone prompts to "pentest this". They have no scope rules, no tool rules and no evidence standard of their own, so they fall under the open-source agent row until a team adds those controls. AI vulnerability scanner explains the first row in detail, and AI pentest agents vs DAST scanners compares the first two.

For named products in each group, with facts taken from vendor pages and repositories, read the automated pentesting tools guide on pentestcompare.com. That site is published by Excession Labs and is editorially neutral. We do not rank tools here.

What must every AI pentest tool show as evidence?#

A finding is worth acting on when you can inspect the proof. Whatever the category, ask to see these five items for a sample finding:

1. The request the tool sent and the response it received, with a timestamp.
2. The effect: a record that changed, or data that belongs to another account, shown through a second session.
3. A control: the same request with a nonexistent identifier, a wrong token or no token, and what came back.
4. The role or account that sent each request.
5. A label that separates what the tool observed from what it infers.

Two ideas sit behind this list. A write is proven by reading the changed record back through another session, and a bare "success" response is not proof. A control that behaves like the exploit means the finding was a false trigger. AI pentesting false positives shows both with real request shapes. The OWASP Web Security Testing Guide is the open reference for what the underlying tests should check, and you can use it to judge a vendor's test list.

If a tool cannot produce these items for one finding, treat its output as leads for a person to verify. The scorecard in how to evaluate a penetration test report applies the same checks to a finished report.

Tool or service: who checks the results?#

The main buying decision is who reads the output. A tool puts that job on your engineers. A service puts it on the vendor, and the report is the product.

SOC 2 and similar programs do not decide this for you. One CPA firm writes that a pentest "is not required" because the AICPA criteria do not provide the specific controls, and auditors set their own evidence standard. Ask your auditor which evidence they accept before you buy.

At Excession Labs, the lab's AI testing agents do the repetitive testing under tool rules, and Excession Labs checks the evidence for each finding and writes the report. The human in the loop guide explains the split. Other vendors place the review step in other places, so the next section matters.

What five questions should you ask an AI pentest vendor?#

Send these in writing before you buy. A good answer points to a document, a sample or a setting.

1. Who reads the findings before I do? The answers differ. XBOW's blog says of its bug bounty submissions: "All findings were fully automated, though our security team reviewed them pre-submission to comply with HackerOne's policy on automated tools." That is one vendor's account of one setting. Ask your vendor to state its own.
2. What can the agent do without approval? Look for a rule for each tool: runs on its own, waits for approval, or is blocked. Ask for the list for your scope.
3. Can I see one finding with its request, response, read-back and control? Ask for a sample report, not a feature page. Our own sample is on the example assessment page, on an invented API.
4. What does the report say was not tested? A "Limitations and open questions" section means someone thought about gaps. No such section means no one did.
5. What does it cost, and what is bounded? Ask whether tests that send email, upload files or call paid third-party APIs have a limit, and how the report states their cost. We publish no prices on this site, and quotes follow scoping.

These five questions work for a scanner vendor, a platform vendor, a service or an open-source maintainer. For open source, "who reads the findings" means you.

Where do open-source agents fit?#

Open-source agents suit teams that want to learn how agentic testing works, or test their own staging systems with full control. You run them, you pay for the model calls, you read the logs and you carry the risk.

That last point deserves care. An agent that can send state-changing requests can create users, edit records or trigger email. Use only systems you own or have written permission to test, use staging, and use test accounts. The pentestcompare.com guide quotes the authorized-use warnings that several projects publish in their READMEs.

Research agents show both promise and limits. Google's Project Zero wrote that its Big Sleep agent found "an exploitable stack buffer underflow in SQLite", and in the same post called the results "highly experimental" and said "a target-specific fuzzer would be at least as effective". That work targets memory-safety bugs in libraries. On the CyberGym benchmark, the paper's abstract reports that "top-performing combinations only achieve a ~20% success rate" at reproducing real vulnerabilities from a description. That number is for the 2025 models in the paper. Neither source measures business-logic or authorization flaws in a web app, so neither tells you how a tool will do on yours. Where AI pentesting falls short goes through the limits.

Where does our service fit?#

Our service fits teams that want checked findings and a written report for a web app, API or multi-tenant SaaS product, and that can give written authorization and test accounts. It does not fit teams that want a dashboard to run on every commit, or that need social engineering, physical tests or internal network tests.

How it works in outline: we agree the scope with you in writing. The lab's AI testing agents split the engagement into tasks, run them in parallel, and may report a finding only if it links to evidence. Each tool has a rule: it runs on its own, waits for approval, or is blocked. Excession Labs checks the evidence for each finding and writes the report in HTML, PDF and Markdown, with proof-of-concept scripts. We can retest fixed findings, and prices are quoted after scoping.

The pages that go further:

• How agentic pentesting works for the pipeline
• Automated penetration testing for the automation limits
• Safety rules for the tool rules and stop conditions
• Free pentest pilot for a small first scope. Places are limited and reviewed by hand

We do not claim that our agents find all vulnerabilities, and a clean result means nothing was found in that scope.