Shannon AI pentester: test your web apps by real exploitation

Shannon AI pentester, open source: scope, prerequisites, SARIF report, responsible use on staging and its place next to Dynatrace runtime detection.
Shannon pentester IA : tester ses applications web par exploitation réelle

Shannon AI pentester: the KeygraphHQ/shannon repository describes itself as an autonomous pentester for web applications and APIs. It reads the application’s source code, derives attack paths, then tries to exploit them for real against a running instance. Its rule: “No exploit, no report”, an unproven hypothesis never reaches the report. The repository page shows roughly 50,600 stars and 5,800 forks on 30 October 2026 (figures rounded by GitHub), under the AGPL-3.0 licence, and the latest release is v3.5.0 from 27 October 2026.

For an SRE the interest is twofold: replay an exploitation campaign against staging at every release, and know where the tool stops compared with the runtime detection of Dynatrace Application Security or a Syft SBOM.

Honesty note: span.observer has not run Shannon; commands, excerpts and screenshots come from the README, docs/ and benchmark/. Responsibility reminder: the project requires written authorisation from the target’s owner, forbids production and warns that its agents create accounts, modify data and emit outbound traffic. This article is not an offensive how-to.

Shannon AI pentester: what the repository says

  • Vendor: Keygraph, which also sells Pro and Enterprise editions powered by an “enhanced” build of the same engine. No external code contributions accepted for now.
  • Licence: GNU AGPL-3.0, checked in the LICENSE file on 30 October 2026; strong copyleft, with a commercial licence offered to those who cannot accept it.
  • Language and distribution: a TypeScript monorepo (pnpm), published on npm as @keygraph/shannon (3.5.0 on 27 October 2026); the engine runs in a Docker container orchestrated by Temporal.
  • What it tests: five classes, always all enabled: injection, XSS, SSRF, broken authentication and authorisation. An optional “agentic” static analysis (agentic_sast.enabled: "true") feeds the exploitation queue.
  • What it does not test: according to the “Coverage” page, vulnerable dependencies, misconfigurations and policy findings are out of scope because they cannot be proven by exploitation. SCA, secrets and black box are reserved for the paid editions.
  • What made it trend: a comparison published by the project against a Doyensec study (sponsored by Aikido) on Photoview 2.4.0: 23 true vulnerabilities for 115 dollars of tokens with Claude Opus 5, the same critical SQL injection for 6.10 dollars with DeepSeek v4 Flash. The project’s own figures, measured by itself, reported without guarantee.

Tour: what a run produces

The deliverable is a “Security Assessment Report” in PDF and Markdown at the root of the workspace, plus a SARIF 2.1.0 file in exploit mode. The benchmark/ folder publishes the reports from the Photoview comparison; here is the executive summary of the one obtained with Claude Opus 5.

Page 4 of the Security Assessment Report published in the benchmark folder of the Shannon repository: executive summary of the Photoview assessment of 28 August 2026, 24 exploited vulnerabilities, scope of the five classes, then the list of findings INJ-01, INJ-02 and AUTH-01 to AUTH-08
Source : github.com

To read on this page: “Assessment Status: Complete” distinguishes a finished scan from a partial one, and “no XSS or SSRF was confirmed in the tested surface” reflects the project’s rule: not proven does not mean absent.

The SARIF files each finding under one rule per class (shannon/injection, shannon/xss, shannon/auth, shannon/authz, shannon/ssrf, shannon/miscellaneous); critical and high become error, medium becomes warning, everything else note.

Step-by-step installation (according to the documentation)

Step 1 — Prerequisites

Docker, Node.js 18 or later, and a model provider key: Anthropic, OpenAI, xAI and AWS Bedrock are built in, any other provider from the Pi catalogue goes through SHANNON_AI_MODEL=<provider>:<model-id>, with SHANNON_AI_BASE_URL for an LLM gateway. The default model is anthropic:claude-sonnet-4-6; the README states that Keygraph never proxies model traffic. On Windows, only WSL2 is supported.

Anthropic and OpenAI apply real-time safeguards to cyber-security workloads: the documentation asks you to complete their procedure for legitimate testers before the first scan, and --validate-model checks it without launching a pentest.

Step 2 — Configure, then validate authentication

npx @keygraph/shannon@latest setup
npx @keygraph/shannon@latest start -u https://staging.example.com -r ./repo \
  -c ./my-app-config.yaml --validate-auth

The configuration file carries the target description, free-text rules of engagement, areas to avoid or focus on (types url_path, subdomain, domain, method, header, parameter, code_path) and report filters. This is where responsible use is decided: capped rate, brute force forbidden, /logout excluded.

Excerpt of the configuration file documented by Shannon: target description, agentic_sast enabled, rules of engagement with a cap of 5 requests per second, avoid rule on /logout and focus rule on /api, report options min_severity and sarif, then the commands npx setup, start --validate-auth, start -w q1-audit --follow and status --json

Step 3 — Launch and follow

npx @keygraph/shannon@latest start -u https://staging.example.com -r ./repo \
  -c ./my-app-config.yaml -w q1-audit --follow
npx @keygraph/shannon@latest status q1-audit --json
npx @keygraph/shannon@latest logs q1-audit

The repository is mounted read-only inside the container; results go to ~/.shannon/workspaces/. According to the documentation a full run takes between 1 h and 1 h 30 and resumes with the same -w. In CI, the GitHub Action (private repository and Docker runner required) only fails the pipeline on status: exploited findings; a GitLab CI/CD component also exists.

GitHub Actions workflow from the Shannon README with KeygraphHQ/shannon-action@v1, staging url, api-key from a secret, fail-on-severity high and upload-sarif, followed by an excerpt of the benchmark SARIF file: version 2.1.0, rules shannon/injection, shannon/auth, shannon/authz, shannon/miscellaneous and one result at level error

AI pentest, runtime detection and SBOM: three views to cross-check

Shannon answers “can this application be exploited, here, now?”, not “which vulnerable libraries are running in production?”. That second question belongs to Runtime Vulnerability Analytics in Dynatrace SaaS: according to the “Vulnerability evaluation” page (updated 7 October 2026), OneAgent reports a library when a process loads it and compares name and version against vulnerability feeds; it does not check configurations, runtime information or the operating system. See runtime vulnerability exposure.

The two tools overlap on one point. Dynatrace also detects code-level vulnerabilities at runtime (Java 8+, .NET, Go, backends with an ORM; SSRF evaluation to enable separately), and Runtime Application Protection detects, then blocks in “Block” mode, SQL injection, JNDI injection, command injection and SSRF attacks. A useful staging scenario: run Shannon against an application instrumented by OneAgent and check that successful exploits show up on the Dynatrace side as detected attacks; if not, the technology is not covered or the monitoring rule excludes the process. In Dynatrace Managed, third-party vulnerability detection also exists; DQL queries on security.events assume Grail, hence SaaS.

The SBOM completes the picture: Syft inventories everything shipped, Dynatrace reports only what is loaded, Shannon keeps only what it exploits: three shrinking lists, three levels of proof. The multi-agent approach echoes the Cloudflare security audit skill, human review included.

Monitoring the tool itself

Shannon runs on your infrastructure, consumes billed tokens and manipulates a target: it deserves its own monitoring, with what the project actually exposes:

  • a .shannon/workflow.log per run and one file per agent under .shannon/agents/, to ingest like any log file;
  • the Temporal UI on http://localhost:8233 and status --json for a machine-readable snapshot;
  • in CI, the report, the SARIF and a log artefact kept even on failure.

The project documents no OpenTelemetry export and no native metrics: we do not invent any. Three signals are enough: duration and final state of each run (complete or partial), token cost read from the agent logs, and the number of exploited findings per severity in the SARIF. A run that goes from 1 h 30 to 5 h, or whose cost doubles, signals a looping model or a target that has become slow.

Limits and open points

  • Findings still need review: the documentation acknowledges that LLM-generated reports can contain weakly supported or incorrect details; the project’s own comparison counts one false positive.
  • An adversarial repository can contain prompt injections: scan only code you control. And have the AGPL-3.0 licence validated before any integration into an exposed service.
  • Roadmap held by a single vendor, which reserves SCA, secrets and black box for its paid offers; quality depends on the chosen model, and local models should be evaluated first.

Best practices

  • A disposable environment per run, test accounts, synthetic data and written authorisation from the owner.
  • Explicit rules of engagement in the configuration, validated with --validate-auth and --validate-model before the first full scan.
  • Pin the npm version rather than @latest in CI; model key in a pipeline secret.
  • Version the SARIF files and compare them from one release to the next; an “exploited” finding that disappears without a fix deserves an investigation.
  • Cross-check with runtime detection: an exploit proven by Shannon and not seen by Dynatrace is a coverage gap; a CVE seen by Dynatrace and not exploited by Shannon still needs fixing.

Frequently asked questions

Does Shannon replace a human pentest?

No, and the project says so: it targets the long tail of internal applications never tested and frequent releases, leaving chained attacks and business understanding to experts.

Can it be used without sending code to a third party?

The code stays in a local container and results on your disk, but requests to the model carry code context; only a self-hosted model (Ollama, vLLM, LM Studio, via --models-config) keeps everything in-house.

How does it differ from the vulnerabilities Dynatrace shows?

Dynatrace observes production and reports loaded vulnerable libraries, some code-level vulnerabilities and ongoing attacks; Shannon attacks a test environment and reports only what it exploited. One measures exposure, the other proves exploitability.

Conclusion

Shannon makes a simple idea credible: replay a pentest at every release, with a report that contains nothing but evidence. Its limits are clear: five classes, source code required, dependence on the model, findings to review. On staging, framed by rules of engagement and cross-checked with Dynatrace runtime detection and an SBOM, it completes the chain rather than replacing it. Sources: KeygraphHQ/shannon repository (README, docs/safety.md, docs/configuration.md, docs/ai-providers.md), Vulnerability evaluation and Runtime Application Protection.

span.observer is an independent community blog, not affiliated with Dynatrace LLC or Keygraph. Trademarks belong to their owners. Use an exploitation tool only on systems you own or for which you hold written authorisation; tools evolve fast, check the commands in their official documentation.

Total
0
Shares
Leave a Reply

Your email address will not be published. Required fields are marked *

Previous Post
Lightpanda navigateur headless en Zig : agents IA, mémoire et supervision

Lightpanda headless browser in Zig: AI agents, memory and monitoring

Next Post
Elastic Agent collecteur OpenTelemetry 9.3 : ce qui change

Elastic Agent OpenTelemetry Collector 9.3: what changes

Related Posts