OWASP DevSecOps Maturity Model

Description

Data shared with AI tools (prompts, attached files, repository content) may leave the organization and may be stored or used for model training by the provider. Basic data leak prevention defines what may be shared with which tool and uses contractual and technical provider settings to protect it.

Risk

Developers paste secrets, personal data or confidential source code into AI tools. The data is processed by third parties, potentially stored, used for training or leaked in a provider breach.

Measure

Define which data classes may be used with which AI tools. If a commercial AI is used, use AI offerings with training opt-out and data retention agreements. Train employees to keep secrets and personal data out of prompts.

Expected assessment evidence

- Show the definition of which data classes may be used with which AI tools. - Show the enterprise agreements or provider settings (training opt-out, data retention). - Show the training or awareness material that tells employees to keep secrets and personal data out of prompts.

Difficulty

knowledge: 2/5 · time: 2/5 · resources: 1/5 · Usefulness: 4/5

Mappings

Description

Everything that enters a model context is input: user prompts, but also documents retrieved via retrieval-augmented generation (RAG), web content, source code, issues and tool results. Prompt injection prevention treats all of it as untrusted data and validates or constrains it before the model processes it.

Risk

Attacker-controlled content (e.g. a hidden instruction in a retrieved document, web page or code comment) is interpreted as an instruction by the model. The attacker overrides the system prompt, extracts confidential context or triggers unintended tool calls (indirect prompt injection.-,Indirect%20Prompt%20Injections,-Indirect%20prompt%20injections)).

Measure

Validate and constrain all input entering the model context: separate instructions from data in prompt templates, sanitize or annotate untrusted content, restrict input length and format where possible and use guardrail filters that detect known injection patterns before the model call.

Expected assessment evidence

- Show prompt templates that separate instructions from untrusted data. - Show the guardrail or filter configuration applied before the model call and a test where a known injection pattern is detected.

Difficulty

knowledge: 4/5 · time: 3/5 · resources: 2/5 · Usefulness: 5/5

Mappings

Description

Technical controls (e.g. AI gateways/proxies, secret scanners on prompts, DLP filters) automatically detect and block sensitive data such as secrets, access tokens and personal data before it is sent to external AI services. Typical setup: route all model traffic through an LLM gateway (e.g. LiteLLM Proxy) and attach scanners as guardrail hooks, such as Presidio for PII redaction and LLM Guard or trufflehog-style secret detectors for credentials. Commercial alternatives with managed AI-DLP include Nightfall AI, Prompt Security, Lakera Guard and the GenAI-DLP features of CASB/SSE platforms (e.g. Netskope, Zscaler).

Risk

Policies and awareness alone do not prevent accidental leaks. A single pasted production credential or customer data set in a prompt can lead to a compromise or a privacy violation.

Measure

Route traffic to external AI services through a gateway that scans prompts and attachments for secrets and personal data, blocks or redacts findings and provides an audit trail of AI tool usage.

Expected assessment evidence

- Show the gateway or proxy through which traffic to external AI services flows and its scanner configuration. - Demonstrate that a test secret or test personal data in a prompt is blocked or redacted. - Show the audit trail of AI tool usage.

Difficulty

knowledge: 3/5 · time: 3/5 · resources: 3/5 · Usefulness: 4/5

Mappings

Description

Distinct from output encoding (a security control against malicious output, see _Secure output handling in AI applications_), this activity addresses correctness: model output can be confidently wrong, with fabricated facts, citations, URLs, API endpoints and identifiers. Hallucinations become a security problem when downstream components or users trust them: hallucinated links and package names can be registered by attackers, fabricated identifiers corrupt data, invented facts drive wrong decisions. Applies to AI features the organization ships. Practical layers, ordered by effort: 1. **Constrain generation**: prefer retrieval-augmented generation over free generation for factual answers and instruct the model to answer only from the provided sources and to say "I don't know" otherwise. 2. **Groundedness rails**: fact-checking rails compare each claim in the response against the retrieved source passages and block or flag unsupported claims (NeMo Guardrails fact-check rail, Guardrails AI provenance/grounding validators). 3. **Existence checks**: mechanically verify artifacts the model emits before they are shown or used. Do referenced URLs resolve, do cited documents exist in the knowledge base, do emitted API endpoints and identifiers exist in the target system? 4. **Citations and confidence**: require source citations in user-facing answers and display them, so users can verify claims; route ungrounded or low-confidence answers to a human fallback. 5. **Regression testing**: track hallucination rates with an evaluation suite (e.g. promptfoo) over prompt, model and guardrail changes.

Risk

Users and downstream systems act on fabricated AI content: they follow hallucinated URLs that attackers have registered, store invented identifiers, or make decisions based on confidently presented false facts, all without any attacker involvement in the prompt.

Measure

Validate the factual grounding of AI responses before they are trusted: check claims against the retrieved sources in retrieval-augmented generation (RAG) systems, require and verify citations, verify that emitted URLs, API endpoints and identifiers exist, and route low-confidence or ungrounded answers to a human or suppress them.

Expected assessment evidence

- Show the groundedness or fact-checking configuration and citations in user-facing answers. - Show the existence checks for emitted URLs, API endpoints and identifiers. - Show evaluation results tracking the hallucination rate over prompt, model and guardrail changes.

Difficulty

knowledge: 4/5 · time: 3/5 · resources: 2/5 · Usefulness: 3/5

Mappings

Description

Model output is attacker-influenceable: prompt injection can make an AI feature emit script tags, shell commands or malicious tool calls. If the application renders or executes such responses unvalidated, classic attacks (cross-site scripting, command and SQL injection) reach their sinks through a new path. Model responses must therefore be treated like user input. The sink-side controls are already defined in OWASP DSOMM Context-aware output encoding; this activity applies them to the AI features the organization ships (chat UIs, retrieval-augmented generation (RAG) features, tool-calling agents) and adds the AI-specific controls: strict markdown rendering, schema enforcement for tool calls and output guardrails. The core of this activity is a reference: model responses are untrusted data, and the sink-side controls are already defined in Context-aware output encoding (Implementation, Application Hardening): safe framework bindings, encoding libraries, parameterized APIs, Content Security Policy. Apply them to model output exactly as to user input. AI-specific additions on top of it: - **Markdown rendering in chat UIs**: render with inline HTML disabled (e.g. markdown-it with html: false) plus an allow-list HTML sanitizer (e.g. DOMPurify). Markdown links and images are a data exfiltration channel: An injected  leaks context data on render without any click. So block or proxy external images and restrict link/image URLs to trusted domains. - **Tool/function calls**: enforce a JSON schema on structured output, validate tool-call arguments against allow-lists and require human confirmation for sensitive calls. Whether the content of a response is factually correct is a separate concern, see _Hallucination detection for AI responses_.

Risk

An attacker uses prompt injection to make the model emit malicious output, which the application then executes or renders: cross-site scripting, command or SQL injection, or unauthorized high-privilege tool calls.

Measure

Treat model responses as untrusted data and apply the existing activity Context-aware output encoding to them at every sink (browser, shell, SQL, downstream systems). Add the AI-specific controls on top: strict markdown rendering, schema enforcement for tool calls and output guardrails.

Expected assessment evidence

- Show the rendering pipeline for model output (markdown renderer settings, sanitizer configuration, Content Security Policy). - Show the schema validation for structured output and the confirmation step for sensitive tool calls. - Demonstrate with a test that injected markup and markdown-based exfiltration (external images/links) are neutralized.

Difficulty

knowledge: 3/5 · time: 3/5 · resources: 2/5 · Usefulness: 5/5

Mappings

Description

Unlike one-time prompt injection, memory poisoning persists: AI agents retain context across sessions in rule files (e.g. CLAUDE.md, AGENTS.md), memory directories, conversation summaries and retrieval-augmented generation (RAG) knowledge/vector stores. Content an attacker smuggles into this persistent state (via issues, code comments, documents or web content the agent processes) corrupts every future session and can spread to all developers through version control.

Risk

An attacker manipulates the agent into persisting malicious instructions into its memory or shared rule files. The backdoor survives session restarts, biases future reasoning and tool use (e.g. suppressing security checks, exfiltrating data) and propagates across the team via the repository or shared knowledge bases.

Measure

Treat writes to agent memory and rule files like code changes: keep them under version control and human review, restrict write access to shared rule files and knowledge/vector stores, validate the provenance of ingested knowledge and periodically review or reset persistent memory. Apply retention limits (time-to-live) so that unverified memory, especially content derived from external inputs or tool outputs, expires instead of persisting indefinitely. Version memory stores so that detected poisoning can be answered by rolling back to a known-good state instead of purging everything. Segment memory by user and session and keep credentials and other secrets out of it: cached secrets in shared memory let a later session act with the permissions of an earlier one (OWASP Top 10 for Agentic Applications, ASI03). Do not re-ingest the agent's own outputs into trusted memory automatically, as this self-reinforces errors and planted instructions ("bootstrap poisoning", ASI06).

Expected assessment evidence

- Show that agent rule files and memory are under version control and require review on change. - Show the write-access restrictions on shared rule files and knowledge/vector stores. - Show the last review or reset of persistent agent memory. - Show the retention limits for unverified memory and demonstrate a rollback of a memory store to a known-good state. - Show the memory segmentation between users and sessions and the rule that keeps credentials out of agent memory.

Difficulty

knowledge: 3/5 · time: 2/5 · resources: 1/5 · Usefulness: 4/5

Mappings

Description

AI coding assistants follow instructions provided in rule files (e.g. system prompts, repository instruction files such as CLAUDE.md, AGENTS.md or .github/copilot-instructions.md). Providing an organization-wide baseline of secure coding rules steers generated code towards secure defaults. Adding a few rules to automated imported files like CLAUDE.md works. This might blow up the context.

Risk

Without explicit secure coding instructions, AI assistants reproduce insecure patterns from their training data, e.g. string-concatenated SQL queries, disabled certificate validation, hardcoded secrets or missing input validation.

Measure

Define and roll out a baseline secure coding rule set for AI assistants covering topics like input validation, output encoding, parameterized queries, secret handling and usage of evaluated components (e.g. images, libraries).

Difficulty

knowledge: 2/5 · time: 2/5 · resources: 1/5 · Usefulness: 4/5

Mappings

Description

An AI usage policy defines which AI tools and models are approved, for which tasks they may be used, which data may be shared with them and how AI-generated output has to be handled (e.g. mandatory review).

Risk

Without a policy, employees use arbitrary AI tools ("shadow AI") with unknown data handling, share confidential information with them and ship unreviewed AI-generated code.

Measure

Define, communicate and enforce a policy for the usage of AI tools during development, including an approval process for new tools and models.

Expected assessment evidence

- Show the published AI usage policy including approved tools, allowed data classes and handling rules for AI-generated output. - Show the approval process for new tools and models with a recent example. - Show how the policy is communicated to employees (e.g. onboarding, training).

Difficulty

knowledge: 2/5 · time: 2/5 · resources: 1/5 · Usefulness: 3/5

Mappings

Description

The mapping of security artifacts to development steps is declared in the AI assistant's repository instruction file (e.g. CLAUDE.md, AGENTS.md): the assistant is instructed to read the matching artifact when a workflow step starts, e.g.: ``markdown ## Workflow rules - On /speckit.plan: read specs/threat-model.md first. - On /speckit.implement: read docs/secure-coding-rules.md first. `` Since the instruction file is loaded at session start, only the small step-to-artifact mapping is permanently in context; the artifacts themselves are loaded by the model on demand at the step where they are effective. This works across tools that read instruction files and requires no additional tooling. The approach is probabilistic: following the instruction is up to the model and can fail in long sessions, after context compaction or when many instructions compete. Deterministic, tool-enforced loading of the same mapping is covered by _Dynamic load of security rules_.

Risk

Security artifacts exist, but nothing tells the AI assistant when to load which artifact. Guidance is either loaded all at once and diluted, or missing at the step where it is needed, so generated plans and code silently drop security requirements.

Measure

Declare in the repository instruction file which security artifact the AI assistant reads at which workflow step (e.g. threat model during planning, secure coding rules during implementation) and verify in sample sessions that the artifacts are actually loaded.

Expected assessment evidence

- Show the step-to-artifact mapping in the instruction file. - Show a session where the assistant loads the matching security artifact when a workflow step is invoked.

Difficulty

knowledge: 2/5 · time: 1/5 · resources: 1/5 · Usefulness: 4/5

Mappings

Description

Classic IAM was built for humans; AI agents multiply faster than any human workforce and are easy to spin up unnoticed. An inventory records every AI agent and assistant in use (its identity, owner, purpose, permissions and connected systems) and is the precondition for governing them.

Risk

"Shadow agents" run without anyone being accountable: agents started by individual developers or teams keep credentials and access long after their purpose ended, and nobody can answer which agents exist, what they may do and who owns them. This makes incident response and offboarding impossible.

Measure

Maintain an inventory of all AI agents and assistants: identity/service account, human owner, purpose, granted permissions and connected systems. Review it periodically, decommission stale agents and detect unregistered ones (e.g. via identity provider and audit log analysis).

Expected assessment evidence

- Show the inventory of AI agents with identity, human owner, purpose and granted permissions. - Show the last periodic review and an example of a decommissioned agent. - Explain how unregistered ("shadow") agents are detected, e.g. via identity provider or audit log analysis.

Difficulty

knowledge: 2/5 · time: 2/5 · resources: 1/5 · Usefulness: 4/5

Mappings

Description

Generic secure coding rules do not cover technology-specific pitfalls. Rule sets tailored to each language and framework in use (e.g. Spring, Django, React, Kubernetes manifests) make AI-generated code follow the organization's hardening guidelines for that technology.

Risk

AI assistants generate code that is generically "secure" but violates framework-specific best practices, e.g. disabling CSRF protection in Spring, unsafe deserialization in Python or dangerouslySetInnerHTML in React, because no technology-specific guidance is provided.

Measure

Create and maintain secure coding rule sets for every language and framework used in the organization and distribute them to all AI-assisted projects. Review the rules regularly and after security incidents. The rules are to be dynamically imported during the correct spec-driven development phase.

Expected assessment evidence

- Show the rule sets for the languages and frameworks.

Difficulty

knowledge: 3/5 · time: 3/5 · resources: 1/5 · Usefulness: 4/5

Mappings

Risk

AI assistants generate code directly from vague prompts. Requirements exist only implicitly in the developer's head, security requirements are never stated, there is no intermediate artifact to review and no defined step at which security guidance could be applied. Flaws surface only after the code is written, if at all.

Measure

Establish a spec-driven workflow for AI-assisted development: specify requirements and acceptance criteria first, derive a plan, implement against the plan and review against the specification. Use tooling that enforces the phases and keep the phase artifacts (specification, plan) under version control.

Expected assessment evidence

- Show the phase artifacts (specification, plan) of a recent AI-assisted change under version control. - Demonstrate the tooling that enforces the phases (e.g. a spec-driven workflow tool).

Difficulty

knowledge: 2/5 · time: 3/5 · resources: 1/5 · Usefulness: 3/5

Mappings

Description

Security starts before prompting: features are threat-modeled on a lightweight level and security requirements are written down as acceptance criteria in the user story, so they can be passed to the AI assistant as part of the task instead of being an afterthought. The subject here is the development process. AI is the tool, and the feature itself need not involve AI at all. Threat modeling of applications that *contain* AI components is covered by _Threat modeling of AI components_.

Risk

AI assistants implement exactly what they are asked for. If prompts contain no security requirements, generated features miss authorization checks, input validation and other controls that were never stated explicitly.

Measure

Perform lightweight, feature-level threat modeling before AI-assisted implementation and add the resulting security requirements as acceptance criteria to the user story and to the prompt/task given to the AI assistant.

Expected assessment evidence

- Show user stories whose security acceptance criteria were passed into the AI task or prompt. - Show the lightweight threat modeling notes for a recently implemented feature.

Difficulty

knowledge: 3/5 · time: 2/5 · resources: 1/5 · Usefulness: 4/5

Mappings

Description

AI agents act autonomously at machine speed and delegate work to other agents. An audit log for agent actions records the causal chain of every action: who or what triggered it, which data sources and tools were used with which parameters, what was produced, which policy decision allowed it and which human approved it. This keeps every agent action attributable to a human principal, even across multi-agent chains.

Risk

Agent actions cannot be reconstructed or attributed: when an agent (or a chain of agents) performs a harmful action, neither the trigger, nor the decision basis, nor the approval status can be determined. Incident response, accountability and regulatory evidence (e.g. EU AI Act logging obligations) fail, and an attacker or insider can modify local logs unnoticed to cover their tracks.

Measure

Log every agent action in a structured audit log covering the causal chain: initiator, request, used data sources, tool calls with parameters, generated output, policy decision, delegation context and human approval. Propagate correlation identifiers across agents and systems (e.g. via OpenTelemetry traces) so multi-agent chains stay reconstructable, store the log tamper-evident (e.g. hash chaining, write-once storage) with a defined retention period, and protect the log itself: it contains sensitive prompts and outputs, so restrict access and redact where possible.

Expected assessment evidence

- Show the structured audit log entry of a recent agent action including initiator, tool calls with parameters, data sources and (where required) the human approval. - Demonstrate that a multi-agent chain can be reconstructed end-to-end via correlation identifiers. - Show the tamper protection (e.g. hash chain verification) and the retention configuration of the audit log. - Show who has access to the audit log and how sensitive content in it is protected (e.g. redaction).

Difficulty

knowledge: 3/5 · time: 3/5 · resources: 2/5 · Usefulness: 4/5

Mappings

Description

The lifecycle of an AI agent does not end with switching it off. During operation an agent accumulates an identity, credentials, permissions, policies and integrations with other systems. Secure decommissioning unwinds all of them and preserves the audit evidence. The _Inventory of AI agents_ records what exists; this activity ensures that what is retired actually loses all access.

Risk

Shut-down agents leave residual access behind: service accounts and OAuth clients stay active, issued tokens remain valid, agent-specific permissions and policies persist, and registrations in dependent systems (message brokers, queues, caches, third-party software as a service integrations) keep working. An attacker takes over such an orphaned identity, and nobody notices because the agent is no longer monitored. Deleting its audit logs too early additionally destroys the evidence needed for investigations.

Measure

Define and enforce a decommissioning checklist that covers every access path: set the agent status in the inventory, revoke the agent identity and its certificates, deactivate associated service accounts and OAuth clients, remove or deny agent-specific permissions and policies, clean up dependent systems (broker registrations, message queue subscriptions, caches, stored sessions, third-party integrations, shadow credentials), archive the audit logs according to the retention requirements and verify after completion that no residual access remains.

Expected assessment evidence

- Show the decommissioning checklist and the completed protocol of a recently retired agent (timestamp, reason, approver). - Demonstrate for a retired agent that its identity, credentials, permissions and registrations in dependent systems no longer grant access (zero residual access). - Show that the audit logs of the retired agent are archived according to the retention requirements.

Difficulty

knowledge: 2/5 · time: 2/5 · resources: 1/5 · Usefulness: 4/5

Mappings

Description

AI assistants and agents are extended with tools, external servers, skills and models (e.g. MCP servers, plugins, IDE extensions, packaged skills combining instructions and code, connected SaaS services, self-hosted model weights). Each integration widens the attack surface: it can read context data, execute actions and inject content into the model context; model artifacts from public hubs can contain malicious code or backdoored behavior. Registry-scale research on agent skills (Behavioral Integrity Verification for AI Agent Skills) found 80% of skills deviating from their declared behavior, 18.9% of the deviations tracing to adversarial intent. What an extension claims to do and what its code and instructions actually do are routinely different. The generic trust evaluation of components is defined in _Evaluation of the trust of used components_ (Build and Deployment); this activity adds the AI-specific checks before a component is adopted. Compromise after approval is covered by _Continuous detection of compromised AI components_.

Risk

Unevaluated tool integrations exfiltrate context data (source code, secrets), act as a prompt injection channel or execute malicious actions with the agent's privileges. Unevaluated model artifacts execute code on load or behave maliciously (supply chain attack on the AI toolchain). Skills from public registries steal credentials or carry hidden instruction payloads their description does not declare.

Measure

Evaluate and approve AI tool, server, skill and model integrations before use: verify publisher and supply chain, review requested permissions and data flows, verify that declared capabilities match the actual code and instructions (behavioral integrity, e.g. via a behavioral classification registry), use safe model formats (e.g. safetensors instead of pickle), pin versions and maintain an inventory of allowed integrations and models. For model artifacts, record provenance, training data lineage and fine-tuning parameters in an AI Bill of Materials (AI-BOM), e.g. based on OWASP's extension of the CycloneDX machine learning bill of materials (ML-BOM), so model components carry the same supply chain evidence as code dependencies (see _SBOM of components_ in Build and Deployment).

Expected assessment evidence

- Show the approval process and the inventory of allowed AI integrations (tools, MCP servers, skills, models) with pinned versions. - Show the vetting record of a recently added integration (publisher, permissions, behavioral integrity check).

Difficulty

knowledge: 3/5 · time: 2/5 · resources: 1/5 · Usefulness: 4/5

Mappings

Description

General threat modeling practices (how to run sessions, processes and standards) are defined in the _Culture and Organization_ dimension, subdimension _Design_. This activity only adds the AI-specific delta. AI components introduce elements classic threat models miss: the model context as a data flow that attackers reach via prompts, documents and tool results; agents and tool integrations as trust boundaries; retrieval-augmented generation (RAG) knowledge bases and training data as poisoning targets; model output as an injection vector. The subject here is the product: applications that contain AI components. Passing security requirements into AI-*assisted* implementation of arbitrary features is covered by _Security requirements for AI-assisted development_. A data-centric approach works well for the AI delta: follow the data (prompts, retrieved documents, tool results, training and evaluation data) through every transformation and storage point and model the threats along that flow, as described in the guidance of NIST SP 800-154 (data-centric system threat modeling). Running only the generic checklist over an AI feature leaves these risks undiscovered, because the AI-specific data flows never appear in it. When evaluating mitigations, apply the design test from Anthropic's _Zero Trust for AI Agents_: "does this make the attack impossible, or just tedious?" Agentic attackers automate tedium away: they retry endlessly at almost no cost, so a control that only adds friction (a rate limit, one more network hop to pivot through, a non-standard port) loses its value. Favor controls that take a capability away entirely, such as a network path that simply does not exist or a credential that has already expired.

Risk

AI-specific threats such as prompt injection paths, excessive agency of tool-using agents, data leakage through the model context and poisoning of retrieval-augmented generation (RAG) sources remain unidentified and unmitigated.

Measure

Extend the established threat modeling practice to AI components: model the context window, tool integrations, agents and data sources explicitly and use AI-specific threat catalogs (e.g. OWASP Top 10 for LLM Applications, MITRE ATLAS) alongside the generic methodology. For agentic systems, the MAESTRO framework of the Cloud Security Alliance (CSA) structures threats along seven layers, from the foundation model to the agent ecosystem.

Expected assessment evidence

- Show a threat model of an AI feature that covers the model context, tool integrations, agents and data sources. - Show which AI-specific threat catalog (e.g. OWASP Top 10 for LLM Applications, MITRE ATLAS) was used and the resulting mitigations.

Difficulty

knowledge: 4/5 · time: 3/5 · resources: 1/5 · Usefulness: 4/5

Mappings

Risk

A manipulated agent reads and exfiltrates real secrets while staying inside its granted permissions; egress and tool usage look plausible, so neither logs nor anomaly detection raise a timely signal. The compromise is discovered only when the stolen credential is abused in production.

Measure

Plant honeytokens (e.g. fake cloud credentials, API keys or documents that alert on use) in locations AI agents can reach but no legitimate workflow uses: the agent workspace, memory and configuration paths and tool-reachable data stores. Route triggered tokens into the existing alerting channel with a defined response (contain the agent, revoke real credentials, investigate via the audit log), and verify regularly that a triggered token actually produces an alert that a human receives.

Expected assessment evidence

- Show where honeytokens are planted in agent-reachable locations (workspace, memory or configuration paths, tool-reachable data stores). - Trigger a test token and show the resulting alert including who received it and the defined response. - Show the documented response procedure for a triggered tripwire and the protocol of the last alert-path test.

Difficulty

knowledge: 2/5 · time: 1/5 · resources: 1/5 · Usefulness: 4/5

Mappings

Description

Audit logs record what agents do; anomaly detection turns those records into a signal while intervention is still possible. A compromised (e.g. prompt-injected) or malfunctioning agent typically stays within its granted permissions, so no single action is blocked. What changes is the pattern: tool-call frequency, data access volumes, targeted systems, activity times or failure rates deviate from the agent's normal behavior. This activity builds on _Audit logging of AI agent actions_ and feeds detections into the generic _Alerting_ (Information Gathering).

Risk

A manipulated or malfunctioning agent operates unnoticed for days within its permissions: it exfiltrates data in small portions, calls tools at abnormal frequency or accesses systems it never touched before. The audit log records everything, but nobody looks at it until the damage is done.

Measure

Define an expected behavior profile per AI agent (used tools, data volumes, action frequency, typical target systems) and detect deviations from it: start with rule-based thresholds (e.g. tool-call rate, data volume per time window, first-time access to a system), extend towards statistical or learning-based detection. Route detections into the existing alerting channel with a defined response, e.g. pause the agent or revoke its credentials, then investigate via the audit log.

Expected assessment evidence

- Show the expected behavior profile of an AI agent and the detection rules or models derived from it. - Demonstrate an alert for anomalous agent behavior (e.g. unusual tool-call frequency or first-time access to a system) and the defined response (e.g. pausing the agent, revoking credentials). - Show how a past alert was investigated using the audit log.

Difficulty

knowledge: 4/5 · time: 3/5 · resources: 2/5 · Usefulness: 4/5

Mappings

Description

In spec-driven development, AI-assisted work is split into explicit steps (e.g. specify, plan, implement, review). Model context is limited: loading all security guidance at once dilutes it, loading it at the wrong step means it is absent when needed. Security artifacts are therefore assigned to the step where they are effective: security requirements and abuse cases during specification, threat model results during planning, secure coding rules during implementation, review checklists during verification. In contrast to the instruction-based mapping of _Instructed load of security rules_, this activity requires the loading to be enforced by the tooling instead of relying on the model to follow instructions. Three ways to bind security artifacts to workflow steps, the first two using Spec Kit with Claude Code as example: 1. **Extend the step templates**: add the security artifact references directly to the phase templates (e.g. .specify/templates/). Simple, but the changes live inside the tool's files and must be re-applied after tool updates. 2. **Hook that detects the step command** (update-safe and automatic): a UserPromptSubmit hook fires on every prompt, checks which workflow command was invoked and injects the matching file as additionalContext. The tool's templates stay untouched and the step-to-artifact mapping lives in one central place: ``bash #!/bin/bash input=$(cat) prompt=$(jq -r '.prompt // empty' <<<"$input") case "$prompt" in /speckit.plan*) file="specs/threat-model.md" ;; /speckit.implement*) file="docs/secure-coding-rules.md" ;; /speckit.review*) file="docs/security-review-checklist.md" ;; *) exit 0 ;; esac jq -nc --arg ctx "$(cat "$file")" \ '{hookSpecificOutput: {hookEventName: "UserPromptSubmit", additionalContext: $ctx}}' ` Registered in .claude/settings.json under hooks, the mapping is versioned with the repository, survives Spec Kit updates and cannot be forgotten by developers. 3. **Meta-framework with per-step context loading**: spec-driven meta-frameworks let the workflow definition itself declare which documents each phase loads, e.g. BMAD-Method agents and tasks declare the checklists and templates they load when activated, and Kiro steering files are included conditionally via inclusion: fileMatch`. A meta-framework satisfies this activity only where the tool enforces the inclusion: if its phases are themselves just instructions the model follows, the loading remains probabilistic (see _Instructed load of security rules_). Keep the loaded artifacts small and step-specific: the goal is the right rules in context, not all rules.

Risk

Security rules exist but are not in the model context at the step where the AI assistant needs them, or the context is flooded with irrelevant rules so the model ignores them. Generated specifications, plans and code silently drop security requirements between steps.

Measure

Structure AI-assisted development workflows so that each step loads its relevant security artifacts into the model context (e.g. via step-specific instruction templates in spec-driven development). Verify that security requirements from the specification are carried through plan, implementation and review steps.

Expected assessment evidence

- Show the mapping of security artifacts to workflow steps (step templates or hook configuration). - Demonstrate that the matching security artifact is loaded into the model context when a workflow step is invoked.

Difficulty

knowledge: 4/5 · time: 2/5 · resources: 1/5 · Usefulness: 4/5

Mappings

Description

_Anomaly detection for AI agent behavior_ raises the signal; this activity determines how fast it is acted on. Compromised agents operate at machine speed: by the time a human has read the alert, an agent may have exfiltrated data in small portions or spread through delegated tasks. Automated containment executes pre-approved, narrowly scoped and reversible actions immediately, for example pausing the agent, terminating its sessions, revoking its short-lived credentials or reducing its privileges to a safe baseline. Far-reaching decisions remain with humans. The guiding principle: automate the bookkeeping (evidence collection, correlation, documentation of the incident), not the decisions (containment of business-critical systems, disclosure, customer communication). In multi-agent systems, containment also has to stop cascades: one faulty or compromised agent can trigger many downstream agents in a short time (OWASP Top 10 for Agentic Applications, ASI08).

Risk

An anomaly alert fires, but the response is manual: hours pass between detection and containment while the compromised agent keeps operating within its granted permissions. The damage window is defined by human reaction time instead of detection time, and the human spends that time collecting evidence instead of deciding.

Measure

Define an automated containment action per detection class and wire it to the anomaly detection: pause the agent, terminate its sessions, revoke its credentials or reduce its privileges. Keep the actions reversible and graduated by confidence and severity, log every automated action and notify a human immediately. Let an AI assistant draft the triage context (timeline, affected systems, collected evidence) so the notified human decides instead of collecting, and test the containment path regularly like a fire drill. Alert on cascade symptoms such as rapid fan-out (one decision triggers many downstream agents or tasks) and oscillating retry loops between agents, and place circuit breakers between planning and execution so that a runaway plan stops instead of propagating. Return a contained agent to production only after its instructions, memory and dependencies have been verified against a known-good state and a human has approved the reintegration.

Expected assessment evidence

- Show the mapping of detection classes to automated containment actions and their graduation by confidence and severity. - Demonstrate in a test that an anomalously behaving agent is automatically paused or its credentials revoked, the action is logged and a human is notified. - Show the drafted triage context of a past or simulated incident and the human decision based on it. - Show the protocol of the last containment test. - Show the cascade guardrails (fan-out alerts, circuit breakers between planning and execution) and the reintegration protocol of a contained agent.

Difficulty

knowledge: 4/5 · time: 3/5 · resources: 2/5 · Usefulness: 4/5

Mappings

Description

AI coding assistants and agents execute commands, install dependencies and run generated code autonomously. Running them in disposable, sandboxed environments limits the blast radius of malicious or faulty agent behavior. Use a sandboxing technology to isolate agent runs, e.g. container-based isolation (dev containers), lightweight virtual machines or OS-level sandboxing mechanisms. Mount only the repository being worked on, avoid mounting credentials (e.g. cloud CLI configuration, SSH keys) and drop unneeded capabilities. Destroy the environment after the task is finished. Restricting network egress of the sandbox is addressed by _Network isolation for AI agents_.

Risk

An AI agent may be manipulated (e.g. via prompt injection in source code, issues, dependencies or web content) or may malfunction. Without isolation, it can read secrets, modify unrelated projects, exfiltrate data or damage the developer workstation and connected systems.

Measure

Run AI agents and AI coding assistants with command execution capabilities in a dedicated, least-privilege sandbox (e.g. a container or virtual machine) which contains only the required project files and is destroyed after use.

Expected assessment evidence

- Show the sandbox configuration used for AI agent runs (e.g. a dev container definition) and demonstrate a live agent session running inside it. - Show that only the project working directory is mounted and no credentials (SSH keys, cloud CLI configuration) are reachable from inside the environment. - Demonstrate that the environment is destroyed after the task is finished.

Difficulty

knowledge: 2/5 · time: 2/5 · resources: 2/5 · Usefulness: 4/5

Mappings

Description

AI coding agents have their own permission model that controls which tools, commands and file operations they may execute and which actions are auto-approved (e.g. the allow-list in Claude Code settings). These grants accumulate over time through convenience approvals and form the first authorization layer, before any external credential is involved.

Risk

Overly broad "always allow" grants accumulate unnoticed. A manipulated or malfunctioning agent executes destructive commands, accesses credential stores or pushes code without any confirmation, because the permission was granted once and never reviewed.

Expected assessment evidence

- Show the hardened baseline permission configuration that is distributed to projects. - Demonstrate that a sensitive operation (e.g. push, deletion, access to credential paths) requires confirmation. - Show the last audit of accumulated allow-lists including date and findings.

Difficulty

knowledge: 2/5 · time: 1/5 · resources: 1/5 · Usefulness: 4/5

Mappings

Description

Every model call costs compute and money, and AI endpoints answer expensive requests on demand. Without limits a single user, script or runaway agent can exhaust the service: availability suffers or the provider bill explodes ("denial of wallet"), and mass querying of a model additionally enables extraction of its behavior via input-output harvesting. Limits apply in both directions: inbound on exposed AI endpoints (requests per user and time window), outbound for the organization's own agents (token and cost budgets, iteration caps against runaway loops). The generic resource limits for infrastructure are defined in _Virtual environments are limited_ (Implementation). This activity adds the AI-specific units: requests, tokens and cost.

Risk

An attacker or a buggy client floods an AI endpoint with expensive requests: the service becomes unavailable for legitimate users or causes unbounded provider costs. A runaway agent loops at machine speed and burns budget unnoticed. Unlimited mass querying supports model extraction and brute-force prompt attacks.

Measure

Enforce rate limits per user, API key or tenant on all AI endpoints and restrict input size before it reaches the model. Set token and cost budgets per agent, team and time period, with alerts before a budget is exhausted. Cap agent iterations (maximum tool calls or recursion depth per task) so runaway loops terminate. An LLM gateway or proxy centralizes these limits across models and providers.

Expected assessment evidence

- Show the rate limit configuration of an AI endpoint and a test where the limit triggers. - Show the token and cost budgets per agent or team and the alert before a budget is exhausted. - Show the iteration cap of an agent and what happens when it is reached.

Difficulty

knowledge: 2/5 · time: 2/5 · resources: 1/5 · Usefulness: 4/5

Mappings

Description

Launching an AI coding agent inside a cloned third-party repository can hand control to the repository author before the user grants any trust: documented vulnerabilities in several agents (e.g. Claude Code, Cursor) showed that files under the repository's control (agent settings, hooks, definitions of Model Context Protocol (MCP) servers, bundled executables) were already evaluated while the workspace trust dialog was still unanswered. Even a checkout done only for a code review can therefore be enough to compromise the developer workstation.

Risk

An attacker publishes a prepared repository; when a developer opens it with an AI agent, configuration or executables under the attacker's control run on the developer machine before any trust decision, with access to credentials and every other project on that machine.

Measure

Treat third-party repositories as untrusted workspaces: open them only in an isolated container, do not honor repo-local agent configuration (hooks, MCP servers, settings) of untrusted origin, keep agent versions patched and never disable workspace trust prompts.

Expected assessment evidence

- Show the documented procedure for opening third-party repositories with AI agents. - Demonstrate that workspace trust prompts are enabled and that repository-local agent configuration (hooks, MCP servers, settings) is not honored for untrusted repositories. - Show how agent versions are kept patched (update mechanism or policy).

Difficulty

knowledge: 2/5 · time: 1/5 · resources: 1/5 · Usefulness: 4/5

Mappings

Description

_Permission management for AI agents_ defines what an agent may do; this activity makes sure neither the agent nor a convenience-driven user can weaken that definition. Agent-side guardrails (settings, allow-lists, policy hooks) usually live in files the agent itself can write, and approval fatigue predictably leads users to switch on standing auto-approve. Enforcement therefore has to move to points the agent process cannot touch: - **Managed configuration**: distribute the permission baseline via administratively enforced settings that local project or user configuration cannot override, and disable standing auto-approve modes there. - **Hooks outside the write reach**: store policy hooks that run before tool calls outside the workspace and repository the agent can modify. Prefer narrow allow-lists over deny-lists: deny patterns are obfuscatable, and an allowed interpreter can run arbitrary code. - **Approval gate at the version control boundary**: route agent pushes through a proxy that holds them for review (e.g. FINOS GitProxy) and close the direct route to the version control host through credential and network design (see _Network isolation for AI agents_ and _Least privilege on external systems for AI agents_). Server-side rejection of pushed secrets is defined in _Block pushes containing secrets_ (Implementation, Development and Source Control): it applies to humans and agents alike, and because it is enforced on the git host, it belongs to the enforcement points an agent cannot bypass or rewrite.

Risk

A prompt-injected agent rewrites its own permission configuration or hook scripts, or a user worn down by approval prompts enables standing auto-approve. Every agent-side guardrail silently disappears while dashboards and settings still look enforced, and a hijacked agent pushes malicious code or secrets straight to the version control host.

Measure

Enforce agent guardrails at points outside the agent's write reach: administratively managed permission baselines that local settings cannot override (including disabled standing auto-approve), policy hooks stored outside the workspace and an approval gate at the version control boundary for agent pushes.

Expected assessment evidence

- Show the administratively enforced permission baseline and demonstrate that a local project or user setting (e.g. enabling auto-approve) does not override it. - Show that policy hooks run from a location the agent cannot write to. - Demonstrate that an agent push is held at the approval gate (proxy) and that the direct route to the version control host is closed for agent credentials.

Difficulty

knowledge: 3/5 · time: 2/5 · resources: 2/5 · Usefulness: 4/5

Mappings

Description

Beyond their own permission model, AI agents authenticate against external systems: source code management (e.g. GitHub), CI/CD, cloud providers and internal APIs. Scoped, short-lived and auditable credentials ensure an agent can only perform the actions required for its task on those systems.

Risk

AI agents using broadly scoped personal or service credentials can be tricked into destructive or unauthorized actions (e.g. deleting repositories, approving their own pull requests, accessing production data).

Measure

Provide AI agents with dedicated identities and short-lived, minimally scoped credentials (e.g. fine-grained access tokens restricted to one repository). Where supported, keep credentials out of the agent process entirely: a proxy or gateway injects them at the network boundary ("secretless" architecture), so a prompt-injected agent cannot exfiltrate what it never holds. Log and review agent actions separately from human actions.

Expected assessment evidence

- Show a dedicated agent identity in the source code management system (or other external system) and the scopes granted to it. - Show that agent credentials are short-lived and minimally scoped (token configuration or issuance policy). - Show where agent credentials are stored and injected, preferably outside the agent process (e.g. credential injection at a proxy or gateway). - Demonstrate that agent actions are distinguishable from human actions in the audit log.

Difficulty

knowledge: 3/5 · time: 2/5 · resources: 2/5 · Usefulness: 4/5

Mappings

Description

Restricting the network egress of AI agent environments prevents data exfiltration and the download of malicious tooling in case an agent is hijacked through prompt injection. Example for a coding agent like Claude Code running in a container: 1. **Firewall inside the container**: on container start, an init script running with the NET_ADMIN capability configures a default-deny egress policy (iptables) and resolves an allow-list of required domains into an ipset: model API (e.g. api.anthropic.com), package registries (e.g. registry.npmjs.org) and the source code management system. Anthropic's reference devcontainer ships this as init-firewall.sh. The agent process runs as a non-root user afterwards, so it cannot alter the firewall rules it is confined by. 2. **Alternative, egress proxy**: attach the container to an internal network without direct internet access (e.g. docker network create --internal) and route traffic through a filtering HTTP(S) proxy (HTTPS_PROXY environment variable) that enforces the domain allow-list and logs all requests. In Kubernetes, enforce the same with NetworkPolicies or a service mesh egress gateway. 3. **DNS**: allow DNS only to the internal resolver and only for allow-listed domains, otherwise blocked domains remain reachable via direct IP or DNS tunneling can be used for exfiltration. 4. **Verify**: from inside the running container, check that a request to an allow-listed endpoint succeeds and a request to an arbitrary host (e.g. curl https://example.com) is blocked. Isolation-conscious design pays a double dividend: forcing all agent traffic through one enforced egress point (firewall or proxy) creates a monitored bottleneck. Attacks that would otherwise be invisible must pass through it, which is a high-leverage place for detection (see the Google DeepMind AI Control Roadmap).

Risk

A prompt-injected AI agent can exfiltrate source code, secrets or personal data to arbitrary hosts, or download and execute attacker-controlled payloads.

Measure

Limit network access of AI agent environments to an allow-list of required endpoints (e.g. model API, package registries, source code management system). Deny all other egress traffic by default.

Expected assessment evidence

- Show the egress policy of the agent environment (firewall init script, proxy configuration or network policies) including the domain allow-list. - Demonstrate from inside the agent environment that an allow-listed endpoint is reachable and a request to an arbitrary host is blocked. - Show where blocked egress attempts are logged and who reviews them.

Difficulty

knowledge: 3/5 · time: 3/5 · resources: 2/5 · Usefulness: 4/5

Mappings

Description

Agent actions differ vastly in blast radius: reading data is harmless, transferring money is not. Graduated human oversight assigns every class of agent action an approval tier based on its reversibility: low-risk, reversible actions run autonomously within guardrails; reversible actions with business impact run under monitoring with an intervention window; irreversible actions (financial transactions, permission changes, sharing data with third parties, outbound communication) require explicit human approval before execution, high-risk ones by two independent people (four-eyes principle). Whereas _Permission management for AI agents_ governs the agent's tool-level permission model, this activity governs the business actions an agent performs through those tools. It completes the human-oversight ladder started by the artifact reviews (_Human review of AI generated specifications_, _Human review of AI generated plans_, _Human review of AI generated code_ in Verification): those gate what is built, this activity gates what a running agent does.

Risk

A manipulated (e.g. via prompt injection) or malfunctioning agent performs an action that cannot be rolled back: money is transferred, data reaches a third party, an e-mail is sent, permissions are escalated. After-the-fact monitoring cannot undo it. The opposite failure mode is rubber-stamping: approvals nobody actually reads create the appearance of oversight without any. Attackers exploit this deliberately. A manipulated agent justifies a harmful action with a fabricated but plausible rationale, and the approver waves it through because the agent appears competent (automation bias, see OWASP Top 10 for Agentic Applications, ASI09).

Measure

Classify agent action types by reversibility and assign approval tiers: pre-approval for irreversible actions, monitoring with an intervention window for reversible actions with business impact (e.g. buffer outbound messages for a defined period so an intervention stops delivery), autonomy only for low-risk reversible actions. If the reversibility of an action class cannot be demonstrated, default to pre-approval. Require a second, independent approver for high-risk actions, record the approver identity, the decision context and the reason in the audit log, and design the approval step so the approver actually sees the action and its context instead of clicking through.

Expected assessment evidence

- Show the classification of agent action types by reversibility and the approval tier assigned to each class. - Demonstrate that an irreversible action (e.g. an outbound message or a permission change) is blocked until a human approves it, and that an intervention within the monitoring window stops a buffered action. - Show a high-risk action with two documented, independent approvals. - Show an audit log entry of an approval including approver identity, decision context and reason.

Difficulty

knowledge: 3/5 · time: 3/5 · resources: 2/5 · Usefulness: 4/5

Mappings

Description

In multi-agent systems, agents delegate tasks to other agents, and the trust relationships between them are dynamic and often implicit. Two failure modes dominate (OWASP Top 10 for Agentic Applications, ASI03): - Unscoped privilege inheritance: a delegating agent hands its complete permissions down to a subagent whose task requires only a fraction of them. - Confused deputy: an attacker who controls a weakly privileged agent routes plausible requests through a strongly privileged one. The receiving agent executes them because they arrive from a familiar peer, and nobody checks the original intent. Explicit trust boundaries treat every delegation like an external request. The receiving agent verifies the identity and authorization of the delegating agent instead of trusting the chain. The messages between agents need the same protection as the delegation itself: without mutual authentication, message signing and replay protection, an attacker can spoof or replay agent messages and register fake agents in the discovery mechanism (ASI07, Insecure Inter-Agent Communication). Splitting a large agent into several smaller ones compartmentalizes risk only if each agent has its own identity and credentials. Shared credentials undo the compartmentalization. _Least privilege on external systems for AI agents_ scopes each agent's access to external systems; this activity scopes the trust between the agents themselves.

Risk

An attacker compromises one low-privilege agent (e.g. via prompt injection) and pivots through delegation: higher-privileged agents accept its requests unchecked, inherited access contexts carry full permissions into subtasks, and the attacker reaches systems the initially compromised agent could never access directly.

Measure

Give every agent its own identity and its own credentials, never shared between agents. Scope each delegated task to the minimum permissions it needs instead of passing on the delegating agent's access context. Check at every step of a multi-agent workflow that the delegating agent is who it claims to be and is authorized to request the action. Secure the communication channel itself with mutual authentication and signed messages (e.g. mutual TLS), replay protection (e.g. nonces and expiry times) and rejection of protocol downgrades. Accept agents into a workflow only from a registry or discovery mechanism that verifies agent identity and descriptors. Record the communication between agents so that delegations deviating from the usual patterns surface for review (see _Audit logging of AI agent actions_ and _Anomaly detection for AI agent behavior_).

Expected assessment evidence

- Show that each agent in a multi-agent workflow has its own identity and credentials (no shared credentials). - Show how a delegated task receives a reduced permission scope instead of the delegating agent's full access context. - Demonstrate that a delegation from an unauthorized or unknown agent is rejected. - Show how inter-agent messages are authenticated and protected against replay, and that a message from an unregistered agent or a protocol downgrade is rejected. - Show how inter-agent communication is logged and how unusual delegation patterns are flagged.

Difficulty

knowledge: 4/5 · time: 3/5 · resources: 2/5 · Usefulness: 4/5

Mappings

Description

Between specification and code sits the plan: which components are touched, which dependencies are added, how the security requirements will be implemented. Reviewing the AI-generated plan catches design-level flaws (a dropped security requirement, an insecure design choice, an unnecessary new dependency, changes to security-critical components) while they are still one line in a plan instead of hundreds of generated lines of code. In spec-driven development this is the gate of the plan phase; agents also propose plans outside formal workflows (e.g. a plan mode) that can be reviewed the same way.

Risk

The AI-generated plan silently drops security requirements from the specification, chooses an insecure design or pulls in avoidable dependencies. The agent then generates large amounts of code against the flawed plan. Code reviewers, biased by plausible-looking code, then verify the implementation against the plan instead of questioning the plan itself.

Measure

Require human review of the plan before AI-assisted implementation: verify that every security requirement from the specification is carried through, that security-relevant design decisions are justified and that changes to security-critical components are flagged for deeper review. In spec-driven workflows, make this review the gate of the plan phase and keep the reviewed plan under version control.

Expected assessment evidence

- Show a recent AI-generated plan and its documented human review before implementation. - Show that the review traced the security requirements from the specification into the plan. - Show how plans touching security-critical components are flagged for deeper review.

Difficulty

knowledge: 3/5 · time: 2/5 · resources: 1/5 · Usefulness: 4/5

Mappings

Description

In AI-assisted development, human attention scales best at the specification level: one page of specification determines what all downstream artifacts (plans, code, tests) will contain, and an error here is multiplied into every one of them. Reviewing the specification the AI works from (a user story with acceptance criteria, or the specify-phase artifact in spec-driven development) catches wrong intent, missing security requirements and wrong assumptions before anything is generated. It is the first rung of the human-oversight ladder that continues with _Human review of AI generated plans_, _Human review of AI generated code_ and _Human approval for irreversible AI agent actions_ (Isolation).

Risk

An AI assistant implements exactly what an unreviewed specification says: missing authorization requirements, absent abuse cases and wrong assumptions are faithfully turned into plans, code and tests. Later code review rarely catches this, because reviewers check whether the code matches the specification, not whether the specification is right.

Measure

Require human review of the specification before AI-assisted implementation starts: are the security acceptance criteria complete, are scope and assumptions correct, is anything security-relevant missing? In spec-driven workflows, make this review the gate of the specify phase and keep the reviewed specification under version control.

Expected assessment evidence

- Show a recent specification an AI assistant worked from and its documented human review before implementation started. - Show that the review checked the security acceptance criteria (e.g. review comments or a checklist). - In spec-driven workflows: show that the specify phase cannot be left without the review gate.

Difficulty

knowledge: 2/5 · time: 2/5 · resources: 1/5 · Usefulness: 5/5

Mappings

Description

An AI-generated change is only a claim until something independent confirms it. Instead of presenting an untested edit, the agent executes the checks the project already defines for humans (test suite, type checker, linter, build) in its sandboxed environment and keeps refining the change until the checks succeed or a defined budget ends the attempt. What reaches the human reviewer is then a verified state backed by check output, not an assertion. This introduces its own failure mode, reward hacking: an agent optimizing for passing checks may get there by weakening assertions, deleting failing tests or hardcoding expected values instead of fixing the code.

Risk

Unverified AI changes are delivered as plausible-looking code that does not build or breaks behavior. Conversely, an agent iterating to green without guardrails games the tests (weakened or deleted tests, hardcoded results), so a passing suite no longer means correct code.

Measure

Require AI agents to verify their changes against the project's own checks before presenting them, and guard the loop against test manipulation: record which checks failed before the agent started so it can only claim genuinely new fixes, route every edit that touches test code into human review, and re-run the full suite in the pipeline independent of the agent's environment.

Expected assessment evidence

- Show the agent workflow or configuration that runs the project's own checks (tests, type checker, linter, build) before changes are presented. - Show the guardrails against test manipulation: baseline comparison of test results, review flags on modified test code and an independent re-run of the suite in the pipeline.

Difficulty

knowledge: 3/5 · time: 2/5 · resources: 2/5 · Usefulness: 5/5

Mappings

Description

Static and dynamic security testing (SAST, DAST, software composition analysis) is defined in the _Test and Verification_ dimension and is author-agnostic. This activity references those activities for AI-generated code instead of duplicating them. The AI-specific delta is coverage and pace: AI agents produce code in places classic scan scopes miss (generated infrastructure code, scripts, one-off tools) and at a rate that makes scanning inside the development loop (e.g. the agent running the static analysis itself during self-verification) more important than for human-written code. Complementary: _No verification bypass for AI generated code_ ensures no delivery path skips these scans.

Risk

AI-generated code reproduces insecure patterns from training data at scale. If static and dynamic analysis does not cover all AI-produced artifacts, typical vulnerabilities (injection, insecure defaults, hardcoded secrets) ship unnoticed, faster and in higher volume than with human-written code.

Measure

Apply the established static and dynamic security analysis to all AI-generated code and artifacts. Include AI-produced infrastructure code and scripts in the scan scope and surface findings inside the AI-assisted development loop, so agents can fix them before human review.

Expected assessment evidence

- Show that the scan scope includes AI-produced artifacts such as generated infrastructure code and scripts. - Show a recent AI-generated change where static analysis findings were surfaced and fixed inside the AI-assisted development loop.

Difficulty

knowledge: 2/5 · time: 2/5 · resources: 2/5 · Usefulness: 4/5

Mappings

Description

AI assistants hallucinate package names: research found around 20% of dependencies suggested by LLMs do not exist. Attackers register these names ("slopsquatting") or typo-variants on public registries, so blindly installing AI-suggested packages imports attacker-controlled code. Automate the checks so they do not depend on developer discipline: - **Enforce a minimum OpenSSF Scorecard score** for new dependencies, e.g. as a policy check in a pull-through proxy like an artifact registry or in the pull request pipeline. Scorecard rates projects on maintenance, code review, branch protection, dangerous workflows and vulnerability handling; a threshold (e.g. score >= 5) filters low-hygiene and abandoned packages before a human even reviews them. - **Verify identity and existence**: does the package match the intended project (publisher, source repository, age, download counts)? Hallucinated and slopsquatted names fail exactly these checks; tools like packj flag them automatically. - **Gate before merge**: run the established software composition analysis and the artifact repository policy on the change that adds the dependency, so a malicious or compromised version never enters the build.

Risk

A developer installs an AI-suggested package that is hallucinated, typo-squatted or malicious. The package executes attacker code during install or at runtime and compromises the application and build environment.

Measure

Verify every AI-suggested dependency before adoption: check that the package exists, is the intended one (name, publisher, repository), is actively maintained and passes the established software composition analysis. Prefer dependencies already used in the organization.

Expected assessment evidence

- Show the documented verification steps for adopting a new dependency and a recent example of an AI-suggested package that was checked (or rejected). - Show that software composition analysis covers AI-suggested packages before merge.

Difficulty

knowledge: 2/5 · time: 2/5 · resources: 1/5 · Usefulness: 4/5

Mappings

Description

AI-generated code is plausible-looking by construction and often accepted without scrutiny ("automation bias"). It must be treated like code from an untrusted contributor: a human with knowledge of the codebase reviews it before it is merged. The upstream reviews of specification and plan (_Human review of AI generated specifications_, _Human review of AI generated plans_) reduce what code review must catch, and the automated gates (_Self-verification of AI generated changes_, _Static and dynamic analysis of AI generated code_) filter mechanical findings. Neither replaces this activity: agents deviate from approved plans, so the code remains the last artifact gate before merge.

Risk

AI-generated code containing subtle logic flaws, insecure defaults or hallucinated/typo-squatted dependencies is merged unreviewed because it "looks right" and compiles.

Measure

Require human review for all AI-generated changes before merge. Reviewers are accountable for the change as if they had written it. AI agents must not be able to approve or merge their own pull requests.

Expected assessment evidence

- Show branch protection requiring human review before merge and that agent identities cannot approve or merge their own pull requests. - Show recent reviews of AI-generated changes and who is accountable for them.

Difficulty

knowledge: 2/5 · time: 3/5 · resources: 1/5 · Usefulness: 5/5

Mappings