CodeQL now finds system prompt injection: turn AI security into a CI data-flow problem

Tech

A practical guide to CodeQL 2.26.0’s js/system-prompt-injection query, from trusted prompt boundaries and tool descriptions to evals and runtime authorization.

Security diagram showing user input flowing toward system prompts and tool descriptions, then checked by CodeQL, adversarial evaluations, and runtime authorization gates
Static analysis finds trust-boundary mistakes in code; evaluations and runtime authorization must cover the remaining attack surface.

What happened

Security reviews for JavaScript and TypeScript AI features often stop at whether the prompt “looks safe.” CodeQL 2.26.0 moves the discussion into familiar application-security territory: can user-controlled data flow from a source into a trusted system prompt or tool description sink? The important change is not another prompt-writing rule. It is the ability to review a trust boundary as a path through code.

GitHub announced CodeQL 2.26.0 on July 10, 2026 with the new `js/system-prompt-injection` query. It detects untrusted values flowing into an AI model’s system prompt. The release also expands sink modeling across OpenAI, Anthropic, and Google GenAI SDK APIs, including Sora prompts, OpenAI Realtime session instructions, Google GenAI cached content, and system instructions.

The query help deliberately goes beyond system messages. Concatenating user input into a tool description is also flagged because tool descriptions are trusted instructions that influence selection and behavior. GitHub recommends moving the value into a user-role message or a structured tool parameter. If a dynamic value genuinely must alter a trusted prompt, it should be constrained by a fixed allowlist.

Why it matters

This matters because innocent-looking string composition is easy to miss in review. `instructions: base + req.query.persona` may look like harmless customization. If `persona` is an arbitrary instruction channel, however, it can affect tool selection, data access, and downstream operations. The CodeQL help lists high precision, a security severity of 7.8, and inclusion in the standard JavaScript code-scanning suite.

The query does not solve prompt injection as a whole. Indirect content discovered at runtime—web pages, email, search results, or RAG chunks—cannot always be proven from static code flow. Static analysis also cannot determine whether a model obeyed an attack, whether its refusal harmed legitimate users, or whether a resulting tool call exceeded authorization. CodeQL is the layer for code-created trust-boundary defects, not a complete defense.

What community signals reveal

A recent DevSecOps community discussion illustrates the gap. One team converted attacks found in production traces into a PR evaluation set; commenters suggested scoring system-prompt leakage, role breaking, tool misuse, and customer-data exposure separately. This is a practitioner signal, not a factual source for CodeQL. It reinforces that closing a static alert and repeatedly testing model behavior are different jobs.

Development and operations impact

For development teams, inventory every path from HTTP parameters, webhook payloads, database fields, and fetched documents into `system`, `developer`, `instructions`, cached system content, and tool descriptions. When an alert fires, do not begin with string sanitization. Ask why the value needs instruction-level authority. Most customization can move down into a user message or a validated enum parameter.

Runtime authorization must be the last safety boundary. Separate read and write tools, validate tenant scope on the server, require independent approval for destructive actions, and use idempotency keys. Never store secrets in a system prompt. Access should be decided by server policy rather than by whether the model appears to have refused malicious text.

A practical CI rollout has three stages. First, collect results in audit mode and classify existing debt. Next, gate only new high-confidence paths while running adversarial and benign edge-case evaluations. Finally, rerun the corpus whenever the model, SDK, prompt, or tool set changes, and append patterns discovered in incidents and traces rather than replacing the old set.

A checklist for this week

Confirm JavaScript/TypeScript CodeQL analysis uses a current bundle.

Inventory user-controlled flows into system/developer instructions and tool descriptions.

Move free-form values into user messages or structured tool parameters.

Constrain necessary dynamic instruction values with fixed allowlists and enums.

Test benign edge cases as well as malicious prompt-injection attempts.

Enforce tenant scope, least privilege, approval, and idempotency on the server.

Rerun static analysis and adversarial evaluations after model, SDK, prompt, or tool changes.

Risks and counterarguments

There is a reasonable counterargument: placing locale, persona, or response style in a system prompt is convenient and may produce more consistent output. The answer is not to remove all dynamic behavior. Model those choices as enums, validate them server-side, and map them to fixed templates. That preserves product flexibility without promoting arbitrary user text into a trusted instruction.

The release matters because prompt security is becoming ordinary AppSec. We can talk about sources, trusted sinks, paths, allowlists, and CI gates instead of treating prompts as magic strings. Keep the layers distinct: CodeQL for code paths, evaluations and red teams for model behavior, and least privilege plus server policy for blast-radius control.

Sources