Codex Security CLI: Gate PRs on High Severity

Why this matters this week

OpenAI used DevDay on September 29 to launch Codex Security Cloud for repository scanning, and the part developers can use today without waiting on a waitlist is the open-source package behind it: @openai/codex-security, an Apache-2.0 CLI and TypeScript SDK. It points a GPT-5.6 model at your repository, asks it to build a threat model, hunt for vulnerabilities, try to validate each candidate, and write a report with attack paths and fix suggestions.

That is different from a rule-based scanner. A regex or AST rule flags every exec() call. This tool is meant to follow data across files, decide whether the path is actually reachable, and only then report. The price is that it costs real model tokens, takes minutes instead of seconds, and needs guardrails so it does not surprise you on a large monorepo.

This guide builds a practical setup: a config file, a cost-capped scan of only the files a pull request changed, a CI gate that fails on high severity, and a small script that turns the findings file into a readable triage table. You will also see the traps I hit while installing it, including one that can make you run the wrong package.

One honest note about the examples. A real scan needs an OpenAI login or API key and spends money, so the output shown below comes from the tool's built-in --mock mode, which writes synthetic findings in the real file formats without calling a model. The command flags, exit codes and file layouts are real (I ran them against version 0.1.31); the vulnerability text is fake and labelled [Mock].

Prerequisites

• Node.js 22.13+ (22.x), 24.x or 26.x, and Python 3.10+ (scans, exports and history use Python; on 3.10 you also need tomli)
• A Git repository you own or are authorized to assess. Findings can include source code and reproduction steps, so treat the output as sensitive
• Either a ChatGPT sign-in (login) or an OPENAI_API_KEY / CODEX_API_KEY environment variable for real scans
• Basic comfort with a CI system. The examples use plain shell so they port to GitHub Actions, GitLab or anything else

Step 1: Install it the safe way

Install the package inside a project or globally, then confirm the version:

npm install @openai/codex-security
npx @openai/codex-security --version
# 0.1.31

Always use the scoped name. The command the package installs is called codex-security, but if you type npx codex-security in a folder where it is not installed, npm offers to fetch a different, unscoped package. When I did that, it printed a notice saying I had run the unscoped package and probably wanted npx @openai/codex-security. Nothing harmful happened, but in CI that kind of typo is how the wrong code gets executed with your API key in the environment. The maintainers also recommend installing the CLI outside the repository you are scanning so a hostile checkout cannot shadow it.

Before spending anything, run a dry run. It validates your local inputs without starting a model, authenticating or touching the network:

npx @openai/codex-security scan . --dry-run --json --fail-on-severity high --max-cost 5 --path app

{
  "dryRun": true,
  "repository": "/work/demo",
  "target": { "kind": "paths", "paths": ["app"] },
  "mode": "standard",
  "outputDir": null,
  "authentication": { "method": "stored_credentials", "verified": false },
  "model": "gpt-5.6-sol",
  "reasoningEffort": "xhigh",
  "maxCostUsd": 5
}

Read the defaults in that output: model gpt-5.6-sol at xhigh reasoning effort. Those are the expensive settings. Standard mode is the default; --mode deep runs multi-pass discovery with workers and subagents and can run for up to 96 hours, so do not turn it on in a pull request check.

Step 2: Create a vulnerable test app

So you can see the moving parts, make a tiny repo with two classic bugs, SQL built by string concatenation and a shell command built from user input:

mkdir demo && cd demo && git init
mkdir app && cat > app/server.js <<'EOF'
const express = require("express");
const { execSync } = require("child_process");
const db = require("./db");
const app = express();

app.get("/users", async (req, res) => {
  const rows = await db.query(`SELECT id, name FROM users WHERE name = '${req.query.name}'`);
  res.json(rows);
});

app.get("/ping", (req, res) => {
  res.send(execSync("ping -c 1 " + req.query.host).toString());
});
app.listen(3000);
EOF
git add -A && git commit -m init

A human reviewer would catch both in seconds. The point of using something this obvious is to check your pipeline, not to test the model's limits.

Step 3: Put your settings in a project file

Flags get long. The CLI can write a starter config for you:

npx @openai/codex-security init ./codex-security.yaml

Every setting is commented out by default, so the file does nothing until you opt in. A sensible pull request profile looks like this:

# codex-security.yaml
scan:
  mode: standard
  scope:
    paths: [app]
codex:
  model: gpt-5.6-sol
  model_reasoning_effort: high
limits:
  max_cost_usd_per_scan: 5
policy:
  fail_on_severity: high
output:
  directory: ../scan-results

Load it with -c:

npx @openai/codex-security scan . -c codex-security.yaml --dry-run --json

Three details from the docs are worth knowing. First, precedence runs built-in defaults, then the file, then explicit CLI flags, so a flag always wins. Second, the file is trusted like a command line: it can start MCP server processes and choose model endpoints, so keep it outside the checkout you are scanning. A pull request that edits its own scanner config could otherwise loosen the gate. Third, a missing or invalid config exits with code 2, and nothing is discovered automatically; if you do not pass -c, no file is read.

Step 4: Scan only what the PR changed

Scanning a whole repository on every pull request is slow and pricey. --diff limits the run to committed changes between a base ref and --head (default HEAD):

npx @openai/codex-security scan . \
  --diff origin/main \
  --max-cost 5 \
  --fail-on-severity high \
  --output-dir "$HOME/scan-results/pr-42" \
  --headless --format json > findings-summary.json
echo "exit code: $?"

What each flag does here:

• --diff origin/main scans the commits on your branch relative to main. For uncommitted edits, use --working-tree instead, which compares staged and unstaged changes against --base (default HEAD)
• --max-cost 5 stops the scan and its workers once estimated spend passes five dollars. In-flight requests can finish above the line, so treat it as a soft ceiling rather than a hard one
• --fail-on-severity high turns a report-only scan into a gate: exit code 1 if a completed scan finds high or critical issues
• --output-dir must be outside the repository and its Git worktree, and on macOS/Linux an existing directory has to be private (chmod 700). Skip this and the tool refuses
• --headless replaces the full-screen dashboard with plain status lines, which is what you want in CI logs

Step 5: Read the exit code correctly

The exit code is the contract your pipeline depends on, so get it right. From the package documentation:

Exit code	Meaning
0	Completed report-only scan, or policy passed
1	Policy violation: findings at or above your severity threshold
2	Invalid input, incomplete coverage, or a runtime/export error
130	Interrupted
143	Terminated

The difference between 1 and 2 matters. Exit 1 means the scanner worked and found something, so block the merge. Exit 2 means it could not finish, and an incomplete scan that printed zero findings proves nothing. The docs put it bluntly: zero findings means nothing if the scan skipped an important service. Do not collapse both into a generic failure; route 2 to whoever owns the pipeline, and route 1 to the PR author.

With --format json, an execution failure writes a structured object to stdout ({"status": "failed", "code": "SCAN_FAILED", "message": "..."}) while human-readable diagnostics stay on stderr, so parse stdout and log stderr.

Step 6: See what a run produces

I ran the mock mode against the demo repo with a high threshold to show the artifact layout and the exit behavior:

npx @openai/codex-security scan ./demo --mock --headless \
  --output-dir ./out/r --format json --fail-on-severity high > mock.json
echo $?   # 1

find out/r -type f
# out/r/scan-manifest.json
# out/r/findings.json
# out/r/coverage.json
# out/r/report.md
# out/r/exports/results.sarif
# out/r/checkpoints/<hash>.json

The terminal summary at the end:

  FINDINGS  12 (12 confirmed this scan; 0 previously found; 2 critical, 4 high, 3 medium, 2 low, 1 informational)
  COVERAGE  complete
  ELAPSED   0s

The mock found critical and high items, so the exit code was 1, exactly as the policy says it should be. Four files matter. findings.json is the canonical list. coverage.json says whether the scan was complete, partial or unknown, and which surfaces were reported or skipped. report.md is a readable projection of those files (it was generated from them, so it is not a separate source of truth). exports/results.sarif is SARIF 2.1.0, ready for code-scanning dashboards.

Each finding in findings.json carries more than a title. Fields from the real schema include severity.level, confidence.level, locations[] with path and line range, attackPath.summary, rootCause.summary, codeEvidence[], remediation, remediationTests, taxonomy.cwe, validation and a stable findingId plus per-scan occurrenceId. That last pair is what lets the tool tell you a bug was found before, rather than listing it as new every run.

Step 7: Build a triage table from the findings

Reading raw JSON in a pull request is miserable. This short script, which I ran against the mock output, sorts by severity and prints a Markdown table you can paste into a PR comment:

import json, sys
from collections import Counter

ORDER = ["critical", "high", "medium", "low", "informational"]

def load(path):
    with open(path) as f:
        return json.load(f)["findings"]

def line(f):
    loc = f["locations"][0]
    cwe = ",".join(f.get("taxonomy", {}).get("cwe", [])) or "-"
    return (f["severity"]["level"], f["title"],
            f"{loc['path']}:{loc['startLine']}", cwe,
            f["confidence"]["level"])

def main(path, min_level="medium"):
    findings = load(path)
    counts = Counter(f["severity"]["level"] for f in findings)
    print("Severity mix:", ", ".join(f"{k}={counts[k]}" for k in ORDER if counts[k]))
    cutoff = ORDER.index(min_level)
    rows = sorted((line(f) for f in findings), key=lambda r: ORDER.index(r[0]))
    print(f"\n| Severity | Title | Where | CWE | Confidence |\n|---|---|---|---|---|")
    for r in rows:
        if ORDER.index(r[0]) <= cutoff:
            print("| " + " | ".join(r) + " |")

if __name__ == "__main__":
    main(sys.argv[1], sys.argv[2] if len(sys.argv) > 2 else "medium")

python3 triage.py out/r/findings.json high

Severity mix: critical=2, high=4, medium=3, low=2, informational=1

| Severity | Title | Where | CWE | Confidence |
|---|---|---|---|---|
| critical | [Mock] Export filename reaches a shell command | mock/command-injection.ts:10 | CWE-78 | high |
| critical | [Mock] Shell metacharacters in the export name execute commands | mock/command-injection.ts:10 | CWE-78 | high |
| high | [Mock] Archive entry escapes the extraction directory | mock/path-traversal.ts:10 | CWE-22 | high |
| high | [Mock] Document download omits the ownership check | mock/missing-authorization.ts:10 | CWE-862 | high |
| high | [Mock] Search term is concatenated into a SQL query | mock/sql-injection-626df2c0-d6a2-45b1-8008-4cd2a3c1f8a4.ts:10 | CWE-89 | high |
| high | [Mock] Parent-directory archive names overwrite files outside extraction | mock/path-traversal.ts:10 | CWE-22 | high |

Notice the two critical rows point at the same file and line. Even the synthetic data includes near-duplicates, which is realistic: a model can describe one root cause twice. The findings service in the package has a deduplication workflow for exactly this, but for a first setup, grouping by path:line in your own script is a cheap first filter.

Step 8: Use the TypeScript SDK when you need control

If you want scan results inside your own tooling, the SDK does the same job. This follows the package README:

import { CodexSecurity, loadProjectConfig } from "@openai/codex-security";

const { config, options } = await loadProjectConfig("codex-security.yaml");
await using security = new CodexSecurity(config);

const result = await security.run("/path/to/repository", {
  ...options,
  outputDir: "/path/outside/repository/results",
});

console.log(result.reportPath);
console.log(result.findings.findings.length);

if (
  options.failureSeverity !== undefined &&
  result.hasFindingsAtOrAbove(options.failureSeverity)
) {
  process.exitCode = 1;
}

Two SDK behaviors catch people out. result.findings holds only this scan's findings, while repositoryFindings also includes earlier open ones, and matching them against history can trigger extra model calls. And the SDK records failureSeverity but never throws or sets the exit status itself; your code decides, as the last lines above do. For cost control in code, maxCostUsd is an option and onBudgetApproaching lets a callback raise the limit when spending reaches 80 percent.

A worked example: the whole pipeline

Put the pieces together in a script a CI job can call. It scans the diff, keeps the scanner's exit code, prints the triage table, and maps the result to a decision:

#!/usr/bin/env bash
set -u
RESULTS="$(mktemp -d)"; chmod 700 "$RESULTS"

npx @openai/codex-security scan . \
  -c ../ci-config/codex-security.yaml \
  --diff origin/main \
  --output-dir "$RESULTS/run" \
  --headless --format json > "$RESULTS/summary.json"
code=$?

case $code in
  0) echo "Clean: no findings at or above the threshold." ;;
  1) echo "Blocking: high/critical findings."
     python3 triage.py "$RESULTS/run/findings.json" high ;;
  2) echo "Scan incomplete or failed. Do NOT treat as a pass."
     cat "$RESULTS/summary.json" ;;
  *) echo "Interrupted or terminated ($code)." ;;
esac
exit $code

Note the config lives at ../ci-config/, outside the checkout, following the trust warning from Step 3. On the demo repo, a real scan should flag the string-built SQL query and the execSync call with user input, typically as CWE-89 and CWE-78. I could not run a live model scan in this environment, so treat those two expected findings as what to look for, not as recorded output.

Step 9: Go further with validate, patch and verify-fix

The same CLI closes the loop on a finding. Each of these is documented in the package README:

# Does this candidate hold up?
npx @openai/codex-security validate "Possible SQL injection" --effort high

# Generate and verify a fix for one saved finding
npx @openai/codex-security patch OCCURRENCE_ID

# Patch every high+ finding from the latest scan and open draft PRs
npx @openai/codex-security patch --scan latest --severity high --create-pr

# Check whether a claimed fix really closes the hole
npx @openai/codex-security verify-fix --scan latest --json

verify-fix returns fixed, still_vulnerable or inconclusive with evidence, and its exit code is 0 only if everything is fixed. patch runs in a sandbox; if the sandbox cannot start it fails with SANDBOX_UNAVAILABLE rather than quietly running unisolated. Only pass --external-sandbox when the container you run in already enforces filesystem and network isolation, because that flag tells Codex not to.

For a local safety net, codex-security install-hook /path/to/repo --fail-on-severity high installs an advisory pre-commit check. It is advisory only; the docs are explicit that a required CI check is what actually enforces the policy.

Common pitfalls

• Wrong package name. npx codex-security without the scope can fetch an unrelated package. Use npx @openai/codex-security or a global install you verified
• Output directory rejected. It must be outside the scanned directory and enclosing Git worktree, and private (chmod 700) on macOS/Linux. Use mktemp -d in CI
• Treating exit 2 as a pass. Incomplete coverage exits 2. A green check on a partial scan gives false comfort; read coverage.json
• Trusting --max-cost as a hard cap. In-flight requests finish after the limit is crossed. Budget a little above it. Also, post-scan prompts run after cost tracking ends and sit outside the limit
• Cost numbers are API-equivalent estimates. They show a min and max range using standard API prices, not your ChatGPT plan allowance, and models without known prices get no estimate
• Scanner config inside the checkout. The project file can start MCP servers and choose model endpoints. A pull request that edits it could weaken its own gate. Keep it outside
• Deep mode in PR checks. It is multi-pass, can run up to 96 hours, and only supports repository and path targets, not diffs
• Sensitive output. Reports can include source code and exploit steps. Restrict artifact access and review logs before sharing them
• Pre-1.0 churn. The README says minor releases before 1.0.0 may change the public API. Pin the version in CI and read release notes before upgrading
• Bulk scans and diffs. Bulk scans use shallow clean checkouts and reject diff or working-tree scopes unless each CSV row gives its own path scope

Quick reference

Goal	Command or flag
Check inputs without spending	scan . --dry-run
Scan only PR changes	--diff origin/main
Scan uncommitted edits	--working-tree (with optional --base)
Limit to folders	--path src --path tests
Add threat models or docs	--knowledge-base PATH (repeatable)
Cap estimated spend	--max-cost 5
Fail CI on severity	--fail-on-severity high
Cheaper model/effort	--model, --effort high
Reusable settings	init, then scan -c codex-security.yaml
Export SARIF/CSV/JSON	export (reads a sealed, completed scan)
Fix and check	patch, verify-fix
Pre-commit check	install-hook
Free smoke test	scan --mock

Next steps

• Run scan --mock in your real pipeline first to prove the plumbing, the exit codes and the artifact upload, before you attach a billing credential
• Add --knowledge-base with your architecture notes or threat model so the model stops rediscovering your auth design every run
• Upload exports/results.sarif to your code-scanning dashboard so findings show up next to your other tools
• Use scans compare between two scan IDs to report only new and resolved findings in your weekly review
• After a week of real runs, tune the threshold. If high is too noisy, try critical on PRs and high on a nightly full scan

Sources: the openai/codex-security repository and the package README shipped in @openai/codex-security 0.1.31, the OpenAI developer documentation for Codex Security, and The Decoder's DevDay coverage.