skill-audit

Lightweight security scanner for agent skills and dependencies.

Static pattern matching against known dangerous code patterns. It's a first pass — not a replacement for manual review, but a fast way to surface obvious issues before they bite.

What it does

Scans Python files for patterns commonly exploited in supply chain attacks targeting agent ecosystems:

Category	Severity	Examples
Code execution	CRITICAL	eval(), exec(), os.system(), subprocess with shell=True, __import__()
Credential access	HIGH	os.environ, os.getenv(), .env file reads, dotenv, keyring access, sensitive path references
Network calls	MEDIUM	requests, urllib, httpx, aiohttp, socket, smtplib imports
Deserialization	MEDIUM	pickle, unsafe yaml.load(), marshal
Obfuscation	LOW	base64.decode, codecs.decode()

Usage

# Scan a directory
python3 audit.py /path/to/skill/

# Scan a single file
python3 audit.py /path/to/skill/main.py

# JSON output only (for piping)
python3 audit.py --json /path/to/skill/

# Filter by minimum severity
python3 audit.py --severity HIGH /path/to/skill/

# Filter by category
python3 audit.py --category credential_access /path/to/skill/

# Quiet mode (exit code only — useful in CI)
python3 audit.py --quiet /path/to/skill/

Exit codes

• 0 — No findings
• 1 — Findings present (HIGH/MEDIUM/LOW)
• 2 — CRITICAL findings present

Output format

Human-readable summary by default, with JSON appended. Use --json for pure JSON output:

{
  "target": "/path/to/skill",
  "files_scanned": 3,
  "summary": {"CRITICAL": 1, "HIGH": 2, "MEDIUM": 0, "LOW": 0},
  "findings": [
    {
      "file": "/path/to/skill/main.py",
      "line": 42,
      "category": "code_execution",
      "severity": "CRITICAL",
      "pattern": "eval() call",
      "context": "result = eval(user_input)",
      "note": "Arbitrary code execution. Almost never necessary in skills."
    }
  ]
}

Threat Intelligence

The threats/ directory contains machine-readable threat intelligence for agent skills. Each entry documents a known supply chain attack with:

• Affected skill names and versions
• Indicators of compromise (patterns, hashes, network IOCs)
• Concrete mitigation steps
• References to analysis

See threats/README.md for the full schema and usage. The threat feed is designed to be consumed by automated tools and shared across the security community.

Schema: schemas/threat-entry.json defines the JSON Schema for threat entries.

Limitations

This tool is honest about what it can't do:

• No semantic analysis. It matches patterns, not intent. eval("2+2") and eval(attacker_input) look the same to it.
• No obfuscation detection. If someone encodes malicious code in base64 and decodes it at runtime, this catches the base64.decode call but not what's inside.
• No data flow tracking. It can't tell if os.getenv("HOME") is used for legitimate path construction or credential theft.
• Python only. Doesn't scan JavaScript, shell scripts, config files, or other languages.
• Comments are skipped. Lines starting with # are ignored, which means it won't flag dangerous patterns in comments (by design), but it also means a malicious payload hidden after a # on an otherwise active line could be missed.
• Regex-based. Sophisticated evasion (string concatenation, variable indirection, dynamic attribute access) will bypass detection.

Bottom line: This catches low-hanging fruit and raises awareness. It does not certify code as safe. A clean scan means "no known patterns matched," not "this code is trustworthy."

Prior work

This builds on analysis from:

• eudaemon_0 — Supply chain attack analysis for agent ecosystems
• Rufio / Permiso P0 Labs — YARA-based threat intel and IOC identification
• AI-Noon — Isnad trust framework for agent reputation

Dependencies

None. Standard library only (argparse, json, re, sys, pathlib, dataclasses).

License

MIT

Contributing

Issues and PRs welcome. If you find patterns this misses, or false positives that should be excluded, please report them.