Mitigation & Defense

Restrict who and what can reach AI models, APIs, and training pipelines. The principle: a credential or token should be the smallest possible footprint for the task it enables.

SPECIFIC ACTIONS

• Enforce least-privilege API keys scoped per model endpoint; rotate on a 90-day maximum cycle
• Require MFA on all accounts with access to training data, model registries, and inference APIs
• Implement service mesh mTLS between AI microservices to prevent lateral movement
• Audit and prune stale service accounts in ML pipelines quarterly
• Gate model-management consoles behind jump hosts or PAM vaulting, not VPN alone

Sanitize, validate, and bound every input before it reaches a model. Prompt injection — direct and indirect — is the primary attack surface for deployed LLM systems.

SPECIFIC ACTIONS

• Deploy a prompt firewall layer that detects and strips injection patterns before reaching the model
• Enforce strict schema validation on structured inputs; reject or escape free-text where schema suffices
• Use separate system and user message boundaries enforced at the API level, not just in prompt templates
• Log all prompts verbatim (hashed where PII present) and alert on anomalous token patterns
• Rate-limit per-user and per-session to throttle automated jailbreak probing

Protect the model artifact itself — weights, training data, fine-tune checkpoints — from theft, poisoning, or unauthorized modification.

SPECIFIC ACTIONS

• Encrypt model weights at rest and in transit; use hardware-backed key management (HSM or cloud KMS)
• Sign model artifacts with a verified provenance chain; reject unsigned models in inference pipelines
• Isolate training jobs in dedicated compute environments with no outbound internet access
• Validate training datasets for poisoning indicators (label distribution anomalies, trigger-pattern scans) before each training run
• Version-control all fine-tunes and implement diff-based monitoring between checkpoint versions

Continuous observability across model behavior, infrastructure traffic, and output quality. You cannot defend what you cannot see.

SPECIFIC ACTIONS

• Instrument inference endpoints with output-distribution monitors; alert on statistical drift from baseline
• Deploy behavioral anomaly detection on agent action logs — flag unexpected tool calls or API sequences
• Stream model audit logs to a SIEM with AI-specific detection rules (hallucination rate, refusal bypass patterns)
• Establish red-team cadence: monthly automated probing plus quarterly adversarial human testing
• Monitor data pipelines for exfiltration signals — large token volumes sent to external endpoints

Govern what external tools, plugins, and APIs AI agents can reach. Agentic systems with broad tool access are a primary lateral movement vector.

SPECIFIC ACTIONS

• Maintain an approved tool allowlist; agents may not call unapproved APIs regardless of prompt instruction
• Sandbox tool execution in isolated environments with network egress controls and memory limits
• Require human confirmation before agent actions that write, delete, or transmit data outside the system boundary
• Log all tool call inputs and outputs with full parameter capture for forensic traceability
• Validate tool responses before passing to the model — treat external API output as untrusted input

Third-party models, datasets, and ML libraries are a persistent blind spot. Supply chain risk in AI mirrors software supply chain risk — with the added dimension of pre-trained weight provenance.

SPECIFIC ACTIONS

• Vet foundation model providers against documented security and alignment criteria before adoption
• Pin ML framework dependencies and run automated vulnerability scanning on each build
• Require a model card and dataset provenance attestation for any third-party model entering production
• Scan open-source datasets for copyright violations, PII, and known poisoning indicators before training use
• Maintain a software bill of materials (SBOM) extended to cover model weights and training data lineage

Technical controls fail without human process. This domain covers the organizational practices, training, and oversight mechanisms that give technical controls their force.

SPECIFIC ACTIONS

• Define and document AI-specific incident response playbooks covering model misbehavior, data poisoning, and exfiltration scenarios
• Train developers on prompt injection attack patterns and secure AI coding practices annually
• Implement a human-in-the-loop review gate for high-stakes AI outputs (legal, medical, financial decisions)
• Establish a clear escalation path for employees who observe unexpected AI behavior
• Run tabletop exercises simulating AI system compromise at least twice per year

Validate and filter what the model returns before it reaches users or downstream systems. Output is an attack surface just as input is.

SPECIFIC ACTIONS

• Deploy an output safety classifier tuned for your domain before responses reach production users
• Strip or redact PII patterns (email, SSN, card numbers) from outputs before delivery
• Implement output schema enforcement for structured workflows — reject malformed JSON, code, or commands
• Set hard content policy filters for categories that are non-negotiable in your use case
• Log and review a random sample of outputs weekly; flag for human review when confidence scores are low

When an AI system is compromised or fails, how fast can you detect it, contain it, and restore safe operation? Resilience planning is the last line of defense.

SPECIFIC ACTIONS

• Maintain versioned model snapshots with tested rollback procedures documented and rehearsed
• Design circuit-breaker patterns to degrade AI functionality gracefully rather than fail open
• Define RTO and RPO targets specifically for AI components; include them in DR plans
• Isolate AI systems in blast-radius-limited segments so a compromise doesn't cascade to core infrastructure
• Test recovery procedures quarterly — including restoring a known-good model from backup under incident conditions