# THREAT DOMAIN TAXONOMY: MODEL FLAWS (INHERENT VULNERABILITIES, HALLUCINATIONS, AND ALIGNMENT FAILURES)

## 1. Standardized Phase Taxonomy (The Threat Lifecycle)
Unlike adversarial examples that mathematically force a model to misclassify, the "Model Flaws" domain involves exploiting the inherent, native defects within the model's architecture, alignment, or training limitations. This includes exploiting catastrophic forgetting, tokenizer vulnerabilities, inherent bias, unaligned reasoning failures, or forcing severe hallucinations (confabulations) to degrade system integrity or cause Denial of Service (DoS).

*   **Reconnaissance:** The adversary interacts with the model to map its architectural limitations and alignment gaps. This involves testing context window limits, submitting anomalous tokens (e.g., specific UTF-8 byte sequences known to break tokenizers), and probing for topical areas where the model lacks training data and is prone to hallucination.
*   **Weaponization & Delivery (Initial Access):** The adversary crafts an input designed specifically to trigger the native flaw. This could be a "sponge example" (an input designed to maximize computation time and exhaust GPU memory), a complex logic trap that forces the model into an infinite generative loop, or a prompt that reliably triggers a toxic or factually compromised hallucination. Delivery occurs via standard application interfaces.
*   **Exploitation & Installation (Execution/Persistence):** The model processes the payload and its internal architecture or alignment protocols fail. The flaw is actively exploited. For example, a tokenizer bug may cause the underlying inference engine to crash, or an alignment failure may cause the model to confidently output highly damaging, fabricated information. There is no persistent installation; the vulnerability is an immutable characteristic of the deployed model weights and architecture.
*   **Command and Control (C2):** N/A. Control is asserted transactionally via the crafted inputs that dictate the model's failure mode.
*   **Actions on Objectives (Impact/Exfiltration):** The adversary achieves Denial of ML Service (resource exhaustion or system crash), reputational damage (coercing the organization's AI to output hate speech or fabricated facts), or secondary social engineering (forcing the AI to hallucinate a malicious URL that a legitimate user then clicks).

## 2. TTP Mapping (Tactics, Techniques, and Procedures)
For audit verifiability, this domain maps to MITRE ATLAS, alongside traditional ATT&CK techniques where underlying software vulnerabilities in the ML stack are exploited.

*   **Reconnaissance:** AML.T0040 (ML Model Inference API Access); AML.T0041 (Discover ML Model Ontology).
*   **Weaponization & Delivery:** AML.T0043 (Craft Adversarial Data - targeting complexity/length); T1190 (Exploit Public-Facing Application - targeting tokenizer buffer overflows).
*   **Exploitation & Installation:** AML.T0015 (Evade ML Model - bypassing alignment); T1499 (Endpoint Denial of Service - via computational exhaustion).
*   **Command and Control (C2):** N/A.
*   **Actions on Objectives:** AML.T0029 (Denial of ML Service); AML.T0031 (Erode ML Model Integrity).

## 3. Control Intersection and Segregation (The "Kill" Mechanism)

*   **Reconnaissance**
    *   *Preventive:* System prompts that restrict the model's operational domain, refusing to engage in topics outside its verifiable knowledge base.
    *   *Detective:* Monitoring for abnormally high variance in user inputs (e.g., submitting random character strings, recursive logic puzzles, or non-standard token encodings).
    *   *Corrective:* API rate limiting and connection termination for clients submitting anomalous token structures.
*   **Weaponization & Delivery**
    *   *Preventive:* Strict input validation at the API gateway, including hard caps on input length, character whitelisting, and pre-processing to filter known problematic tokens (e.g., `<|endoftext|>` or raw control characters).
    *   *Detective:* WAF alerting on malformed JSON or HTTP requests targeting the ML endpoint.
    *   *Corrective:* Dropping the malformed request before it is passed to the ML inference engine (e.g., Triton, vLLM).
*   **Exploitation & Installation**
    *   *Preventive:* "AI Red Teaming" prior to deployment to identify and patch alignment flaws. Implementing strict computational bounds (e.g., hard limits on maximum tokens generated, execution timeouts) to prevent GPU resource exhaustion.
    *   *Detective:* Real-time telemetry monitoring inference latency, GPU memory allocation spikes, and high-perplexity output generation (indicating the model is "confused" or hallucinating).
    *   *Corrective:* "Circuit breakers" in the inference engine that automatically kill a generation process if it exceeds a predefined time or computational threshold; automated failover to canned, deterministic error responses.
*   **Actions on Objectives**
    *   *Preventive:* Fact-checking guardrails (e.g., grounding model outputs in a verified RAG vector database and blocking outputs that cannot be cited).
    *   *Detective:* Output semantic monitoring detecting repetitive loops, toxic language, or fabricated URLs.
    *   *Corrective:* Redacting the flawed output and logging the prompt/response pair for subsequent fine-tuning and alignment correction.

## 4. Telemetry and Evidence Generation
To satisfy audit requirements, the following immutable logs must be generated and routed to a centralized, WORM-compliant SIEM:
*   **Pre-Deployment Evaluation Reports:** Immutable documentation of AI Red Teaming results, penetration test findings, and alignment benchmark scores (e.g., HELM, TruthfulQA) proving the model was tested for inherent flaws.
*   **Inference Resource Telemetry:** Logs capturing computational metrics per request, including processing time (latency), GPU VRAM consumption, and total tokens processed/generated.
*   **Inference Engine Error Logs:** Stack traces and crash reports from the ML serving infrastructure (e.g., out-of-memory errors caused by sponge examples).
*   **Guardrail Intervention Logs:** Records of instances where output filters or circuit breakers were triggered due to model hallucinations or runaway generation loops.

## 5. Auditor’s Perspective (CCCE Application)

*   **Criteria:** Organizations must not deploy non-deterministic AI models without first conducting rigorous, documented AI Red Teaming to identify inherent alignment flaws, and must enforce strict computational boundaries and output grounding mechanisms to prevent hallucination and resource exhaustion.
*   **Condition:** Management treats base ML models (e.g., Llama, Mistral) as secure, production-ready software components out-of-the-box. They deploy these models without independent alignment testing, fail to implement hard computational timeouts, and lack mechanisms to detect or correct factual hallucinations.
*   **Cause:** A fundamental misunderstanding of non-deterministic systems. First Line engineering teams apply traditional functional testing (Does it respond?) rather than adversarial alignment testing (How does it behave under duress or edge-cases?), prioritizing rapid deployment over robustness.
*   **Effect:** The organization's AI services are brittle and easily disrupted. Adversaries (or accidental edge-case users) can trigger model crashes causing Denial of Service, or exploit alignment flaws to generate highly damaging, fabricated outputs that carry the authoritative branding of the organization, leading to severe reputational and legal consequences.
*   **Recommendation:** Management must formalize AI Red Teaming as a mandatory stage gate in the SDLC. Inference environments must be configured with aggressive circuit breakers to prevent DoS, and all generative outputs must be subjected to real-time grounding verification (e.g., RAG citation checks) to mitigate the impact of inherent hallucinations.