# THREAT DOMAIN TAXONOMY: INPUTS DESIGNED TO CAUSE MISTAKES (ADVERSARIAL EXAMPLES / EVASION ATTACKS)

## 1. Standardized Phase Taxonomy (The Threat Lifecycle)
Unlike data poisoning, which targets the training phase, adversarial examples (evasion attacks) target the machine learning model entirely during the *inference* (production) phase. The adversary introduces carefully calculated perturbations to the input data—often imperceptible to humans—that force the model to cross a decision boundary and produce a specific, incorrect output.

*   **Reconnaissance:** The adversary interacts with the target model's public or internal API to map its decision boundaries. This involves sending numerous variations of inputs and analyzing the outputs, specifically targeting confidence scores or probability distributions (e.g., via shadow modeling or gradient estimation).
*   **Weaponization & Delivery (Initial Access):** Utilizing mathematical optimization techniques (e.g., Fast Gradient Sign Method [FGSM], Projected Gradient Descent [PGD]), the adversary crafts the adversarial payload. This payload (a perturbed image, manipulated audio file, or specifically structured text prompt) is delivered to the target system via standard, legitimate input channels (e.g., a file upload portal, a chat interface, or a sensor feed).
*   **Exploitation & Installation (Execution/Persistence):** The target ML model processes the adversarial input. Because the mathematical perturbation exploits inherent blind spots in the model's neural weights, the model is successfully manipulated into misclassifying the input or hallucinating a response. There is typically no persistent "installation" in this chain; the exploitation is instantaneous and ephemeral, occurring solely within the active inference compute cycle.
*   **Command and Control (C2):** Traditional network C2 is irrelevant in this threat domain. "Control" is exerted dynamically through the continuous or automated submission of the adversarial payloads, dictating the model's output on a per-transaction basis.
*   **Actions on Objectives (Impact/Exfiltration):** The adversary achieves their operational goal by bypassing the AI-driven business logic. Manifestations include evading biometric authentication, bypassing AI-driven fraud detection, corrupting automated content moderation, or forcing autonomous systems into unsafe states.

## 2. TTP Mapping (Tactics, Techniques, and Procedures)
For audit verifiability, this domain is mapped primarily against the MITRE ATLAS (Adversarial Threat Landscape for AI Systems) framework, as traditional ATT&CK matrices do not adequately cover inference-phase algorithmic manipulation.

*   **Reconnaissance:** AML.T0040 (ML Model Inference API Access); AML.T0041 (Discover ML Model Ontology); AML.T0038 (Discover ML Model Architecture).
*   **Weaponization & Delivery:** AML.T0043 (Craft Adversarial Data); AML.T0015 (Evade ML Model).
*   **Exploitation & Installation:** AML.T0016 (Obtain Capabilities - utilizing adversarial ML toolkits like CleverHans or Foolbox for execution).
*   **Command and Control (C2):** Not conventionally applicable; control is subsumed within the delivery of the crafted input.
*   **Actions on Objectives:** AML.T0031 (Erode ML Model Integrity); T1562 (Impair Defenses - specifically evading AI-based security controls).

## 3. Control Intersection and Segregation (The "Kill" Mechanism)

*   **Reconnaissance**
    *   *Preventive:* Masking or rounding confidence scores returned by the API to prevent gradient estimation; implementing strict API rate limiting.
    *   *Detective:* Heuristic alerting on high-frequency, low-variance API queries originating from a single identity or IP space.
    *   *Corrective:* Automated IP blocking or temporary API token revocation upon detection of probing behavior.
*   **Weaponization & Delivery**
    *   *Preventive:* "Feature Squeezing" or input sanitization (e.g., degrading image resolution, removing inaudible audio frequencies, sanitizing text inputs) to destroy the fragile mathematical perturbations before they reach the model.
    *   *Detective:* Out-of-Distribution (OOD) detection models deployed as a parallel screening layer to identify inputs that statistically deviate from the model's training baseline.
    *   *Corrective:* Rejection of the input payload at the API gateway with a generic error message.
*   **Exploitation & Installation**
    *   *Preventive:* "Adversarial Training" (the process of proactively generating adversarial examples and including them in the training dataset to harden the model's decision boundaries).
    *   *Detective:* Monitoring inference latency (adversarial examples sometimes require longer processing paths in complex neural networks) and tracking extreme fluctuations in internal layer activations.
    *   *Corrective:* Human-in-the-Loop (HITL) fallback triggered when the model produces an output with uncharacteristically low or anomalous confidence metrics.
*   **Actions on Objectives**
    *   *Preventive:* Decoupling the AI output from direct, unverified execution of critical business logic.
    *   *Detective:* Business logic monitoring (e.g., tracking an anomalous spike in approved transactions that normally fail).
    *   *Corrective:* Automated suspension of the affected automated pipeline until manual review is completed.

## 4. Telemetry and Evidence Generation
To satisfy audit requirements, the following immutable logs must be generated and routed to a centralized, WORM-compliant SIEM:
*   **Inference API Gateway Logs:** Comprehensive capture of connection metadata, request rates, and returned confidence scores.
*   **Input Payload Capture:** Hashes and statistically sampled raw inputs of user requests to facilitate post-incident adversarial analysis.
*   **OOD Detection Telemetry:** Logs from auxiliary monitoring models detailing why specific inputs were flagged as anomalous prior to primary inference.
*   **Model Performance Baselines:** Continuous tracking metrics of inference accuracy and error rates to detect systemic evasion campaigns over time.

## 5. Auditor’s Perspective (CCCE Application)

*   **Criteria:** AI systems exposed to untrusted user input must be mathematically hardened via Adversarial Training and protected by rigorous input sanitization (Feature Squeezing) and Out-of-Distribution (OOD) detection, ensuring the model does not fail open when presented with engineered perturbations.
*   **Condition:** Management frequently treats ML models as standard software endpoints, implicitly trusting all input payloads that pass basic format validation (e.g., verifying a file is a valid `.jpg`) while completely ignoring the semantic and mathematical integrity of the data stream.
*   **Cause:** A failure of First Line engineering to conduct AI-specific adversarial threat modeling (Red Teaming) during the SDLC, coupled with an over-reliance on traditional Web Application Firewalls (WAFs) which are entirely blind to algorithmic evasion attacks.
*   **Effect:** Adversaries can reliably and repeatedly bypass AI-driven business logic, fraud detection mechanisms, or safety guardrails using simple, automated toolkits, resulting in unmitigated operational or financial loss without triggering traditional security alarms.
*   **Recommendation:** Management must implement robust Adversarial Training protocols during the model development phase, enforce strict input sanitization at the API gateway, and suppress the exposure of raw confidence scores to external users to prevent active reconnaissance and model extraction.