# THREAT DOMAIN TAXONOMY: CONFIDENTIALITY ATTACKS (DATA EXTRACTION, INVERSION, AND MODEL THEFT)

## 1. Standardized Phase Taxonomy (The Threat Lifecycle)
In the context of artificial intelligence and machine learning, "Confidentiality Attacks" do not primarily target traditional databases; they target the neural weights of the model itself. Machine learning models inherently memorize aspects of their training data. This threat domain encompasses Model Extraction (stealing the model's IP), Membership Inference (determining if specific data was in the training set), and Data Extraction/Inversion (forcing the model to regurgitate sensitive training data, such as PII or cryptographic secrets).

*   **Reconnaissance:** The adversary probes the inference API to determine the verbosity of the model's output. They actively look for exposed confidence scores, logit distributions, or token-generation behaviors that indicate how deeply the model relies on specific training data features.
*   **Weaponization & Delivery (Initial Access):** The adversary crafts targeted inference queries. For LLM data extraction, this involves "prefix queries" designed to trick the model into auto-completing memorized text. For model theft, this involves generating a massive matrix of synthetic inputs designed to comprehensively map the model's decision boundaries. Delivery is executed via standard, legitimate API access.
*   **Exploitation & Installation (Execution/Persistence):** Exploitation occurs dynamically during the inference compute cycle. By exploiting the model's tendency to overfit or memorize, the adversary forces the model to inadvertently decode and expose its internal representations or proprietary training data. There is no persistent installation; the vulnerability is an inherent mathematical property of the trained weights.
*   **Command and Control (C2):** Traditional C2 is replaced by high-volume API orchestration. The adversary utilizes automated scripts to send millions of queries, systematically harvesting the outputs.
*   **Actions on Objectives (Impact/Exfiltration):** The adversary aggregates the leaked outputs to reconstruct sensitive Personally Identifiable Information (PII), proprietary source code, or the mathematical parameters of the model itself, executing intellectual property theft or a massive privacy breach.

## 2. TTP Mapping (Tactics, Techniques, and Procedures)
For audit verifiability, this domain strictly maps to MITRE ATLAS (Adversarial Threat Landscape for AI Systems), as traditional ATT&CK matrices do not account for algorithmic memorization and inversion.

*   **Reconnaissance:** AML.T0040 (ML Model Inference API Access); AML.T0041 (Discover ML Model Ontology).
*   **Weaponization & Delivery:** AML.T0044 (Full ML Model Extraction); AML.T0054 (LLM Prompt Injection - used specifically for extraction prefixes).
*   **Exploitation & Installation:** AML.T0024 (Exfiltration via ML Inference API).
*   **Command and Control (C2):** N/A (Control is established via volume-based API interaction).
*   **Actions on Objectives:** AML.T0025 (Extract ML Model); AML.T0026 (Extract ML Model Data).

## 3. Control Intersection and Segregation (The "Kill" Mechanism)

*   **Reconnaissance**
    *   *Preventive:* API minimization—stripping raw logits, probabilities, and confidence scores from the user-facing output, returning only the final hard label or text.
    *   *Detective:* Heuristic monitoring for accounts systematically querying the API with edge-case or boundary-seeking inputs.
    *   *Corrective:* Throttling or tarpitting API responses for suspicious sessions to disrupt automated probing.
*   **Weaponization & Delivery**
    *   *Preventive:* Deduplication of the training dataset (models are significantly less likely to memorize data that only appears once).
    *   *Detective:* Rate limiting and volume-based alerting (e.g., detecting an IP address making 10,000 queries an hour).
    *   *Corrective:* Automated API gateway blocks triggered by velocity thresholds.
*   **Exploitation & Installation**
    *   *Preventive:* Training the model using Differential Privacy (DP-SGD) to mathematically guarantee that the inclusion of any single piece of data does not significantly alter the model's weights.
    *   *Detective:* Data Leak Prevention (DLP) engines scanning the *output* of the model in real-time for PII (e.g., regex matching for credit cards, SSNs, or proprietary operational keywords).
    *   *Corrective:* Output redaction; replacing extracted sensitive strings with masked characters before transmitting the HTTP response to the user.
*   **Actions on Objectives**
    *   *Preventive:* Strict commercial API quotas and usage caps linked to verified identities.
    *   *Detective:* UEBA (User and Entity Behavior Analytics) detecting anomalous data egress volumes mapped to specific user tokens.
    *   *Corrective:* Immediate cryptographic revocation of the offending user's API keys and suspension of the account.

## 4. Telemetry and Evidence Generation
To satisfy audit requirements, the following immutable logs must be generated and routed to a centralized, WORM-compliant SIEM:
*   **Training Parameter Logs:** Cryptographic proof of data deduplication and logs detailing the privacy budget (epsilon and delta values) if Differential Privacy was utilized during training.
*   **Inference I/O Telemetry:** Complete logging of prompt inputs and model outputs, essential for detecting data extraction attempts retrospectively.
*   **API Gateway Metrics:** Granular logs of API consumption rates, token utilization, and HTTP response sizes to detect extraction scraping.
*   **DLP Output Alerts:** Immutable records of every instance where the output guardrail redacted sensitive information generated by the model.

## 5. Auditor’s Perspective (CCCE Application)

*   **Criteria:** Organizations training and deploying AI models must protect the confidentiality of the training corpus through strict dataset deduplication, Differential Privacy methodologies, and robust Data Leak Prevention (DLP) filtering on all model outputs.
*   **Condition:** Management systematically operates under the dangerous fallacy that compiling sensitive data into neural network weights inherently "anonymizes" or destroys the original data. Consequently, models are exposed via rich APIs without output DLP or differential privacy controls.
*   **Cause:** A fundamental lack of understanding by the First Line (Engineering) and Second Line (Risk) regarding algorithmic memorization. Operational focus is exclusively placed on model accuracy and developer experience (providing rich API outputs), directly at the expense of data privacy.
*   **Effect:** The organization exposes highly sensitive customer data, cryptographic operational details, or proprietary intellectual property to trivial extraction by any user with API access, resulting in catastrophic regulatory breaches (e.g., GDPR violations) and total loss of intellectual property.
*   **Recommendation:** Management must immediately cease exposing raw probabilities/logits via public APIs, implement real-time DLP scanning on all inference outputs, and mandate that all future model training incorporates strict data deduplication and privacy-preserving machine learning (PPML) techniques.