LLM Risk Scenarios

Risk as Cause, Event, Consequence

Each LLM risk scenario is expressed as a relationship: an (Event) caused by (Cause) resulting in (Consequence). This structure matters because LLM-related harm appears indirectly. A malicious prompt is not only an input manipulation event; it may enable disclosure, corrupt a workflow, degrade decision quality, or trigger unauthorized actions through tools or connected systems. Hallucinated content is not merely a quality issue — once embedded in reports, policies, customer communications, or employee decisions, it becomes an operational and governance problem.

Construction rules

• Events are drawn from the eight risk domains enumerated below.
• Causes are grouped into bounded rationality, opportunism, asset specificity, and uncertainty, following the Market Failure Framework (Williamson; Coase). The single most proximate cause is chosen per scenario.
• Consequences are expressed in imperative terms of health, safety, or fundamental rights (per Article 9(a) of the AI Act), choosing the "worst case" direct consequence for the scenario.

Eight top-level risk domains

The first six domains follow an AI Act imperative to address AI-specific attacks. Domains 7 and 8 — prompt injection and shadow AI — receive special treatment because each, on independent grounds, defines a structurally distinct category of organizational exposure.

1. Data poisoning

Manipulation of the training data set such that the model learns adversary-controlled patterns. Becomes operationally significant in Fine-Tuned archetypes where the organization owns the dataset, and remains a supply-chain concern elsewhere when third-party training data is reused.

2. Model poisoning

Compromise of pre-trained components or weights used in training or inference. Most acute under Fine-Tuned and Application archetypes when checkpoints are downloaded from open repositories.

3. Adversarial examples

Inputs crafted (often mathematically) to cause the model to make a confident but wrong decision. Particularly relevant for decision-support and classification deployments where downstream actions depend on model outputs.

4. Model evasion

Inputs designed to bypass safety filters, content policies, or detection logic. Distinct from adversarial examples in that the goal is escape rather than misclassification.

5. Confidentiality attacks

Extraction of training data, system prompts, or contextual information through inference-time queries. Includes membership inference, model inversion, and prompt-leak attacks.

6. Model flaws

Failures arising from the model itself rather than from attack — hallucinations, miscalibration, drift, and unsupported generalizations. Article 15(5) of the AI Act treats these as in-scope for security analysis.

7. Prompt injection — special treatment

Prompt injection receives separate treatment because recent work shows it is not a marginal edge case but a structurally important inference-time risk. Liu et al. formalized and benchmarked prompt injection attacks across multiple models and tasks; Geng et al., synthesizing 128 studies, argue that the core weakness lies in the model's inability to cleanly distinguish instructions from data — making prompt injection a persistent control problem for LLM-enabled systems.

In methodological terms, that finding means the taxonomy must preserve a separate class for prompt-mediated control failure rather than absorbing it into generic application misuse.

8. Shadow AI — special treatment

Shadow AI is included as a distinct organizational-use domain because it operates at a different control layer than the technical risks above. Waters-Lynch et al. define shadow user innovation as employee-led AI experimentation that remains opaque to management and therefore obstructs organizational learning. Silic et al. extend this into the concept of shadow AI as the unsanctioned use of AI systems outside approved governance frameworks, producing a governance drift zone in which formal policies exist but fail to gain traction in practice.

Because the study evaluates organizational control sufficiency, these findings are central to why visibility, accountability, and escalation appear as explicit analytical dimensions in the gap typology.