ai-security

ai-security · 14 min
MCP tool poisoning: four months after the spec, the real-world attacks
In November 2024 Anthropic published MCP and the analysis was at spec level — what the protocol said and what it left to the implementer. In April 2025, Invariant Labs publishes the first paper on Tool Poisoning Attacks: MCP servers hiding adversarial instructions in tool descriptions. Cursor, Claude Desktop and Copilot read those descriptions as prompt and obey. Reproducible PoC with the Python SDK.
5 abr 2025 · Manuel López Pérez
ai-security · 39 min
AI Security 2024 — annual dossier
Twelve months across ten axes. 2024 is the year AI infrastructure emerged as a category with its own CVEs, agents moved from the lab to product (Claude Computer Use, MCP, Salesforce Agentforce), regulation became applicable (EU AI Act in force 1 August, NIS2 deadline 17 October, NIST AI 600-1), and jailbreaks professionalised with reproducible metrics (ArtPrompt, Many-shot, Skeleton Key). Underneath, Recall shipped without threat modeling and got pulled, Arup lost $25M on a deepfake video call, and the pre-positioning chain of incidents (Volt Typhoon, Salt Typhoon, Storm-0558 fallout) runs through the whole year. Canonical annual reference.
15 feb 2025 · Manuel López Pérez
ai-security · 13 min
DeepSeek-R1: the first reasoning model with open CoT and what changes for AI security
On 20 January DeepSeek ships R1 with paper, repo and Hugging Face weights under MIT. It is the first time a reasoning model with RL-trained chain-of-thought is available as open weights. The CoT between <think></think> tags is plain text — inspectable, and attackable.
25 ene 2025 · Manuel López Pérez
ai-security · 10 min
AI security 2024 in review: five patterns that stick
Not a ranking, not a listicle. Five patterns from the year with analysis and cross-links to the monthly technicals: long context as attack surface, agents in production, jailbreaks by optimisation, launches without threat modelling, reasoning models as new surface.
30 dic 2024 · Manuel López Pérez
ai-security · 13 min
Confused deputy revisited: Model Context Protocol and the protocol-level version of the bug
Anthropic publishes MCP on 25 November. The model-to-external-tools link becomes an open spec with three primitives: tools, resources, prompts. The spec says the host SHOULD ask for consent; it concedes the protocol cannot enforce it. The confused deputy pattern we documented in September 2023 is back — now as a standard integration.
28 nov 2024 · Manuel López Pérez
ai-security · 10 min
Claude Computer Use: the agent that moves the mouse and the page that tells it where to click
On 22 October, Anthropic ships Claude 3.5 Sonnet (new) with OS-control capability via screenshots. Two days later, Johann Rehberger publishes the first public PoC: a web page indirect-injects the agent into downloading and running a Sliver implant. Confused deputy at the operating-system level.
26 oct 2024 · Manuel López Pérez
ai-security · 14 min
o1-preview: jailbreaks against a model that thinks where nobody can see
On 12 September, OpenAI ships o1-preview with an RL-trained chain of thought hidden from the user. By the next day there are screenshots showing how to inject instructions into that chain. What changes when an intermediate channel appears that the attacker can't see and the defender doesn't log.
20 sept 2024 · Manuel López Pérez
ai-security · 9 min
Microsoft Recall: anatomy of a launch without threat modeling
Recall captures screenshots every N seconds, OCR + embeddings, and indexes the lot into a local SQLite. Beaumont and Forshaw show in two weeks that any malware running with user permissions reads the visual history of the PC. Microsoft pulls back on 7 June.
30 may 2024 · Manuel López Pérez
ai-security · 7 min
Many-shot jailbreaking: when the context window becomes attack surface
On 2 April, Anthropic publishes a technique that fills the context with hundreds of harmful question/answer pairs before the real prompt. In-context learning does the rest. It scales as a power law into the hundreds of shots.
10 abr 2024 · Manuel López Pérez
ai-security · 9 min
ArtPrompt: ASCII-art jailbreaks and the classifier-model gap
On 15 February, Jiang et al. publish a paper that breaks alignment in GPT-3.5/4, Claude, Gemini and Llama-2 by writing the forbidden word as ASCII art. The classifier sees a harmless cloze; the model reads it and answers.
20 feb 2024 · Manuel López Pérez
ai-security · 30 min
AI Security 2023 — annual dossier
Twelve months across ten axes. 2023 is the year AI security moves from academic discussion to a discipline with its own vocabulary, canonical papers, industry frameworks and the first regulatory apparatus. ChatGPT crosses 100M MAU in January; GPT-4 ships in March; Greshake, Zou+Carlini and OWASP set the terminology; NIST AI RMF, Biden EO 14110 and the political deal on the EU AI Act define the apparatus. The annual reference for the founding year.
15 feb 2024 · Manuel López Pérez
ai-security · 6 min
Sleeper agents: when the attack lives inside the model
Anthropic foreshadows during Q4 a new class of attack: models trained with a hidden trigger that pass safety tests but run adversarial behaviour upon seeing the trigger in production. The paper drops January 2024; the implication lands now.
30 nov 2023 · Manuel López Pérez

Newer posts

Older posts