Microsoft Recall: anatomy of a launch without threat modeling

On 20 May 2024, at the Copilot+ PCs launch event in Redmond, Microsoft announces Windows Recall. The idea: the OS captures a screenshot of the desktop every few seconds, runs OCR and embeddings through a local model, and leaves the lot indexed for semantic search. “Remind me of that pasta recipe I saw last week”, “open the document where the Q3 number appeared”. Sold as the PC’s photographic memory.

Two weeks later, Kevin Beaumont publishes his analysis on DoublePulsar. Recall’s database is a flat SQLite in the user profile, no DPAPI, no process-level protection. Any process with the user’s permissions reads the whole visual history of the machine. Alex Hagenah drops a tool (TotalRecall) that automates the extraction in seconds. James Forshaw (Project Zero) confirms it doesn’t even require elevation. On 7 June Microsoft pulls back: Recall becomes opt-in, Windows Hello is required, the database is encrypted, and Insider availability is delayed.

This post isn’t a PoC on a lab we built (Recall lived only on Copilot+ PCs and the Beaumont/Hagenah base is documented in their repos). What we do here is threat modeling on a product that was announced ready for production and turned out to be an exfiltration dataset waiting to be read.

The product

What Microsoft showed on 20 May:

• Periodic desktop capture. Snapshots every few seconds (Microsoft didn’t give an exact figure in the announcement; later analysis estimated ~5s) while the user works.
• OCR over every capture. Text visible on screen (emails, Signal/WhatsApp Web messages, code in VS Code, password manager entries visible during copy, card numbers on checkout) is extracted and indexed.
• Embeddings + semantic search. The user can query in natural language “the article about quantum computing I read” and Recall jumps to the timestamp where it appeared.
• Local processing. Microsoft insists nothing leaves the device. True: the data isn’t sent to the cloud, it lives in %localappdata%. The problem is exactly where it lives and how it lives.

What the announcement didn’t say, and what shows up in the late-May technical analysis: where it’s stored, in what format, and under what guarantees.

Threat modeling — the exercise that was missing

Three actors any decent threat modeling would have enumerated on the whiteboard before the first commit:

1. Malware with user permissions. The most common scenario. Infostealer (RedLine, Vidar, Lumma, the ones you buy for $200/month on forums) running in the user’s context. Doesn’t need admin, doesn’t trigger UAC.
2. Insider with brief physical access. Someone with 30 seconds in front of an unlocked laptop: a hostile office colleague, hotel staff, a partner in the middle of a divorce. They copy %localappdata%\CoreAIPlatform.00\ to a USB. Read it at home later.
3. Laptop theft / loss. The whole industry’s threat model since 2010. Not aggravated here because BitLocker is on… but only while the attacker doesn’t have the password.

For each actor, two questions. What do they need to read the user’s full visual history? What would it cost them?

What Beaumont and Hagenah show by the end of May: in all three cases, the cost is trivial. And Microsoft hadn’t put any barriers up.

Where the data lives

What Hagenah documents in TotalRecall (public GitHub repo, 4 June 2024):

C:\Users\<USER>\AppData\Local\CoreAIPlatform.00\UKP\{GUID}\ukg.db

ukg.db is a standard SQLite. No DPAPI, no SQLCipher, no EFS, nothing. The relevant tables the README enumerates:

• WindowCaptureTextIndex_content: OCR text extracted from each capture, in plaintext.
• App: which binary had focus at each moment.
• Window: window title at the time of capture.

On top of that, the same directory stores the capture PNGs in ImageStore/. Unencrypted.

The extraction flow TotalRecall implements fits in four conceptual lines:

# (Conceptual — over an extracted copy of the DB)
sqlite3 ukg.db ".tables"
# WindowCaptureTextIndex_content  App  Window  WindowCapture ...

sqlite3 ukg.db "SELECT c1 FROM WindowCaptureTextIndex_content LIMIT 10;"
# (returns OCR text — messages, code, visible passwords, whatever)

TotalRecall’s first release assumed admin to bypass the directory ACLs. James Forshaw posts on X (short version) that the assumption is unnecessary: the token of the user owning the profile already has the permissions needed; you just have to load the right context. His exact wording: “Turns out I was wrong about Recall being secure.” TotalRecall is updated within 48 hours following that note and stops asking for admin.

Net result: a process running as the current user reads the full visual history of the machine in less than a second. No UAC, no consent dialog, no audit log. Recall is transparent to the process.

Broken assumptions

Recall’s design assumes things that aren’t true on Windows desktop:

• Assumption 1: “if it lives under the user profile, it’s protected by the user profile’s guarantees”. This is the hand-wave version of the model. On Windows, any code the user runs has the same privileges as the user. Recall turns a standard infection (a malicious PDF, a compromised browser extension, an NPM installer with postinstall) into a complete exfil of the visual history with zero additional cost. DPAPI exists precisely to put a barrier between “the user ran me” and “the user wants to give me their master credential”. Recall didn’t use it.

• Assumption 2: “local data is safe because it doesn’t leave the device”. This is the favourite line of on-device AI press releases. The problem is that what’s local is exactly what the user’s malware reads first. The modern threat isn’t a sniffer on the wire; it’s an infostealer walking %appdata% and %localappdata% for 15 seconds before deleting itself. The data concentration (the whole history in an indexed, searchable SQLite) makes the local store a better crown jewel than any remote vault.

• Assumption 3: “BitLocker handles at-rest”. It handles theft of a powered-off laptop. It handles nothing when the user is sitting in front, nor when a process in the user’s context is reading. BitLocker is orthogonal to this threat model.

The serious bit isn’t that the design had these gaps: many early designs do. The serious bit is that it was announced as ready to ship with a release date (18 June on the first Copilot+ PCs) without a public threat model covering these three actors. The question a reasonable security reviewer asks in sprint review (“what happens if a normal infostealer runs as the user”) produces, in this design, the worst possible result.

The reversal

On 7 June, Pavan Davuluri (President, Windows + Devices) publishes the changes on the Windows Experience Blog:

• Opt-in by default. Recall is no longer installed enabled on Copilot+ PCs. It has to be accepted explicitly during setup.
• Windows Hello required to enable Recall. Without biometrics / PIN configured, no Recall.
• Proof of presence to query the timeline. Being signed in isn’t enough — you have to reauthenticate with Hello for snapshots to decrypt.
• Just-in-time encryption with Enhanced Sign-in Security (ESS). Captures decrypt only during the user’s authenticated session. Search index database encrypted.
• Launch delay. What was going to ship on 18 June moves to Insiders. Public availability is pushed back again in August (October) and again in October (December).

What the list teaches: the three broken assumptions each have a matching patch. Missing DPAPI/ESS → encryption. Missing barrier between “user is logged in” and “data is accessible” → Hello proof-of-presence. Missing opt-in to neutralise the “my mum buys a Copilot+ PC and doesn’t know Recall exists” scenario → opt-in.

None of those changes are hard. All are obvious to anyone who has written a threat model during an internal review. The problem wasn’t technical capability; it was process.

Operational lessons — for product, not for press release

No lecture. Three readings useful for the next AI product up for review:

1. On-device isn’t secure-by-default. The “it’s processed locally” argument is about privacy versus the cloud, not security versus local malware. Any feature that creates a structured dataset of user behaviour on disk is a crown jewel for infostealers. If it’s going to live local, it lives encrypted at-rest with a key derived from a user credential (DPAPI / ESS / TPM-protected keys + Hello), and only decrypts during authenticated use.

2. Aggressive capture demands aggressive threat modeling. Recall isn’t Windows Search expanded: it’s a new component that records the screen. The delta in capability versus baseline requires a proper threat model (actors, capabilities, explicit assumptions, mitigations). Publishing a blog with “it’s encrypted and lives locally” isn’t threat modeling; it’s marketing.

3. Vendor responsible disclosure is also disclosure. Microsoft could have published the threat model alongside the announcement. It didn’t. The threat model was written by Beaumont and Forshaw after the fact, with worse PR for Microsoft and worse protection for early users. The asymmetry (vendor knows how it works, defenders have to reverse-engineer the binary to find out) is a product decision. It can be broken by publishing detailed architecture on announcement day.

Why this case matters for 2024

Recall is exhibit A for a pattern that’s going to repeat through the rest of the year: AI feature with new capability (mass capture, agentic action, computer use), launched with an implicit threat model of “trust me, it’s secure”, withdrawn or restricted after two weeks of external analysis. We’ll see the same arc in Computer Use (October, indirect injection via screen — covered in its technical), in MCP (November, confused deputy at protocol level), and in o1 (September, CoT as a new attack surface).

What changes in Recall versus the others: the bug isn’t AI-specific, it’s classic Windows desktop threat modeling (sensitive data concentration in the user profile without DPAPI). The AI part justifies the why of the dataset; the broken part is from the 90s.

As a CISO fielding the question “should we roll out Recall to the fleet?” in July 2024: the short answer is “wait for the technical post-mortem of Microsoft’s changes and for CISA / SANS guidance”. The long answer is “design the policy before the rollout — which machine classes don’t get Recall (workstations with secrets: finance, legal, dev), what retention you apply (you don’t need 3 months), and which SIEM event covers creation / access to the CoreAIPlatform.00\ directory”.

References

• Microsoft, Introducing Copilot+ PCs (Yusuf Mehdi, 20 May 2024): https://blogs.microsoft.com/blog/2024/05/20/introducing-copilot-pcs/
• Kevin Beaumont, Stealing everything you’ve ever typed or viewed on your own Windows PC is now possible with two lines of code — inside the Copilot+ Recall disaster (DoublePulsar, May 2024): https://doublepulsar.com/recall-stealing-everything-youve-ever-typed-or-viewed-on-your-own-windows-pc-is-now-possible-da3e12e9465e
• Kevin Beaumont, Microsoft Recall on Copilot+ PC: testing the security and privacy implications (DoublePulsar): https://doublepulsar.com/microsoft-recall-on-copilot-pc-testing-the-security-and-privacy-implications-ddb296093b6c
• Alex Hagenah (xaitax), TotalRecall — automated Recall extraction (GitHub, June 2024): https://github.com/xaitax/TotalRecall
• James Forshaw, threads on X / Mastodon about the admin bypass: https://infosec.exchange/@tiraniddo
• Pavan Davuluri (Microsoft), Update on the Recall preview feature for Copilot+ PCs (7 June 2024): https://blogs.windows.com/windowsexperience/2024/06/07/update-on-the-recall-preview-feature-for-copilot-pcs/