Recall: random notes about the security model

Disclaimer: As everything on this blog is, this is my own opinion, and not my employer’s.

The conventional Unix security model is based around users, not applications. This implies that every application running in a given user context has access to all user data, even if that’s not related at all to what the app actually needs. And administrator users have access to all data within the system.

Today, when a hacker hacks an average Windows system to administrative privileges¹, they get access to browser history, its cache and other local data. What Recall does is add a searchable activity log that includes regular snapshots².

Is that a fundamental change or a change in degree? If the latter, then I’d argue that this isn’t such a big thing, as it fits within the well understood³ security model of Windows, which implies that access to an administrative account implies that you can see what’s going on across all of the system⁴. If the former, then Recall is crossing the Rubicon by collating all of this data and making it searchable across a single place.

What’s the biggest prior assumption that Recall breaks? Recall doesn’t redact anything⁵ on the screen from its dataset, except if the application or website⁶ is on a deny list. Unless specifically deleted by the application⁷, snapshots are kept. There’s no way⁸ for a website to signal that some data is too sensitive to keep by default⁹. This breaks the assumption that data on screen is only recorded when the user explicitly initiates a recording session or takes a screenshot. Special care has to be taken by applications¹⁰ to delete an activity when the underlying data is¹¹. However, there’s no current documentation¹² covering that aspect of the security model, and we’re around two weeks away from launch.

Does Recall allow an attacker with administrative access to have access to more info than they could collect otherwise? I’d argue yes. Browser caches expire, might not cover uncached pages, and are not deterministic in data collection the way Recall is. When data is deleted from the system¹³, Recall might still retain a copy unless proper integration is done to also revoke those snapshots from the Recall side.

Do other desktop operating systems have ways to make a folder inaccessible even if you are an administrator? macOS does, through System Integrity Protection. Rootless¹⁴ systems do not allow even an administrator to access to folders protected by SIP, even if running as root. SIP also restricts debugging of signed binaries except if those explicitly allow them to be debugged. This makes exfiltration of data from folders protected by SIP significantly harder¹⁵.

If I’m a business, should I care? Yes, you very likely have data retention and accuracy requirements¹⁶. Recall can make compliance with those significantly harder, especially as your line of business applications are probably not integrated with Recall APIs. Customer data might as a result linger on decentralised systems significantly longer than necessary, heightening business risks.

edit on June 2: Added the “What’s the biggest assumption that Recall breaks?” and “If I’m a business, why should I care?” sections.

1. I’d argue that a lot of Windows systems are single-user, with UAC being the only boundary to administrative access. UAC is not a security boundary is a saying that’s often pronounced in some circles, not without reason. ↩︎
2. Screenshots, enhanced by data provided by the application through activity APIs. ↩︎
3. At least I hope so. It’s not something new. ↩︎
4. Protected processes (incl. light) and VSM trustlets exist. The former are used for csrss since Windows 8.1 and DRM workloads notably, and the latter are used for Credential Guard and biometric data protection on some systems. However, I’m not aware of Windows components using those to protect bulk user data on a given machine. ↩︎
5. DRMed content goes through the Protected Media Path, and as such is not accessible to Recall. ↩︎
6. When using Microsoft Edge ↩︎
7. Activity APIs provide a way for an application to delete an activity or all activities associated to it, see https://learn.microsoft.com/en-us/uwp/api/windows.applicationmodel.useractivities.useractivitychannel.deleteactivityasync?view=winrt-22621 ↩︎
8. Hmm, the website can perhaps use a hardware-backed DRM solution to display 2FA recovery codes for example. But that’s not exactly the wisest idea. ↩︎
9. Users ideally must be able to override this if they want to. ↩︎
10. And utilities. Are you really sure that your print preview window is covered too? ↩︎
11. An activity can directly be associated with an underlying file location: https://learn.microsoft.com/en-us/uwp/api/windows.applicationmodel.useractivities.useractivity.contenturi?view=winrt-22621 ↩︎
12. The only developer docs about Recall right now are at https://learn.microsoft.com/en-us/windows/ai/apis/recall. For a feature with such an impact, this is very lacking. Application developers should have told much earlier and given enough time to adapt their applications. ↩︎
13. Or the server. ↩︎
14. Systems where SIP is enabled, which is the default. ↩︎
15. Historically, there were quite a few SIP bypasses. However, SIP is still generally effective. ↩︎
16. In plenty of jurisdictions, customers have the right to correct errors in business records. Does that include everything that Recall might have about a given customer? ↩︎