Distilling what a Red Team does can be difficult at times, because frequently people focus on the tactical findings: this vulnerability, that misconfiguration, some control that didn’t work or doesn’t exist. Those are all valuable and important aspects of the work, no doubt, and they’re the common artifacts of operations. But they’re not the essence.

I like Andy’s characterization of red teaming as a truth-finding function, and it’s very close to how I’ve been describing the work as well. Taking a contrarian position to a given claim (“Our data is secure.”) and then working to validate that hypothesis, to find the truth.

It seems intuitively true that it is not possible to secure a system with an approach that’s enmeshed with the input mechanism through which attacks also arrive; we’ve done much better using out-of-band controls to avoid commingling code and data. Still, it’s great to see this research land and confidently show this:

Role tags were a formatting trick that became the security architecture and the cognitive scaffolding of modern LLMs. We’ve shown that this architecture doesn’t survive into the model’s actual representations, and that such role confusion is linked to prompt injection.

Unless LLMs achieve genuine role perception, we think injection defense will remain a perpetual whack-a-mole game. And the continuous nature of role boundaries opens the threat of injections designed to subtly shift LLM states through seemingly innocuous text, legally and at scale.

The grifters and the hucksters and the influencers selling impossible things succeed because audiences reward certainty and punish doubt. They honor confidence and resist complication. A clean story about a genius who will fix everything travels faster than a difficult story about tradeoffs.

Skepticism is always healthy. Mostly everything has trade-offs, apparent or not, so you have to keep going until you can see their shape.

A nice bit of technical detail on how to achieve a privacy-preserving implementation for email push. I hope many people would care about such details, though realistically I’m unsure of it. It’s one of those things that can be done quickly, easily, and poorly; or with a little more effort and attention, quite well.

This is one of the more interesting ideas I’ve come across in terms of securing agentic workloads.