AI (in)Security

It’s been five years since my title formally contained the term “security.” It’s not that I’ve put it down, but rather that I’ve put the bulk of my energy toward AI and product engineering. It’s been a nice detour, yet as I look towards the horizon of the current wave of technology we’re all surfing I recognize it’s time to put my security hat back on.

Quick aside - readers who want to know more about my qualifications in the space can check out my personal page on the Blink Build site, my GitHub, and my LinkedIn.

tl;dr, I've...

• Been hacking on (and into) things since I was 12 (thanks mom and dad!)
• Spent years as a penetration tester, most notably as one of Kevin Mitnick’s go-to hackers for hire
• Founded the Georgia Tech student hacking organization
• Led the internal hacking team at Snapchat
• Led the application security team at Bird Rides
• Headed up Security Engineering for Clubhouse
• Have ~30 CVEs to my name
• Have given talks around the world on hacking techniques, most notably key-noting DEF CON China 1

And perhaps my greatest qualification - I have a glider tattoo.

I liked working in security. I started on the breaking side and then graduated to the (harder) work of building. Over the course of a decade I saw deeply meaningful improvements to the general state of information security across the board:

• BeyondCorp patterns took root, pushing access control issues to the top layer of the OSI model (where they belong).
• Cloudflare provided a 1,000 pound gorilla of security in front of vast swathes of American technology infrastructure for free.
• Duo and other multi-factor authentication tooling mitigated the impact of credential compromise.
• 1Password and other password vaults limited the re-usability of compromised credentials.
• Okta and Google single sign on grew like wildfire and created a choke point for the application of strong security controls.
• Higher tech companies started buying Apple laptops in droves, shrinking the blast radius that outdated Microsoft protocol compromise could result in.
• Security education and its associated tooling grew substantially.
• SOC 2 compliance was born into existence, aligning business incentives with security teams’ desires (younger me would be aghast that I am tipping my hat to compliance).
• Platforms like HackerOne and BugCrowd crowd-sourced security research for companies that were concerned about their security but didn’t have the in-house expertise.

...and this isn't anywhere close to an exhaustive list!

We were converging on a model where data stayed in defined channels, access was controlled, and systems were observable. With the notable exception of application security, most security problems could be addressed via a mix of smart planning and vendor solutions.

Then, the wave of AI began. It wasn’t clearly a massive security problem at first - more a fun chat technology to put to use. The Turing test was looking shaky.

Then, the wave of AI grew. All of a sudden our IDEs started suggesting larger and more complicated chunks of working code. It went from auto-complete to something new and never before seen.

Then, the wave of AI became a tsunami. Non-engineers vibe coding full solutions and (rightfully) freaking out when their software services were compromised. Agentic development churning out directories full of (semi) working code.

Now, we are adrift.

There is a noise floor that rises every day as more voices, many of which are thinly-veiled marketing mouthpieces for AI platforms, join the fray. It is difficult to distinguish signal from noise when so much money is being spent to trick you into mistaking noise for signal. Even so, it is clear that we have passed an inflection point in AI capabilities and adoption. The velocity only picks up from here.

Why does this matter for security?

I have long said that the art of hacking relies upon discovering the delta between what a piece of software is intended to do and what the piece of software can actually do.

As an example - let’s say you built a web site that contains functionality for downloading any number of PDF files by name that sit in a specific directory. The name of the file is specified as a query string parameter, the software looks for the file with that name on disk and, if found, returns the file to the user’s browser. A nefarious user then specifies their own custom file name like “../../../etc/passwd” and the software, upon validating that the file exists, returns a sensitive Linux operating system file.

The software was intended to allow users to download one of a set of predefined PDF files. The software allowed users to download any file on the filesystem. These sorts of gaps, chained together, result in attack chains with innumerable consequences. Whenever you hear about some security breach in the news, you can rest assured that a complicated compilation of gaps like this were involved.

Take this observation, and pair it with the fact that one of the premier frontier model firms (Anthropic) is shipping so much code so quickly that it accidentally released the source code for its premier product (Claude Code). This mistake happened at a firm that knows the technology best, is staffed by some of the brightest, and is deeply financially incentivized to prevent this sort of mess up from happening.

If Anthropic can’t manage to solve this problem, how will the rest of American technology companies with their far thinner resources fare? The barrier to writing software has collapsed, the volume of code has exploded, and we no longer reliably understand what we’re shipping.

On the offensive side of the coin Anthropic just launched Project Glasswing, which is an effort to work with infrastructure providers to get ahead of the problems that advancing models in the hands of malicious hackers will create. Anthropic claims that their next generation model has found thousands of zero day vulnerabilities in critical Internet infrastructure. I have no reason to doubt it, and the AI model responsible for this initiative will be in the hands of the general public in the coming months. Meanwhile, core infrastructure vulnerabilities can take months if not quarters if not years to fully patch.

Layer in the facts that…

• All your enterprise secrets and IP are being copy-pasted / uploaded to who knows how many AI platforms
• All your meetings are being surreptitiously recorded by at least one participant via Granola
• All your anti-scraping and anti-bot technologies are falling flat against agent-in-a-browser automation

Wild times!

The security industry built nice little channels and pipes for data to securely flow through, and AI proved our construction materials to be deeply porous.

As is so often the case in security, we are now playing catch up. The good news is that we can also put this new wave of technology to use in solving these problems! The bad news is we’re just now starting to figure out what that should look like.

This is the problem I am going to work on, and I’m excited to be jumping back into the fray.