95% of AI Projects Didn't Fail Because of Tech
In 2025, MIT found 95% of enterprise AI pilots delivered no measurable P&L impact (MIT, "The GenAI Divide").
Open 6 AI-built landing pages and tell me which you remember.
Execution is free. It’s already creating thousands of products that look and feel the same, producing no EBITDA lift.
Many of the people ‘building’ in public today have no experience building software. They’ve never talked to a user, never iterated on feedback, and never scaled anything.
There's a term for this: AI Psychosis. You build something, AI tells you it's great, so you ship it. But no one outside of that echo chamber ever looked at it.
I’m not the only one noticing this.
Good idea ≠ Good product
AI is self-affirming. I’ve worked in tech over a decade and I experienced this firsthand with my vibe-coded project, AdvantageGuide.
AdvantageGuide.healthI had an objectively good idea: Medicare is difficult to navigate and Medicare Advantage members are leaving money on the table because they don’t use their benefits. So I built a website that walks members through what products they can get for free.
Claude told me the landing page looks great, the user flow is intuitive, and the copy is clear. Yet, it has no idea if a 67-year-old with arthritis can navigate the benefits quiz on a phone screen.
Claude optimizes for “does this look like a good product?” not “does this work for the person who uses it?” Until I got this in the hands of users I had no idea how much real building there was to do.
It’s easy to tell when someone has AI writing for them. The same can be said when AI is building for them. The first few iterations of AdvantageGuide looked like every other AI-created webpage: same hero, same gradient, same-sounding copy.
It’s technically functional, but visually identical. More output doesn’t mean more differentiation given the same models are building everything, it means the opposite.
Taste and curation are the real skills (and they always have been, we forgot because execution used to be hard).
The Real Skill
There's no advantage now because the tools are free and getting better every week.
Knowing what to keep, what to throw away, and what to expand on is where the edge lives. That's the thing that can’t be automated.
AI can’t feel a user’s pain before they tell you about it, but you can.
So what does it look like when someone actually gets this right? Pylon.
I used Pylon to build a support function from scratch. This was my first time building AI agents and I extended our need for headcount by ~9 months.
Pylon didn't bolt Slack support onto Zendesk. They asked what B2B support actually looks like in 2024, built the infrastructure around that answer, and designed the product so it's easy to adopt and customizable enough to feel native to any team.
Don’t be in the 95% that failThe reason that 95% of AI pilots had no impact on P&L is because they failed in taste, not technical capabilities. The product worked, just not for anyone in particular.
Ask yourself one question: could someone swap your product's UI with a competitor's and your users wouldn't notice?If the answer is yes, you don't have a product you have an output.
The fix isn't more features. It's more opinion, reps, and time in seat watching real users hit real friction.
Execution is free now, but that's not the advantage everyone thinks it is.
Don't be in that 95%.
