Why Most AI Projects Fail (And What to Ship Instead in 2026)

I get a version of this call every week.

A founder — sometimes technical, usually not — has spent three to nine months and somewhere between $40k and $400k on an AI feature. The model works in the demo. It falls apart the moment real users touch it. The team is burned out. The board is asking questions. They want to know if they should start over, ship what they have, or kill it.

I've shipped AI into a dozen real products at this point — RAG systems, chatbots, copilots, automated workflows, content tools, customer-support agents, internal search. I've watched the same five failure modes kill the same kinds of projects over and over. None of them are about the model.

This is the playbook I use to figure out what to actually build before I build it — and the kill-criteria I set up so the project doesn't become another cautionary tale.

The five ways AI projects die#

1. The demo is the product#

The most common failure mode by a wide margin. Someone builds a clever prompt, demos it on three cherry-picked examples, and the team confuses the demo with the product.

Real users do not send cherry-picked examples. They send:

• Empty inputs.
• Inputs in languages the prompt was never tested in.
• Inputs that look correct but are semantically nonsense.
• Inputs that contradict the system prompt.
• Adversarial inputs designed to break the system.
• Inputs that are technically valid but socially inappropriate.

If your team has never watched a real user use the product — not a beta tester you recruited, not your co-founder, but a stranger who found the product through SEO — the demo is the product. The actual product is undefined.

Kill criterion: if the team can't articulate what the model should do on at least 50 real-world inputs before the user touches it, the project isn't ready to ship.

2. No success metric, only a model metric#

I see this constantly. The team tracks token usage, latency, hallucination rate on a synthetic eval set. None of those are success metrics. They are proxy metrics.

A success metric answers: did the user get value? Examples:

• Support deflection: did the AI resolve the ticket without a human?
• Conversion: did the AI-assisted flow produce more conversions than the baseline?
• Time-to-answer: did users complete the task faster?
• Retention: did the cohort that used the AI feature retain better than the cohort that didn't?

If you can only describe your project in terms of the model and not in terms of the user, the model is not the product.

Kill criterion: if the founder can't name the user-facing success metric in one sentence, the project is research, not a product. Fund research as research.

3. Scope creep dressed up as "the AI can do that"#

This one kills projects slowly. The team gets excited about what the model can do. The roadmap expands. Six months in, you've built a general-purpose assistant when the original ask was "summarize a contract."

Every "the AI can do that" expansion should be answered with three questions:

1. What's the smallest evaluation set that proves the AI does this thing reliably?
2. What does the failure mode look like for this specific task?
3. What does it cost to fix when the AI breaks this specific task?

If the team can't answer those three for a proposed expansion, the expansion is not ready. Add it to the backlog. Don't ship it.

Kill criterion: if the roadmap has more than three "core" use cases, the project has already lost.

4. The data isn't there#

This is the failure mode nobody warns founders about. AI products need labeled data — even modern foundation models need it for fine-tuning, evaluation, and regression testing. Most startups don't have it.

The first thing I do on any AI engagement is a brutal audit:

• Where do we get inputs?
• Where do we get labels (even noisy ones)?
• How do we store them?
• How do we version them?
• Who owns them?

If the answers are "we don't have a label store" and "we'll figure it out later," the project will stall in month four. There is no AI product without a data pipeline.

Kill criterion: if there's no plan to collect, store, and version the data the AI will be evaluated on, the project is a demo.

5. The model is being asked to do work a database should do#

This is the failure mode I'm most opinionated about. AI products get accused of being unreliable because the team has used the LLM to do work that should have been done by a deterministic system.

Examples I see in real engagements:

• "Use GPT to extract the order ID from the email." (Use a regex.)
• "Use GPT to check if the user is authenticated." (Read the session.)
• "Use GPT to format this date." (Use Date.toISOString().)
• "Use GPT to validate the email address." (Use a Zod schema.)

Every one of these uses the model to do work a 1990s system could do better, faster, and cheaper. The model is for the part of the problem that's actually ambiguous — the part where deterministic code can't help. The rest is overhead.

Kill criterion: if you can describe the AI's job without using the word "probably" or "approximately," you're using the wrong tool.

The 4-week scoping playbook I use instead#

When a founder comes to me with an AI idea, this is the process I run. It's not glamorous. It's not where the demo videos come from. But it's the difference between shipping a product and shipping a project that ends in the kill-criterion list above.

Week 1: Define the job#

One sentence. Active voice. Concrete user. Concrete outcome.

Help [user] do [task] in [context], so [outcome].

Examples that pass:

• "Help support agents draft a response to a refund request in under 30 seconds, so ticket close-time drops by 40%."
• "Help lawyers find the relevant clause in a 50-page contract in under one minute, so first-pass review time drops by 60%."

Examples that fail:

• "Build an AI assistant." (For whom? Doing what?)
• "Use AI to make the product smarter." (Not a job.)
• "Add a chatbot." (Doing what job? Replacing what?)

If the founder can't write the one-sentence job, the project isn't ready to scope.

Week 2: Build the eval set before the model#

Write down 50 real inputs the user would send. For each, write down what a good response looks like. This is your eval set. It doesn't need to be perfect. It needs to exist.

Then write down 20 inputs the user won't send, but the system has to handle gracefully. Empty inputs. Out-of-domain questions. Adversarial inputs. This is your safety eval.

Now you have something the model can be measured against. Without it, every decision about the model is a guess.

Week 3: Pick the smallest architecture that could work#

Resist the temptation to use the latest model, the latest vector DB, the latest orchestration framework. Pick the architecture that solves the job in week 1 using the eval set from week 2.

In 2026, the smallest architecture for most AI products is:

• One LLM call per request, with structured output.
• One retrieval step if the job requires knowledge the model doesn't have.
• A deterministic wrapper around the whole thing for validation, logging, and fallback.
• A regression test that runs the eval set on every deploy.

That's it. No agents. No multi-step plans. No LangChain. If the smallest architecture can't hit the eval set, the problem is the job definition, not the architecture.

Week 4: Ship to five real users#

Not a waitlist. Not a "private beta." Five real users — people who would use the product even if it were terrible — using it for a week. Watch them. Take notes. Update the eval set with what they actually do.

If the product survives week 4 with a real success metric improving, you've shipped something. If it doesn't, you have a much cheaper failure than the founder I described at the top of this article.

What this means for your AI project#

If you're a founder reading this and recognizing your project in the five failure modes above, here's the hard truth: the project isn't broken, the scope is. Killing the project and starting over is cheaper than shipping a version of it that no one uses. I've watched founders burn $400k learning this. The lesson is the same at $4k.

If you're a founder who hasn't started yet, the playbook is the same in reverse: write the one-sentence job, build the eval set, pick the smallest architecture, ship to five users. Do all of that before you spend real money on model costs or engineering.

And if you're past the point where you can do this internally — if you have a real product and a real user base and a real AI feature that needs to be production-grade rather than demo-grade — that's the work I do. The first conversation is free, and I reply within a day.