Four questions to ask before you build an AI feature
Most failed AI projects don't fail because the model wasn't clever enough. They fail because nobody agreed what success meant at the start.
Every few weeks we get a request that sounds roughly the same: 'we want to add AI to our product'. That isn't a bad request, but it isn't yet a workable one. Before the first line of code, four questions need answers.
1. What decision is a human making today?
AI systems are most useful when they replace or accelerate a decision that already exists, rather than inventing a new one. If you can't point to a person making that call manually today, you probably haven't found the problem yet.
2. What does one mistake cost?
The answer shapes the entire architecture. Cheap mistakes permit full automation. Expensive mistakes demand human review, confidence flagging, and audit trails — all of which is engineering work that has to be budgeted from the start.
3. How will you measure output quality?
We build the evaluation harness before the feature. Without one, the only way to judge the system is 'it feels better' — and a feeling is not a sufficient basis for deciding to ship.
If you can't measure it, you aren't building a system — you're building a demo.
4. What does it cost per month at full scale?
Model call costs look trivial in a prototype and can become a serious line item once the whole company uses it. Do the arithmetic early. Sometimes it shows that a non-AI approach makes more sense — and that's a legitimate conclusion.
