Today, AI plays some part in every software delivery. In the early discussion of one customer implementation for a small Finnish organization, I told the customer this directly:
"AI can reduce enough manual work to make 'the perfect' tool for the customer's process feasible.
But only if the customer and the delivery team do the work together."
The condition is that the customer still has to confirm assumptions, make decisions and review results. Without that, we just produce the wrong solutions faster. In the same delivery, I saw how quickly a workshop note could become project truth: something everyone treats as settled even though nobody settled it.
One exception case was described in detail because handling it took resources. My AI assistant analyzed the customer’s material, using guidelines and grounding I had given it. The amount of text gave the exception more weight than it deserved, and in the analysis it came out looking like a confirmed priority.
It wasn’t. It was an assumption that looked like a fact.
I noticed the imbalance from experience, and a discussion with the customer confirmed that the exception was not a real priority.
I worry less about the strange AI answer than about the clean one. A strange answer makes people stop and ask. A clean one looks professional enough that people stop asking where it came from, so it travels: from the workshop note into the backlog, from the backlog into the design, from the design into the tests.
By the end, the documents agree with each other because they all repeat the same conclusion. Reviewers see that consistency and may take it as proof that someone checked the reasoning. But the error may simply have been copied carefully. Mechanically comparing one document with the next will not catch it at all. Catching it means going back to the source and, when needed, to the people who were in the room.
AI did not create the assumption. It made it cheaper to turn that assumption into seemingly consistent project material.
A better prompt cannot recover missing context.
When the output looks weak, rewriting the prompt is tempting. I do it too. It is easier than going back to the customer, but going back is usually the right fix. A prompt cannot put back context that was never in the source. It only gives you a cleaner version of the same uncertainty. And when the output looks good, we rarely check the source at all.
What helps is keeping facts, interpretations, assumptions and decisions apart before anything is built on them. AI can help with this too. It can make a first sort of the material and list the open questions. A person still checks every item against the source, but the sorting no longer has to be the slow part.
Usually, we talk about AI that writes documents. With agents that act, the same problem gets bigger. A wrong assumption in a backlog can still be challenged before someone acts on it. Give an agent the authority to continue the process, and that assumption can move straight into the next decision or action, past the hand-offs where a person would have had a chance to question it.
That is why the points where work stops for a human check have to be built in from the start. Each one needs a person who owns it and has the time and the authority to say no.
A review that can't stop anything and has no time to look critically is a signature, not a review.
The human in charge needs to be able to follow a result back to a decision, and the decision back to what the customer said. Before AI output becomes project truth, these are the five questions I want answered:
What source material was used?
Which assumptions were made?
What evidence supports the recommendation?
What is still uncertain?
Who has confirmed the result?
If an AI-generated requirement or decision cannot answer them, it is not ready to build on, however good it looks.
AI speeds up whatever is already in the delivery process, including assumptions nobody has tested. It does not fix the way we work. It shows it.