The Monday-morning test
It was a good demonstration. A customer had submitted an application with a missing document. The new AI agent read the request, found the relevant policy, identified the gap and drafted a helpful message. For a moment, the future felt very close.
Then imagine the same agent on a busy Monday morning. One customer uploads an outdated statement. Another uses a trading name that does not match the registered entity. The policy source has been updated but the search index has not.
Nothing in that list is particularly exotic. It is ordinary business. Yet it changes the question completely. We are no longer asking whether the model can produce an impressive answer. We are asking whether the organisation can rely on a system to perform a real task when evidence conflicts, systems fail and the consequences belong to someone.
