What Can Go Wrong? Understanding Risk & Failure Modes in Agentic AI
Agentic AI systems don’t fail like traditional software - they hallucinate facts, pursue the wrong goals, overuse tools, and forget context. These failures look “correct” to traditional test cases, but feel dangerously wrong to users. One team tested an AI support bot - it passed every check, but in production, it gave refund advice that violated company policy. Not a code error. A reasoning failure.