Systems | Development | Analytics | API | Testing

This talk explores the challenges of testing agentic AI systems—AI that autonomously reacts to events and initiates processes. Drawing on decades of experience, Robert Sabourin emphasizes that testing begins and ends with risk. A three-dimensional model (business impact, technical risk, autonomy) guides evaluation. Testers generate ideas using a broad taxonomy, from capabilities and failure modes to creative and adversarial approaches. Continuous testing and monitoring ensure findings inform business decisions, emphasizing learning over correctness.

AI promises speed, but testers are often the first to notice where it quietly breaks down. Beneath the impressive outputs lie hidden issues like hallucinations, false confidence, and blind spots that can easily go unnoticed yet cause real damage if left unchecked. This atomic talk explores the subtle ways AI can fail, why speed without reliability is risky, and how testers play a critical role in supervising and strengthening AI systems. It highlights practical strategies for working alongside AI to make its outputs more trustworthy, reliable, and genuinely useful.

Testers are great at finding flaws in systems. But what happens when the system under test is the tester themselves? In today’s world of constant change, rising stress, and growing uncertainty, resilience has become just as critical as technical skill. From handling pushback and tight timelines to navigating burnout and self-doubt, testers face pressures that often go unseen.

As AI agents take on critical roles in testing and automation, their weaknesses can become hidden risks waiting to surface. This session explores how applying red teaming techniques, borrowed from cybersecurity, can help uncover vulnerabilities in AI agents before they cause real-world failures.

Business logic sits at the core of every system, yet it often becomes the hardest part to change. Rules get buried in complex code, feedback arrives too late, and even small updates turn into slow, risky efforts. This session explores why teams struggle with something that should be simple and how to bring clarity back into the process.

AI has transformed how software is built and tested, yet many teams still rely on traditional metrics like pass rates, coverage, and defect counts. While these numbers look good on dashboards, they often fail to answer the most important question in the AI era. Can we actually trust what the system is doing?

Agentic code generation has dramatically accelerated development, but QA often remains slowed by coordination gaps. Manual planning, delayed handoffs, and fragmented reviews continue to create friction, while AI agents operate in isolation without orchestration or governance. This session explores how to compose these siloed agents into a unified, AI-powered STLC where workflows are dependency-aware, auditable, and automatically triggered as soon as prerequisites are met.

Before building artificial intelligence, it’s worth asking whether we truly understand natural intelligence. Just as early pioneers of flight studied the principles of aerodynamics and observed how birds fly, this session argues that progress in AI requires a deeper understanding of human intelligence and the knowledge that already exists across related disciplines.

Just like humans, AI systems are shaped by how they are brought up. In the case of Large Language Models, this upbringing happens through data collection, training, and productization. At each of these stages, bias can quietly enter the system through the data we select, the way models are trained, or the assumptions embedded into the final product. These biases, whether intentional or accidental, influence how models think, respond, and interact with users in the real world.

Generative AI can produce tests instantly, but speed alone does not guarantee quality or safety. Without proper validation, AI-written tests can become brittle, redundant, or misleading, creating a false sense of coverage. This session looks at the risks of relying on AI-generated tests without the right controls in place.