How to Test RAG Pipelines for Reliable AI | Aparana Gupta | Testflix 2025 | #testingcommunity
Retrieval-Augmented Generation can sound convincing while still being wrong. This session focuses on moving beyond surface-level metrics and turning stochastic AI outputs into evidence-backed, verifiable results. It explores how to test the entire RAG pipeline, from ingestion and indexing to retrieval, grounding, and answerability, ensuring every response is traceable to the right source, policy, and user context.
The talk introduces practical testing techniques such as rationale overlap, abstention SLOs, robustness checks for typos and paraphrases, counterfactual testing, and drift detection through embedding and index analysis. It also covers chaos testing, poisoning detection, and enforcing “smart silence” when answers lack support. The outcome is a CI-ready testing approach that transforms RAG from impressive demos into auditable, reliable systems, giving QA teams confidence, clarity, and control.
#AutomationTesting #TestAutomation #ITindustry #TheTestTribe #Leadership
This video is of one of the sessions presented at #Testflix - World’s Leading Virtual Software Testing Conference.
About the Speaker:
With over 15 years of trailblazing expertise at the nexus of artificial intelligence and data science, Aparana Gupta stands as a beacon of leadership in the technology arena. Her illustrious career, underpinned by an distinguished academic lineage and a relentless commitment to product excellence and design thinking, has propelled her to the forefront of the industry. Renowned for fostering data-centric, AI-empowered cultures, Aparana has consistently ignited innovation and accelerated strategic growth across multiple domains.
Her expansive interdisciplinary skill set spans Statistical and Stochastic Modeling, Machine Learning, Prompt Engineering, and Data Strategy – expertise that not only places her among the elite leaders in AI and analytics but also makes her a highly sought-after thought leader, speaker, and mentor.
At Microsoft Digital & Employee Experience, Aparana Gupta leads a vibrant and diverse team as Principal Software Engineering Manager, passionately engineering next-generation AI solutions. By artfully integrating data, design, technology, and leadership, she is actively shaping the future of innovation.
Connect with Aparana on LinkedIn - https://www.linkedin.com/in/aparanagupta/
Testflix 2025 Proud Sponsors:
Premier Sponsor – BrowserStack: https://www.browserstack.com/
Platinum Sponsor – AutoTester.ai: https://autotester.ai/
Platinum Sponsor – UiPath: https://www.uipath.com/
Gold Sponsor – ACCELQ: https://www.accelq.com/
Gold Sponsor – Sahi Pro: https://www.sahipro.com/
Silver Sponsor – Beeceptor: https://beeceptor.com/
Bronze Sponsor – Baseline11: https://baseline11.com/
Learn from industry experts with Thrive EdSchool courses, grow your network with software testers at The Test Tribe events, and join Asia's largest testing community on Discord.
Upskill yourself with Thrive EdSchool courses: https://bit.ly/thrivettt
Grow your network with software testers with the events at The Test Tribe: https://bit.ly/tttevents
Become a member of Asia's largest testing community: https://bit.ly/3FONxJP
About The Test Tribe:
The Test Tribe is the World’s Largest Software Testing Community turned EdTech Startup. Founded in 2018, our mission is to give the Testing Craft the glory it deserves while empowering testers globally through courses, memberships, cohorts, meetups, hackathons, and global conferences. With 800+ events and 165K+ members across 152+ countries, we’re helping testers collaborate, learn, and grow together.