Systems | Development | Analytics | API | Testing

Why the real device versus virtual device debate is the wrong frame for engineering leaders and what the right one looks like.

Traditional observability was built around requests. A request comes in. A service processes it. A response goes out. You trace what happened in between. That model works well for APIs. It breaks down for AI. With agents, a single user request can trigger multiple model calls. An agent might invoke a tool, call another agent, query an MCP server, retry a model, hit a guardrail, and call another tool before finally producing a response. And all of this happens non-determinstically.

Across teams, organizations, and industries, people are starting with AI instead of the problem they want to solve. As a result, AI outputs from different tools: This is a process failure, and it's accelerating. Human-in-the-loop (HITL) AI is a framework that integrates human oversight directly into the machine learning lifecycle. Rather than relying on fully autonomous systems, HITL uses humans and machines collaboratively to train models, evaluate outputs, and handle complex decision-making.

If you only read the headlines this year, you’d think AI makes shipping good software easier than ever. Yet, this was the year of very public AI-coding incidents. PocketOS’ production database deletion and a Vercel AI agent shipping unverified code are just two examples of AI-authored code shipped with confidence that turned out to be wrong. And, of course, these AI-coding incidents are distinct from Agentic AI orchestration incidents like the HuggingFace hack by OpenAI.

Your developers are shipping more code than ever, and a growing share of it is written by AI. Your QA team, meanwhile, is probably the same size it was a year ago. You’re not imagining the squeeze. In a recent survey of 300 QA practitioners, 52% said bug volume went up after their developers started using AI, and 58% said their own testing workload grew. Not one of them reported getting extra QA headcount to match.

Most enterprise AI infrastructure conversations land on the same tension. Builders want self-service: instant access to GPUs, a stable endpoint for a model, a notebook, or a fine-tuning run that starts without a ticket. Platform teams want per-team quotas, identity-aware access, audit trails, tenant isolation, and cost attribution they can defend. Both sides are right, and most platforms make you choose between them because self-service and control get layered onto a base that was designed for neither.