Systems | Development | Analytics | API | Testing

If you run a software engineering tools team, you have almost certainly had this conversation: a developer asks for production data access to debug a real incident, and someone in the room says no. Not because the request is unreasonable (it isn’t), but because nobody wants to be the person who said yes when something goes wrong. That instinct is understandable. Production environments carry real risk. But the reflex to lock everything down has a cost that rarely gets accounted for.

Testing a FastAPI app that calls OpenAI, Anthropic, or Gemini gets expensive fast. The problem is not just the API bill in production. It is all the repeated traffic in development: prompt tweaks, CI runs, regression checks, and the load tests you keep putting off because every run burns tokens. Hand-written mocks do not help much once the app is doing multi-step LLM work.

Most teams know they are spending money on AI in production. Far fewer realize how much they are spending outside production. It’s easy to get lost as you evaluate which model has the best responses, is fast enough, and cheap enough to run in production. That is because the AI bill usually shows up as a giant blob. It is easy to see the total.

Recent outages tied to AI-assisted code changes have pushed companies into a corner. After several incidents with massive “blast radius” impacts, organizations like Amazon introduced stricter controls—mandating that senior engineers manually review all AI-generated code before it hits production. That response makes sense on paper, but it exposes a fatal flaw in the modern development pipeline.

The pitch was simple: let AI write your code so you can focus on the hard problems. Three years into the AI coding revolution, and developers are focused on hard problems alright, just not the ones anyone expected. Instead of designing systems and solving business logic, engineers in 2026 spend a startling amount of their day managing the AI itself. Should you use Fast Mode or Deep Thinking? Haiku or Opus? Cursor or Claude Code or Windsurf? Should you write a SKILL.md file or a custom system prompt?