Few things are scarier than a database slowly losing integrity over weeks or years. For a while, nobody notices anything. Then users start reporting bugs, yet you can't find any code that's broken. By the time you realize the problem, it may be happening for so long that your backups are unusable. We can avoid problems like these with skillful use of transactions.

The N+1 Queries Problem is a perennial database performance issue. It affects many ORM’s and custom SQL code, and Django’s ORM is not immune either. In this post, we’ll examine what the N+1 Queries Problem looks like in Django, some tools for fixing it, and most importantly some tools for detecting it. Naturally, Scout is one of those tools, with its built-in N+1 Insights tool.

When it comes to data, let’s start with the obvious. Averages suck. As developers, we all know that percentiles are much more useful. Metrics like P90, P95, P99 give us a much better indication of how our software is performing. The challenge, historically, is how to track the underlying data and calculate the percentiles. Today I will show you how amazingly easy it is to aggregate and create SQL based percentile rollups with Postgresql and t-digest histograms!