Systems | Development | Analytics | API | Testing

Business intelligence (BI) is the art of extracting actionable insights from your datasets. There’s a whole stack of technologies under the hood.

Every day, hundreds of thousands of residents and commuters in San Francisco, California, use the public transportation services of the San Francisco Municipal Transportation Agency (SFMTA). In addition to the city’s buses, subway system, and famous cable cars, the SFMTA manages comprehensive services including bicycle and e-scooter rentals, as well as permits for road closures.

Recently, I worked with a large fortune 500 customer on their migration from Apache Storm to Apache NiFi. If you’re asking yourself, “Isn’t Storm for complex event processing and NiFi for simple event processing?”, you’re correct. A few customers chose a complex event engine like Apache Storm for their simple event processing, even when Apache NiFi is the more practical choice, cutting drastically down on SDLC (software development lifecycle) time.

Earlier this year at Snowflake Summit 2021 , we announced Snowpark Accelerated , a new program for partners who integrate with Snowpark. It provides them with access to technical experts and additional exposure to Snowflake customers. It’s been incredibly exciting to watch what our partners have been building with the help of our new developer experience, which brings deeply integrated, DataFrame-style programming to the languages developers like to use.

The story of the last year+ is one of disruption and change across every aspect of our lives. As we all navigated a ‘new norm,’ businesses naturally had to pivot as well, with some sectors finding new opportunities while others scrambled to reimagine their entire go-to-market strategies.

BigQuery is capable of some truly impressive feats, be it scanning billions of rows based on a regular expression, joining large tables, or completing complex ETL tasks with just a SQL query. One advantage of BigQuery (and SQL in general), is it’s declarative nature. Your SQL indicates your requirements, but the system is responsible for figuring out how to satisfy that request. However, this approach also has its flaws - namely the problem of understanding intent.