Analytics

The only certainty in today’s world is change. And nowhere is that more apparent than in the way organizations consume data. A typical company might have thousands of analysts and business users accessing dashboards daily, hundreds of data scientists building and training models, and a large team of data engineers designing and running data pipelines. Each of these workloads has distinct compute and storage needs, and those needs can change significantly from hour to hour and day to day.

As businesses increasingly leverage data-driven decision making, the ability to use and understand data at the company-wide level becomes mission critical. While tech behemoths like Netflix, Airbnb, and Spotify have strong data cultures built over the last decade, most companies often face challenges getting up and running with data.

Today we are introducing a new series of blog posts that will take a look at recent enhancements to Apache Impala. Many of these are performance improvements, such as the feature described below which will give anywhere from a 2x to 7x performance improvement by taking better advantage of all the CPU cores. In addition, a lot of work has also been put into ensuring that Impala runs optimally in decoupled compute scenarios, where the data lives in object storage or remote HDFS.

Today, companies can measure every aspect of business health, except the health of their data which drives business decisions. Data is vital to inform critical decisions such as identifying new routes to market, systems to support business agility, and more resilient supply chains. As Harvard Business Review puts it, “Your organization’s data is the source of both the opportunity and the challenges to your innovation.

Data architecture is a hot topic right now. And rightfully so. Technological advances bring out a myriad of new solutions that go beyond the traditional relational databases and data warehouses. They enable companies to accelerate their entire data pipeline (or at least remove painful bottlenecks) and shorten the analytic cycles. The portfolio of data assets managed by companies is also growing.

For enterprise applications and startups to scale, they need to manage large volumes of data in real-time. Customers must have the ability to search for any product or service from your database within seconds. When you manage a relational database, data is spread across multiple tables. So, customers may experience lag during search and data retrieval. However, this is different with Elasticsearch and other NoSQL databases.

How can your organization ensure data access and transparency? Measure yourself against this model, courtesy of our partners at Slalom.