Systems | Development | Analytics | API | Testing

The data lakehouse is a promising new technology that combines aspects of data warehouses and data lakes.

Companies have had only mixed results in their decades-long quest to make better decisions by harnessing enterprise data. But as a new generation of technologies make it easier than ever to unlock the value of business information, change is coming. We’ve already reaped gains at Hitachi Vantara, where I run a global IT team that supports 11,000 employees and helps more than 10,000 customers rapidly scale digital businesses.

Since the data lake concept emerged more than a decade ago, data lakes have been pitched as the solution to many of the woes surrounding traditional data management solutions, like databases and data warehouses. Data lakes, we have been told, are more scalable, better able to accommodate widely varying types of data, cheaper to build and so on. Much of that is true, at least theoretically.

Cloud-based data warehouses offer unlimited scalability with the best features of both traditional data warehouses and data lakes.

Building a data lake in Amazon S3 using AWS Spectrum to query the data from a Redshift cluster is a common practice. However, when it comes to boosting performance, there are some tricks that are worth learning. One of those is using data in Parquet format, which Redshift considers a best practice. Here's how to use Parquet format with Xplenty for the best data lake performance.

The data lakes concept has come back into popular focus with Amazon Athena, an innovative, serverless solution. But does it fit into your organization’s data stack? This article covers Amazon Athena capabilities, pros and cons, competitors, and use cases.

Public preview of the data lake export feature is now available. Snowflake announced a private preview of data lake export at the Snowflake virtual summit in June 2020. Data lake export is one of the key features of the data lake workload in the Snowflake Data Cloud. The feature makes Snowflake data accessible to the external data lake, and it enables customers to take advantage of Snowflake’s reliable and performant processing capabilities.

Imagine if you bought a beautiful lake house, invited all your friends to come and visit, and the lake was dry? Not much value and a little embarrassing, right? Now imagine you have that beautiful lake house and you have special water valves to control not just if there is water in the lake but also control the water quality, clarity, and what fish the lake is stocked with? Much more impressive, correct?