Systems | Development | Analytics | API | Testing

Ever since E. F. Codd introduced the first relational model for storing data at IBM in 1970, the industry has picked up the database technology and used it for its competitive advantage. The relational database management system - or RDBMS - was the default technology for storing and accessing data for a long time. It supported transactional data storage, the building of data products, and was the go-to model for data that was used in data-driven decisioning.

ETL and ELT refer to two patterns of data storage architecture within your data pipelines. The letters in both acronyms stand for: So both ETL (extract, transform, load) and ELT (extract, load, transform) processes help you collect data, transform it into a usable form and save it to permanent storage, where it can be accessed by data scientists and analysts to extract insights from the data. What is the difference?

The world of big data is getting bigger every day. As the volume of data increases exponentially, businesses of all sizes try to capture raw data, process it, and extract insights for competitive decision-making. The end-to-end operation of extracting value from data is called the ETL process. It stands for: A crucial component of the ETL process is the data storage aspect. The two main contentious architectures for storage solutions are databases and data warehouses. But how do they differ?

Whether you saw or missed our webinar, we thought it would be useful to provide a step-by-step guide on how to set up quick competition monitoring (or, any other web scraping and data processing automation) with Apify and Keboola. Thank you Apify and Revolt.bi for the collaboration! So what can you do with automated competition data processing? In this article, we’ll take an example of daily monitoring of the best-sellers list at Amazon.

You will have to consider multiple solutions and their tradeoffs when setting up your enterprise data architecture. In this article, we contrast a data lake and a data warehouse side-by-side to make your choice easier.

We rely on advanced data platforms that extract data from multiple sources, clean it, and save it so data scientists and analysts can gain insights from data. Data seems to flow seamlessly from one location to another, supporting our data-driven decision-making. The entire system runs smoothly because the engineering operations under the hood are correctly set and maintained.

Greetings, my dear readers. It’s been some time since I’ve posted my last article. This is the third chapter of the introduction to the internal data world of Keboola. In the previous chapters, I’ve posted about an introduction to our internal reporting and communication with our users. Since the last time, a couple things have happened.

Kindred Group is a publicly-traded gambling operator with offices across four continents, offering entertainment options such as online poker, sports betting, and online casinos. Since its founding, Kindred has experienced fast growth acquiring nine different gambling brands over the last 20 years. With over 30 million customers globally and numerous brands to manage, the Kindred team had a pressing need for a good data management system.

Distributed systems have unlocked high performance at a large scale and low latency. You can run your applications worldwide from the comfort of your Amazon Web Services (AWS) platform in California, but the user adding an item to their shopping cart in Japan will not notice any delay or system faults. However, distributed systems - and specifically distributed database systems - also malfunction.