Democratization of data within an organization is essential to help users derive innovative insights for growth. In a big data environment, traceability of where the data in the data warehouse originated and how it flows through a business is critical. This traceability information is called data lineage. Being able to track, manage, and view data lineage helps you to simplify tracking data errors, forensics, and data dependency identification.
Businesses today have a growing demand for real-time data integration, analysis, and action. More often than not, the valuable data driving these actions—transactional and operational data—is stored either on-prem or in public clouds in traditional relational databases that aren’t suitable for continuous analytics.
Continuous evaluation—the process of ensuring a production machine learning model is still performing well on new data—is an essential part in any ML workflow. Performing continuous evaluation can help you catch model drift, a phenomenon that occurs when the data used to train your model no longer reflects the current environment.
Retail businesses understand the value of demand forecasting—using their intuition, product and market experience, and seasonal patterns and cycles to plan for future demand. Beyond the need for forecasts that are as accurate as possible, modern retailers also face the challenge of being able to perform demand planning at scale.
Businesses increasingly gather data to better understand their customers, products, marketing, and more. But unlocking valuable and meaningful insights from that data requires powerful, reliable, and scalable solutions. We hear from our BigQuery and Looker customers that they’ve been able to modernize business intelligence (BI) and allow self-service discovery on the data the business collects.
Data analysts can spend hours writing SQL each day to get the right insights. So it’s crucial that the tools in the Google Cloud Console make that job as easy and as fast as possible. Now, we’re excited to show you how BigQuery’s Cloud Console UI has been updated with radical usability improvements for more efficient work, making it easier to find the data you need and write the right SQL quickly.
BigQuery offers the ability to quickly import a CSV file, both from the web user interface and from the command line: Indeed, try to open this file up with BigQuery: and we get the errors like: This is because a row is spread across multiple lines, and so the starting quote on one line is never closed. This is not an easy problem to solve — lots of tools struggle with CSV files that have new lines inside cells. Google Sheets, on the other hand, has a much better CSV import mechanism.
From rice genomes to historical hurricane data, Google Cloud Public Datasets offer a world of exploration and insight. The more than 20 PB across 200+ datasets in our Public Dataset Program helps you explore big data and data analytics without a lot of cost, setup, or overhead. You can explore up to 1 TB per month at no cost, and you don’t even need a billing account to start using BigQuery sandbox.
We launched BigQuery ML, an integrated part of Google Cloud’s BigQuery data warehouse, in 2018 as a SQL interface for training and using linear models. Many customers with a large amount of data in BigQuery started using BigQuery ML to remove the need for data ETL, since it brought ML directly to their stored data. Due to ease of explainability, linear models worked quite well for many of our customers.
At Google Cloud, we’re invested in building data analytics products with a customer-first mindset. Our engineering team is thrilled to share recent feature enhancements and product updates that we’ve made to help you get even more value out of BigQuery, Google Cloud’s enterprise data warehouse.
