Real-time feature engineering is valuable for a variety of use cases, from service personalization to trade optimization to operational efficiency. It can also be helpful for risk mitigation through fraud prediction, by enabling data scientists and ML engineers to harness real-time data, perform complex calculations in real time and make fast decisions based on fresh data, for example to predict credit card fraud before it occurs.

Our recent blog discussed the four paths to get from legacy platforms to CDP Private Cloud Base. In this blog and accompanying video, we will deep dive into the mechanics of running an in-place upgrade from CDH5 or CDH6 to CDP Private Cloud Base. The overall upgrade follows a seven-step process illustrated below. In the video below we walk through a complete end to end upgrade of CDH to CDP Private Cloud Base.

Digital transformation has been talked about for many years, but the pandemic has accelerated the digital transformation journeys for many enterprises. Forced to adapt to changes in the business landscape and customer behavior, businesses have adopted more digital tools and technologies to drive innovation and increase resilience.

The digital race is on. To pull ahead of the pack, a company needs to know what to do with its data. Without a data-driven strategy, you’re bound to lose ground to competitors who apply their data to operational improvements, product development, go-to-market strategies, and the customer experience. It isn’t enough to collect, interpret, and act on the data. You have to do it fast.

Who are my top sellers? What kinds of deals have the highest close rate? How have our sales opportunities changed over time? As a sales leader, these are just some of the questions you ask yourself every day to keep your team on track. And the answers are in your data. Every form fill, cold call, and MQL is another data point you can use to assess the health of your sales organization.

Feature stores have arrived in 2021 as an essential piece of technology for operationalizing AI. Despite the enthusiasm for feature stores in high-tech companies, they are still absent from most legacy ML platforms and can be relatively unknown in many enterprise companies. We discussed how feature stores are critical to the data-first approach of next-gen ML platforms in our previous blog, but they are important enough to get their own treatment in a full article.

Cloudera Data platform (CDP) provides a Shared Data Experience (SDX) for centralized data access control and audit in the Enterprise Data Cloud. The Ranger Authorization Service (RAZ) is a new service added to help provide fine-grained access control (FGAC) for cloud storage. We covered the value this new capability provides in a previous blog.

Unless you’ve hidden under a rock for the past decade, you can’t have failed to notice that data in today’s enterprise is very much alive. It’s always moving, constantly changing, and we’re continually using it to create new business value. However, while data fluidity and visibility have blossomed, the opportunity to use that data to drive business actions seems to have withered in comparison.

In a recent webinar by TDWI, 45% of analysts reported that “every day seems to be a different fire drill.” No surprise to anyone in the industry. As much as analysts need to be focused on more strategic tasks, their skills are frequently deployed to answer basic questions. Greater self-service capabilities for end-users would no doubt alleviate these fire drills, but this is not yet a reality for the majority of companies.

In financial services, data has always been viewed as a strategic asset. To manage this data, organizations have invested heavily over several years and across a number of technology generations in the underlying data infrastructure. This approach has left a large data technology legacy along with silos of data linked to specific infrastructure and applications.

Run Fivetran on different clouds to gain flexibility and control while reducing costs.

BigQuery is a fully-managed enterprise data warehouse that helps you manage and analyze your data with built-in features like machine learning, geospatial analysis, and intelligent caching for business intelligence. To help you make the most of BigQuery, we’re offering the following no cost, on-demand training opportunities.

One of the most substantial big data workloads over the past fifteen years has been in the domain of telecom network analytics. Where does it stand today? What are its current challenges and opportunities? In a sense, there have been three phases of network analytics: the first was an appliance based monitoring phase; the second was an open-source expansion phase; and the third – that we are in right now – is a hybrid-data-cloud and governance phase. Let’s examine how we got here.

In our modern digital society, data is abundant, and storage is affordable. Businesses, governments and even individuals can (and do) collect every transaction, click, swipe, location, message and attribute in their datasets. With just a few clicks on my smart device, I can review data on every place I’ve been, how much I spent, every step I took, what the weather was like and who I was with. Businesses collect the same abundance of data.

Sometimes I walk through the grocery store and marvel at the way customers float through the aisles, blissfully unaware of the logistical nightmare it probably took to stock the shelves. They have no idea how many people, systems, and modes of transportation it takes to make everything magically appear on their grocery shelves. But I do. As the Senior Director of Software Engineering at KlearNow, I spend my days preserving the bliss of those grocery shoppers.

The development of a digital product has been redefined to involve only 4 phases, as TCGen and Product Plan propose: However, having an easier-to-follow process is not the only improvement that you can implement: cost and time efficiency can be taken a huge step further when you incorporate analytics insights. So, with this infographic, we propose some tools that can help you analyze data sets to enrich the phases of each development process.

For modern businesses faced with increasing volumes and complexity of data, it’s no longer efficient or feasible to rely on analyzing data in BI dashboards. Traditional dashboards are great at providing business leaders with insights into what’s happened in the past, but what if they need actionable information in real time? What if they want to use their data to estimate what may happen in the future? Companies are taking notice.

A slow car has never won a Formula One race. The Olympics doesn’t reward slow times in swimming, track or any other clock-timed sport. Likewise, slow data speeds don’t win over customers or colleagues in the real-time business world. Microsoft’s own research once reported that a person visiting a website on a connected device is likely to wait no more than 10 seconds to see it before moving to a competitor’s site.

Does it feel like your business is paying too much for cloud services? You are not alone. Cloud costs are expected to increase at a compound annual growth rate of 10.5% to 13.1% through 2025, according to the International Data Corporation (IDC). While getting a handle on those cloud costs may be tricky, you don’t have to worry — we’ve got you covered.

With COVID-19’s ever-changing conditions – growing infection rates, shifting and new vaccine mandates, variant outbreaks and office closures and re-openings – HR has stepped up and taken on a significant role in helping organizations navigate every employee’s personal and work life needs. COVID-19 accelerated the evolution already underway in HR, with HR growing beyond being a policy and procedure hub into a strategic business partner.

Since the start of the pandemic, business demands on your IT team have skyrocketed. You need granular, actionable insights to keep up with the speed and volume of digital transformation projects and IT incidents occurring across your organization. Canned reports from SaaS-based systems like ServiceNow aren’t fundamentally built for analytics.

Our joint customers can remain within Azure for all their cloud services, facilitating compliance and minimizing data movement costs.

When organizations track metrics by the thousands, millions, or even billions, it’s helpful in many ways to understand which metrics have close relationships, meaning when one metric behaves in a certain way, one or more additional metrics can be expected to behave in a similar or opposite way.

Learn how you can keep track of all of your organization’s data assets in one place

Do you ever feel like connecting with the right customer audience is just a matter of luck? We’ve met a CDO who leaves audience targeting up to chance. Cosmo, CDO is not a Chief Data Officer — he’s a Chief Destiny Officer. While we focus on data here at Talend, we’re trying to understand the 36% of business executives who say they don’t base the majority of their decisions on data.

Many customers looking at modernizing their pipeline orchestration have turned to Apache Airflow, a flexible and scalable workflow manager for data engineers. With 100s of open source operators, Airflow makes it easy to deploy pipelines in the cloud and interact with a multitude of services on premise, in the cloud, and across cloud providers for a true hybrid architecture.

Businesses are increasingly embracing a cloud-first approach to increase market responsiveness and flexibility. The cloud-first approach refers to a cloud-like experience consisting of on-demand metered consumption of IT infrastructure, whether on the public cloud or inside private data centers. The rapidly evolving consensus among the tech leaders and vendors has led to an emergence of hybrid IT.

Fivetran agrees to acquire HVR, the leader in enterprise database replication, and raises $565M in Series D funding.

There are many ways that Apache Kafka has been deployed in the field. In our Kafka Summit 2021 presentation, we took a brief overview of many different configurations that have been observed to date. In this blog series, we will discuss each of these deployments and the deployment choices made along with how they impact reliability.

With the explosion of the machine learning tooling space, the barrier to entry has never been lower for companies looking to invest in AI initiatives. But enterprise AI in production is still immature. How are companies getting to production and scaling up with machine learning in 2021? Implementing data science at scale used to be an endeavor reserved for the tech giants with their armies of developers and deep pockets.

Ecommerce customers can now connect Fivetran to PCI-validated data sources and transmit cardholder data to a PCI-validated destination.

As businesses advance their digital transformation efforts, enterprise resource planning (ERP) systems are evolving — arguably as significantly as the shift from materials resource planning (MRP) to ERP. Just as businesses reimagined operations by leveraging advances in hardware and software in the 1980s, they’re now turning to next-generation ERP.

Whether you are single, married, in a civil partnership, engaged or focusing on friendships, there is one relationship we all have in common that plays out daily in modern life – and that is our relationship with data.

My whole life I’ve been curious. You have to be, to become an entrepreneur. I’m curious about trends, about looking at data and finding patterns, which might show you where the next opportunity lies. And, as I discussed with Joe DosSantos in the latest episode of Data Brilliant, I’m a big believer in experimentation and learning by putting the data and analysis into practice.

An in-depth discussion of Fivetran’s SOC 2 Type 2 compliance and Powered By Fivetran (PBF).

Consumption-based, aka usage-based, pricing is hardly new. Anyone with an electricity, gas, or water bill knows that the amount you pay each month varies depending on your usage. More recently, disruptive companies have pushed other industries (transportation, hospitality, communications, and insurance) to transform by providing usage-based products and services via software applications. As consumers, we see this all around us, when we hail an Uber or choose a short-term rental on AirBnB.

When I began working at Qlik nearly seven years ago, it was fairly common for us to end our first meeting with a Federal agency with their team saying: “We need this.

Integrate Fivetran into your infrastructure-as-code development.

After migrating a Data Warehouse to Google Cloud BigQuery, ETL and Business Intelligence developers are often tasked with upgrading and enhancing data pipelines, reports and dashboards. Data teams who are familiar with SQL Server Integration Services (SSIS) and SQL Server Reporting Services (SSRS) are able to continue to use these tools with BigQuery, allowing them to modernize ETL pipelines and BI platforms after an initial data migration is complete.

Leveraging the Internet of Things (IoT) allows you to improve processes and take your business in new directions. But it requires you to live on the edge. That’s where you find the ability to empower IoT devices to respond to events in real time by capturing and analyzing the relevant data.

Analysts have found that roughly 50% or more enterprise resource planning (ERP) projects fail, and approximately half of all projects are considered challenging.

Data is the new oil. It’s a phrase we’ve heard a lot in recent years, and it’s not hard to understand why. We’re generating more data every day than ever before, and companies are scrambling to find ways to store that information without running out of space.

The scientific method is a proven route to successful, tested and verified improvement. Here’s how to combine it with BI.

The new tools offer improved usability over the REST API for developers in Go.

Account-based marketing, or ABM, is more often used as targeted demand generation—not one-to-one marketing. In a 2020 study of more than 300 organizations worldwide, Forrester found that “a significant number of respondents claimed they were using an ABM approach but weren’t doing what we would consider the basics of ABM, such as working with sales.”1 ABM isn’t just about assigning one siloed team the responsibility of targeting and revealing high-potential prospects.

There are two big gaps in the Apache Kafka project when we think of operating a cluster. The first is monitoring the cluster efficiently and the second is managing failures and changes in the cluster. There are no solutions for these inside the Kafka project but there are many good 3rd party tools for both problems. Cruise Control is one of the earliest open source tools to provide a solution for the failure management problem but lately for the monitoring problem as well.

With customer-managed keys, Fivetran Business Critical users running AWS gain full control over credential encryption.

Shared Data Experience (SDX) on Cloudera Data Platform (CDP) enables centralized data access control and audit for workloads in the Enterprise Data Cloud. The public cloud (CDP-PC) editions default to using cloud storage (S3 for AWS, ADLS-gen2 for Azure). This introduces new challenges around managing data access across teams and individual users. To solve these challenges for S3 and ADLS-gen2, Cloudera has introduced a new service — the Ranger Authorization Service (RAZ).

Across the federal government, agencies are struggling to identify, organize, analyze, and act on troves of data. It’s a problem that leaders are working actively to tackle, but they’re in a race against immeasurable volumes of data that is continuously being generated in perpetuity in stores known and unknown. At the Internal Revenue Service, decades’ worth of data exceeds even the most cutting-edge processing capabilities.

Learn how off-the-shelf, open-source dbt packages make data modeling frictionless.

Advertising agencies are faced with the challenge of providing the precision data that marketers require to make better decisions at a time when customers’ digital footprints are rapidly changing. They need to transform customer information and real-time data into actionable insights to inform clients what to execute to ensure the highest campaign performance.

The CDP Operational Database (COD) builds on the foundation of existing operational database capabilities that were available with Apache HBase and/or Apache Phoenix in legacy CDH and HDP deployments.

Spark is known for being extremely difficult to debug. But this is not all Spark’s fault. Problems in running a Spark job can be the result of problems with the infrastructure Spark is running on, inappropriate configuration of Spark, Spark issues, the currently running Spark job, other Spark jobs running at the same time – or interactions among these layers.

When you build a data warehouse, the important question is how to ingest data from the source system to the data warehouse. If the table is small you can fully reload a table on a regular basis, however, if the table is large a common technique is to perform incremental table updates. This post demonstrates how you can enhance incremental pipeline performance when you ingest data into BigQuery.

In recent years there has been increased interest in how to safely and efficiently extend enterprise data platforms and workloads into the cloud. CDOs are under increasing pressure to reduce costs by moving data and workloads to the cloud, similar to what has happened with business applications during the last decade. Our upcoming webinar is centered on how an integrated data platform supports the data strategy and goals of becoming a data-driven company.

Growth. It’s the mountain every startup founder must learn to climb in order to run a successful business. And as with any great mountain, the journey to the top never feels more daunting than at the base. How your startup earns its first 10 customers will set the tone for the rest of the trek and determine how fast your team reaches the summit — if at all.

Have you ever wished you had a crystal ball? We tracked down a CDO who actually uses one. See, Cosmo, CDO is not a Chief Data Officer — he’s a Chief Destiny Officer. We’re all about data at Talend, but sometimes it’s good to see things from another perspective. We sat down with Cosmo to ask him about his job, his background, and his methods.

In the wake of COVID-19, we saw a significant shift toward as-a-service offerings, something we haven’t seen in years. From conversations with CIOs over the past 12 months, we know they are looking for the flexibility, efficiencies and cost savings they get from the as-a-service model. This is especially important to them as they evolve their business models in a hybrid IT direction and become consumers of IT.

Beaumotica combines smart lighting, design, and top brands to create the perfect mood and atmosphere for any room. And with help from Talend, the company can now combine data, analytics, and automation to optimize business decisions and accelerate growth. Last year alone the company tripled its business and expanded into new territories across Europe. Based in The Netherlands, Beaumotica has been growing steadily since 2007.

Fivetran Business Critical secures data traffic with AWS PrivateLink.

“If we didn’t have Stitch, we would have to recruit and hire data engineers, buy space for hundreds of millions of rows that we’re sinking into the database, and on and on. For us, Stitch is essential.” –Tomasz Eitner, BI and Data Analyst, Simba Sleep Simba Sleep has always been a data-driven company. Before the firm was even formally launched, the founders purchased research profiles from more than 10 million sleepers—including 180 million body profile data points.

Integrating digital technologies into every area of your business can vastly improve your finance analytics.