Systems | Development | Analytics | API | Testing

Advertising agencies are faced with the challenge of providing the precision data that marketers require to make better decisions at a time when customers’ digital footprints are rapidly changing. They need to transform customer information and real-time data into actionable insights to inform clients what to execute to ensure the highest campaign performance.

Sometime in 2019, Netflix cracked a conundrum that stumped them for years. The company had so much data about its content and subscribers, it had to sync multiple heterogeneous data stores like MySQL and Elasticsearch continuously, which brought seriously stressful challenges like dual writes and distributed transactions. So Netflix created its own CDC tool that processes captured log events in sequence and takes dumps for specific tables and primary keys of tables. Problem sorted. Case closed.

Imagine this: less than half an organization’s structured data is used in decision-making. Think of the missed opportunities for customer acquisition and revenue by not taking advantage of that information. According to an IBM study, 87 percent of CEOs regard data as a strategic asset. So why then are companies not harnessing the power of this information?

The CDP Operational Database (COD) builds on the foundation of existing operational database capabilities that were available with Apache HBase and/or Apache Phoenix in legacy CDH and HDP deployments.

When I started out in my career, it was very much all about the numbers, and thinking, not feeling, when it came to communicating important results found in business data.

Spark is known for being extremely difficult to debug. But this is not all Spark’s fault. Problems in running a Spark job can be the result of problems with the infrastructure Spark is running on, inappropriate configuration of Spark, Spark issues, the currently running Spark job, other Spark jobs running at the same time – or interactions among these layers.