Systems | Development | Analytics | API | Testing

Ingesting Google Cloud Storage files to BigQuery using Cloud Functions and Serverless Spark

Apache Spark has become a popular platform as it can serve all of data engineering, data exploration, and machine learning use cases. However, Spark still requires the on-premises way of managing clusters and tuning infrastructure for each job.

Do Data Companies Need Chief Ethics Officers?

Sometimes it takes a billion-dollar mistake to bring the murkier side of data ethics into sharp focus. Equifax found this out to their own cost in 2017 when they failed to protect the data of almost 150 million users globally. The catastrophic breach was bad enough on its own — but Equifax waited three months to go public with the news. As the public furore rose to a crescendo, the credit organization dragged its feet on disclosing exactly what kind of information had been leaked.

The Best Integration Platform for Your Business

The crucial metrics for evaluating the best integration platform for businesses are: With the right integration platform in your hands, your IT environment will be faster, more efficient and better connected. However, finding the best integration platform for your businesses is no easy feat. There are many integration platform options for organizations with different needs, from small businesses to massive corporations.

SNCF: Getting the customer experience right

As a major railway operator, SNCF has a whole lot of data on its hands, including more than 50 million customer records. The first step in using that data more effectively was integrating and consolidating all of its diverse data sources and databases into a single Customer Platform. In this video, Yann Jourdain, CIO at SNCF Voyages, explains how he made it happen.