As fraud continues to grow in prevalence, SecOps teams are increasingly investing in fraud prevention capabilities to protect themselves and their customers. One approach that’s proved reliable is the use of log analytics and telemetry data for fraud prevention. By collecting and analyzing data from various sources, including server logs, network traffic, and user behavior, enterprise SecOps teams can identify patterns and anomalies in real time that may indicate fraudulent activity.

This recent article from Finextra regarding the UK payments watchdog consulting on new authorized push payment (APP) fraud reporting rules has me thinking. How can we prevent fraud and protect consumers – and ourselves from scams? APP fraud happens when fraudsters deceive consumers or individuals into sending a payment under false pretenses to a bank account controlled by the fraudster. Real-time payment schemes are irrevocable, so victims cannot reverse a payment once sent.

In part 1 of this blog we discussed how Cloudera DataFlow for the Public Cloud (CDF-PC), the universal data distribution service powered by Apache NiFi, can make it easy to acquire data from wherever it originates and move it efficiently to make it available to other applications in a streaming fashion.

In a previous blog of this series, Turning Streams Into Data Products, we talked about the increased need for reducing the latency between data generation/ingestion and producing analytical results and insights from this data. We discussed how Cloudera Stream Processing (CSP) with Apache Kafka and Apache Flink could be used to process this data in real time and at scale. In this blog we will show a real example of how that is done, looking at how we can use CSP to perform real-time fraud detection.

Much of the hype around big data and analytics focuses on business value and bottom-line impacts. Those are enormously important in the private and public sectors alike. But for government agencies, there is a greater mission: improving people’s lives. Data makes the most ambitious and even idealistic goals—like making the world a better place—possible.

Across the federal government, agencies are struggling to identify, organize, analyze, and act on troves of data. It’s a problem that leaders are working actively to tackle, but they’re in a race against immeasurable volumes of data that is continuously being generated in perpetuity in stores known and unknown. At the Internal Revenue Service, decades’ worth of data exceeds even the most cutting-edge processing capabilities.