Let’s try to figure out what happens with the application when the source file is much bigger than the available memory. The memory in the below tests is limited to 900MB […]. Naively we could think that a file bigger than available memory will fail the processing with OOM memory error. And this supposition is true.

Once upon a time there was only one way to use Apache Spark but support for additional programming languages and APIs have been introduced in recent times. A novice can be confused by the different options that have become available since Spark 1.6 and intimidated by the idea of setting up a project to explore these APIs.

RELATED BLOG POSTS Unraveling the Complex Streaming Data Pipelines of Cybersecurity Best Practices Blog 5 Min Read Security is top of mind for every enterprise these days. There are so many threats they can hardly be counted, but one commonality exists: data is always the target. Unravel’s mission is to help organizations better understand and improve the performance of their data-based applications. We’re a data business, so we appreciate the scope and implications of these threats.

A very important question is how long something takes and the answer to that is fairly straightforward in normal life: Check the current time, then perform the unit of work that should be measured, then check the time again. The end time minus the start time would equal the amount of time that the task took, the elapsed time or wallclock time.

At Unravel, we have been working with AWS for some time to help our joint customers simplify data migrations to the cloud. Announced at AWS re:Invent, we are proud to be partnering with AWS on the newly created AWS Partner Network Global Startup Program. Unravel provides detailed insights and recommendations to help enterprises of all sizes slice and dice application workloads to ensure an effective cloud migration strategy.

Unsurprisingly, Modern data apps have become crucial across all areas of the financial industry, with apps for fraud detection, claims processing and compliance amongst others playing a business critical role. Unravel has been deployed by several of the world’s largest financial services organizations to ensure these critical apps perform reliably at all times.

As technology evolves at a rapid pace, the healthcare industry is transforming quickly along with it. Tech breakthroughs like IoT, advanced imaging, genomics mapping, artificial intelligence and machine learning are some of the key items re-shaping the space. The result is better patient care and health outcomes.

Over the last few years, there has been a mad rush within enterprise organizations to move big data workloads to the cloud. On the surface, it would seem that it’s a bit of a keeping up with the Joneses syndrome with organizations moving big data workloads to the cloud simply because they can.

We are very excited to announce that Unravel is now available for Google Cloud Dataproc. If you are already on GCP Dataproc or on a journey to migrate on-premises data workloads to GCP Dataproc then Unravel is now available immediately to accelerate and de-risk your Dataproc migration and ensure performance SLAs and cost targets are achieved.

We are super excited to announce our support for Azure Databricks! We continue to build out the capabilities of the Unravel Data Operations platform and specifically support for the Microsoft Azure data and AI ecosystem teams. The business and technical imperative to strategically and tactically architect the journey to cloud for your organization has never been stronger.