I’m a geneticist, which is really just a technical way of saying that I obsess about the minutiae of your family history. Now, while that sounds rather stalkerish, this is in fact a molecular and data-driven addiction, which is fuelled by my need to understand the nature of evolution and human behaviour via the data within our DNA. Our genomes are the single-most densely packed dataset that we have ever encountered.
The “Quality at Speed” movement – or delivering high-quality products in a short period – has expanded beyond the software industry: it appears in the standard playbook of companies in health care, finance, etc. This new movement pushes QA teams to continuously reinvent their software development cycle with advancing technological practices.
Do you need faster time to value? Does your organization’s success depend on immediate delivery of new reports, applications, or projects? When you go to Central IT for support, are you blocked by insanely long wait times for the resources needed to meet your business goals? If so – you are likely one of the growing group of Line of Business (LoB) professionals forced into creating your own solution – creating your own Shadow IT.
These bundled analytics tools help organizations facilitate and increase the adoption of self-service BI practices among regular business users in a specific operational domain, such as finance, marketing and sales. It does so by improving the availability and measurement of important, relevant historical data for your end users’ decision-making.
Modern applications don’t function in isolation. To get the most out of the enterprise apps you build or buy, you’ll have to connect them to other applications. In other words, data engineers have to engage in effective application integration to achieve their business goals. Sometimes, this means connecting one application directly to another. But this is a rare occurrence in digitally transformed industries.
From system logs to web scraping, there are many good reasons why you might have extremely large numbers of small data files at hand. But how can you efficiently process and analyze these files to uncover the hidden insights that they contain? You might think that you could process these small data files using a solution like Apache Hadoop, which has been specifically designed for handling large datasets.
In this last installment, we’ll discuss a demo application that uses PySpark.ML to make a classification model based off of training data stored in both Cloudera’s Operational Database (powered by Apache HBase) and Apache HDFS. Afterwards, this model is then scored and served through a simple Web Application. For more context, this demo is based on concepts discussed in this blog post How to deploy ML models to production.
Digital transformation is a hot topic for all markets and industries as it’s delivering value with explosive growth rates. Consider that Manufacturing’s Industry Internet of Things (IIOT) was valued at $161b with an impressive 25% growth rate, the Connected Car market will be valued at $225b by 2027 with a 17% growth rate, or that in the first three months of 2020, retailers realized ten years of digital sales penetration in just three months.
