Microsoft Azure unveils collaboration with Databricks

Introducing Microsoft Azure Databricks, a collaboration between unified analytics company Databricks and tech giant Microsoft, that aims to help customers pull value from big data and AI.

Now generally available, Azure Databricks is Databricks' Unified Analytics Platform offered as an integrated service within the Azure cloud platform. The Unified Analytics Platform is powered by Apache Spark.

Databricks CEO and co-founder Ali Ghodsi comments on the collaboration, saying the collaboration with Microsoft builds on the company's momentum as a cloud platform for Apache Spark-based analytics.

"The ability to provide our Unified Analytics Platform to all Microsoft Azure users in such an integrated fashion is invaluable to end users looking to simplify big data and AI," says Ali.

Azure Databricks combines Databricks' Spark-based cloud service with Azure to power collaboration between data scientists, data engineers, and business analysts.

The integrated service provides Databricks' Unified Analytics Platform integrated with the Azure cloud platform, encompassing the Azure Portal; Azure Active Directory; and other data services on Azure, including Azure SQL Data Warehouse, Azure Cosmos DB, Azure Data Lake Store; and Microsoft Power BI.

From Microsoft, corporate vice president of Azure Data, Rohan Kumar calls the platform the "most efficient way for us to meet customer demand for Spark on Azure.

"We experienced a lot of interest and engagement in the preview from organizations in need of a high-performance analytics platform based on Spark," adds Kumar.

"With Azure Databricks, deeply integrated with services like Azure SQL Data Warehouse, our customers are now positioned to increase productivity and collaboration and globally scale analytics and data science projects on a trusted, secure cloud environment.

The platform already has a number of cross customers, including Lennox International, and E.ON.

The latter is an international private energy company, which, according to the company's data services product owner, Sam Julian, analyzes nearly a terabyte of wind turbine data every day.

"Before, that took several hours. With Microsoft Azure Databricks, it takes a few minutes. This opens a whole range of possible new applications.

Director of Information Technology at Lennox International, Sunil Bondalapati, also comments: "We supply WiFi enabled thermostats. With Microsoft Azure Databricks, we moved from 60% efficiency to 94 percent efficiency on detecting switch failures.

"Using Azure Databricks has opened the flood gates to all kinds of new use cases inside Lennox International. In our previous process, 15 devices in 15 homes, which create 2 million records, took 6 hours to process.

"With Azure Databricks, we are able to process 25,000 homes – 10 billion records – in 14 minutes.