DATA TO DECISIONS

Doug Henschen on Analytics, Big Data & Smart Apps

SAP Taps Hadoop & Spark For IoT Innovation

SAP signaled a deeper embrace of big data platforms on Wednesday at SAPPHIRE with the announcement that it will create a deeper integration with Hadoop using Apache Spark as a low-latency, in-memory connector and data-filtering mechanism.

The announcement, made by SAP Executive Board Member Berndt Leukert, signals that SAP is counting on Hadoop as a high-scale storage container in Internet-of-Things (IoT) deployments and other scenarios where data-collection and data-processing needs are geographically distributed. Spark, the open-source, in-memory streaming and analysis platform, will be used as an intelligent processing engine to provide accelerated access to data in Hadoop, said Leukert.

SAP's Berndt Leukert discusses new uses of Hadoop and Spark at Sapphire.

SAP’s Berndt Leukert discusses new uses of Hadoop and Spark at Sapphire.

In an oil & gas and utility IoT deployments, for example, diverse sets of sensors in far-flung locations stream out readings at high scale. Hadoop provides high-scale, low-cost storage. Spark would provide low-latency connectivity to that data. But sensor data is diverse, lacking standardized formats, so Spark would also be used to filter, transform and harmonize the data, sending only the crucial signals on to a core IoT app running on the Hana Cloud Platform.

SAP has long had Hana data connectors for Hadoop, and it also has a Spark connector certified by Databricks, the commercial development-and-support company behind Spark. The architecture announced Wednesday signals a deeper level of integration and a deeper commitment to work with these platforms. Irfan Khan, senior vice president and general manager of SAP Big Data, confirmed that in a briefing with Constellation Research, and he noted that the edge-to-core processing approach could also be applied in non-IoT scenarios such as retail price optimization.

SAP has yet to flesh out the exact details of its new architecture, but David Parker, global VP, SAP Platform Solutions, told Constellation that SAP stream processing and mobile database technologies will also figure in the architecture, with lightweight choices at the edge and SAP Hana at the core.

MyPOV: There’s no shortage of big vendors declaring their intention to support IoT deployments these days, but SAP’s announcement shows that it has thought through the workloads and bottlenecks likely to be encountered in real-world IoT deployments. (Indeed, it’s pretty clear that co-innovation customers such as Siemens have played a significant role in evolving SAP’s thinking.) And what’s good for IoT will surely bring advances in other high-scale, real-time applications. It’s a welcome sign of maturation that points toward interesting and innovative deployments ahead.

One comment on “SAP Taps Hadoop & Spark For IoT Innovation

  1. Doug Henschen
    May 7, 2015

    Hasso Plattner revealed today that the lightweight embedded in-memory engine recently introduced in Lumira Server for the BI Platform is the same engine that will be used in highly distributed IoT deployments where analysis is needed at the edge of the network. It was developed by the same crew that developed the TRex database, so the project name was “Velocity Raptor,” but SAP CTO Quentin Clark kind of cringed when he heard that name and interjected that the engine has yet to be named.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Information

This entry was posted on May 7, 2015 by in analytics, big data, in-memory, real-time and tagged , , , .
%d bloggers like this: