Microsoft adds stream processing and pipeline tools to Azure

Microsoft continued its rollout of new Azure cloud services on Wednesday, with a trio of features to help users get a better handle on their data. Two of the new features — Stream Analytics and Event Hubs — deal with the processing of data in real time while the third, Data Factory, lets users visually diagram how data moves from one store to another and what happens to it at each step.

Microsoft’s biggest cloud competitors, [company]Amazon[/company] Web Services and [company]Google[/company], already have their own stream-processing and pipeline services, so they’re really more like table stakes at this point than they are true points of distinction — except, of course, to the degree that any one is better than the others. [company]Microsoft[/company] probably has the strongest hybrid cloud story among the three companies, which might make for more natural connections as pipelines span cloud and local data stores, but the AWS pipeline tool works with local sources as well.

Source: Microsoft

Source: Microsoft

The alternative for companies that don’t want to use cloud-provider-specific tools, at least for handling streaming data, is to use some combination of open-source technologies including Kafka, Storm, Samza and/or Spark Streaming. These are very popular among large web companies such as LinkedIn and [company]Netflix[/company], which prefer open-source data storage, analysis and transfer technologies even if (in the case of Netflix) they’re running on public clouds.

As an industry observer, it’s fun to watch how cloud computing has evolved from a point where [company]Amazon[/company] was basically competing against itself on the innovation front (and doing a heck of a job at it) to a point where it, Microsoft and Google are in a perpetual competition to one-up each other. Last week, Microsoft announced a handful of new cloud features, including a very beefy virtual machines and persistent storage options, and a hardware appliance that mimics Azure inside customer data centers. Next month, it’s AWS’s turn to show off what it has been working on at its annual re:Invent conference.