Report: All clouds are not equal: differentiating an enterprise cloud

Our library of 1700 research reports is available only to our subscribers. We occasionally release ones for our larger audience to benefit from. This is one such report. If you would like access to our entire library, please subscribe here. Subscribers will have access to our 2017 editorial calendar, archived reports and video coverage from our 2016 and 2017 events.
cloud screen
All clouds are not equal: differentiating an enterprise cloud by Janakiram MSV:
Until recently startups and small to medium businesses drove cloud adoption. Now, with cloud service providers maturing, many businesses are considering migrating applications to the cloud. In particular, the IT industry has witnessed a rapid adoption of the public cloud. This research report examines the current state of the cloud, barriers to enterprise cloud adoption, and the key factors to consider when choosing an enterprise cloud platform.
To read the full report, click here.

Report: Best practices and technologies for cloud API management and governance

Our library of 1700 research reports is available only to our subscribers. We occasionally release ones for our larger audience to benefit from. This is one such report. If you would like access to our entire library, please subscribe here. Subscribers will have access to our 2017 editorial calendar, archived reports and video coverage from our 2016 and 2017 events.
Cloud Computing
Best practices and technologies for cloud API management and governance by David S. Linthicum:
Cloud computing’s continued growth has led to an increase in the use of APIs, which can abstract the complexities of back-end resources such as databases, platforms, storage, compute, and middleware. APIs provide developers and users with the ability to leverage and releverage services, allowing enterprise IT to construct applications from prebuilt component parts.
However, the number of APIs that exist within public cloud providers, or even around traditional enterprise systems, continues to increase well beyond most enterprises’ ability to manually manage them. What’s needed to address this is a new approach and a new technology to manage and govern these services. The use of API management and governance technology is the most logical path, but planning and deployment best practices are still required.
In this report, we’ll explore the concept of API management and governance, and look at its purpose and value as well as the details behind planning and deployment.
To read the full report, click here.

Why BI’s shift to stream intelligence is a top priority for CAOs

Nova is co-founder and CEO at Bottlenose.
A quick search on LinkedIn reveals thousands of professionals in the United States now hold the recently established title of Chief Analytics Officer (CAO). Analytics officers have ascended to the C-suite amongst a constellation of new roles and responsibilities that cut across departmental lines at Fortune 1,000 companies. These positions are driven by the influx of data with which companies now need to contend, across even industries that were not previously data-oriented.
The CAO’s role most closely aligns with business intelligence, leveraging data analytics to create real business value and inform strategic decisions. Yet, the CAO’s responsibilities also encompass discovering the various and constantly changing threats and opportunities impacting a business.
The most dramatic shift in data-driven business intelligence that has necessitated this role is the sheer volume, variety, and velocity of data now available to the enterprise. Data is no longer just static or historical, but real-time, streaming, unstructured, and abundant from both public and proprietary sources.
Unstructured data is the fastest growing category of data, and within it stream data – time-stamped series of records – is the fastest growing sub-category. Stream data spans messaging, social media, mobile data, CRM, sales, support, IT data, sensor and device data such as the emerging internet-of-things, and even live video and audio.
The CAO’s charge is to enable the enterprise to deal with all of this data and generate timely, actionable intelligence from it – increasingly in real-time. I’ve been calling this process of business intelligence for streaming data “stream intelligence” for a while now. Among the dozens of CAOs I’ve spoken with recently, moving from classical business intelligence on static data to stream intelligence is one of their biggest priorities for 2016. This emerging form of BI creates unique problems for enterprise companies, but it also creates unique opportunities for those companies to discern and discover trends early, while there is still time to act on them.

Understanding BI 3.0

Thomas Davenport is a professor at Babson College, a research fellow at the MIT Center for Digital Business, and a senior advisor to Deloitte Analytics. He has written eloquently about these topics since 2010 and offers a framework for thinking about the past, present, and future of analytics.
For Davenport, BI 1.0 was about traditional analytics, providing descriptive reporting from relatively small internally sourced data. It was about back-room teams and internal decision reports.
BI 2.0 was about complex, much larger unstructured data sources. It was also about new computational capabilities that ran on top of traditional analytics. With big data, we saw data scientists first emerge, alongside several waves of new data-based products and services. This is where we are today.
BI 3.0 is about rapid, agile insight delivery – analytical tools at the point of decision, and decision making at scale. Today, analytics are considered a key asset enabling strategic decision making, not merely a mirror reflecting an organization’s past and present.
The “how” of accomplishing this vision amounts to balancing support for the “three V’s” of data — volume, variety, velocity — in the enterprise analytics stack. Most big data and BI technologies to-date were engineered to solve volume and variety, with very little emphasis placed on the velocity of data and analytics. This has to change.
Analysts are already drowning in volume, variety, and velocity of data. To make matters worse, the rate at which new analysts are being trained is far less than the growth rate of demand for analysts and data scientists. In fact, the gap between the supply of analyst hours and the demand for analyst cycles is growing exponentially. This means that there will never be enough data scientists to cope with the rise of unstructured stream data in the enterprise.
To solve the growing “analyst gap” we either need to figure out how to make exponentially more analysts, or we have to figure out how to make the finite supply of analysts exponentially more productive. I prefer the latter solution, but to accomplish it, analysts need automation.
Manual analysis by humans is still possible for structured data, but not for streaming data. Streaming data is just too complex, and changes too fast for human analysts to keep up with on their own. Automation is the only practical way to keep up with changing streams of big data.
BI 3.0 emphasizes real-time business impact and makes use of automation in the analytics process. This will increasingly be achieved with a seamless blend of traditional analytics and big data. BI 3.0 analytics are now integral to running the business day-to-day and hour-to-hour.

Following the flow of big data investment

I’ll close by talking about where big data investment dollars are starting to go. In short, real-time stream data is now a major priority, and historical data is now riding in the back seat.
According to Harvard Business Review, 47 percent of expenditures are directed towards process improvement. 26 percent are directed towards accommodating a greater variety of data, and 16 percent address a greater volume of data. Velocity of data today represents a very small slice, at three percent of overall investment, but that slice will grow quickly in 2016.
In fact, organizations that have prioritized real-time data are outpacing all others, according to Aberdeen Group. Companies that are competent across volume, variety, and velocity alike have seen 26 percent growth in their pipelines, 15 percent increase in cash generated, and 67 percent in operational cost reduction.
It’s hard to argue with those numbers. CAOs understand that BI 3.0 is happening now, which is why it’s become a top priority for 2016.

The BI conundrum: Delivering trust and transparency at speed

Brad is chairman and chief product officer at Birst.
Historically, the pendulum of the business intelligence (BI) market has moved between centralized governance of data and self-service and agility. Today, the industry is swaying back and forth, as CIOs struggle to find the right balance of control, transparency, and truth.
Recently, the pendulum has swung too far in the direction of data discovery. While data discovery tools provide speedy data discovery and manipulation, these tools can create analytical silos that hinder the ability of users to make decisions with confidence.
These self-service capabilities create challenges for CIOs, according to Gartner, Inc. Without proper processes and governance in place, self-service tools can introduce multiple versions of the truth, increase errors in analysis and result in inconsistent information.

Is ‘imperfect but fast’ a fair trade-off?

That said, CIOs have come to accept data inconsistency as the price to pay – a tradeoff to achieve speed – to give business users the ability to analyze data without depending on a central BI team. Both parties seem to have adopted the maxim “imperfect but fast is better than perfect but slow.” But is this price too much? The answer is worth delving into.
Backlashes include siloed and inconsistent views of key metrics and data across groups. For instance, lead-to-cash analysis requires data from three different departments (Marketing, Sales, and Finance) and three separate systems (marketing automation, CRM, ERP). A consistent and reliable view of the information between departments and systems – one that provides a common definition of “Lead” or “Revenue” – is necessary to avoid confusion and conflicting decisions.
Finding consistency with data-discovery tools requires the daunting task of manually delivering a truly governed layer of data without a comprehensive understanding of core business logic. This means having the ability to build and test integrated data models, tools for performing extraction, transformation and loading (ETL) routines across corporate systems, channels for proliferating enterprise-wide metadata, and a demand for governance-centric business procedures. All of which are a burden to CIOs.

The end goal: Transparency and speed

Trusted data does not have to be synonymous with restrictive access and long wait times. By implementing transparent governance, CIOs can enable local (decentralized) execution with global (centralized) consistency, reconciling speed with trust at enterprise scale.
But to deliver the agility of data discovery with enterprise governance, CIOs must work to do the following:

  • Adopt a data-driven culture. CIOs need to create a team approach to BI that balances the use of skilled resources and the development of more localized business skills to deliver ongoing success.
  • Enable data access to all business users. Creating protocols to access new datasets ensures that all business users can identify opportunities that add value. Multiple layers of security during discovery and consumption are crucial to uphold security. With simple and secure access, users can easily identify opportunities to derive insights.
  • Create a consistent understanding and interpretation of data. Certify and manage key input datasets and governing information outputs to help align organizational accountability for data discovery. A single view of governed measures and dimensions, for users in both decentralized and centralized use cases, ensures consistency.

Following the aforementioned points to deliver trust and transparency results in big gains. According to Dresner Advisory Services, organizations that view data as a single truth with common rules are nearly 10 times more likely to achieve BI success than organizations with multiple inconsistent sources.
There is a powerful and direct correlation between business success and having a trusted view of enterprise data. Companies evaluating BI solutions must look for modern architectures that support transparent governance at business speed and deliver a unified view of data without sacrificing end-user autonomy. The companies that do will continue to win.

The rise of self-service analytics, in 3 charts

I’m trying really hard to write less about business intelligence and analytics software. We get it: Data is important to businesses, and the easier you can make it for people to analyze it, the more they’ll use your software to do it. What more is there to say?

But every time I see Tableau Software’s earnings reports, I’m struck by the reality of how big a shift the business intelligence market is undergoing right now. In the fourth quarter, Tableau grew its revenue 75 percent year over year. People and departments are lining up to buy what’s often called self-service analytics software — that is, applications so easy even those lay business users can work with them without much training — and they’re doing it at the expense of incumbent software vendors.

Some analysts and market insiders will say the new breed of BI vendors are more about easy “data discovery” and that their products lack the governance and administrative control of incumbent products. That’s like saying Taylor Swift is very cute and very good at making music people like, but she’s not as serious as Alanis Morrisette or as artistic as Björk. Those things can come in time; meanwhile, I’d rather be T-Swift raking in millions and looking to do it for some time to come.

[dataset id=”914729″]

Above a quick comparison of annual revenue for three companies, the only three “leaders” in Gartner’s 2014 Magic Quadrant for Business Intelligence and Analytics Platforms (available in the above hyperlink) that are both publicly traded and focused solely on BI. Guess which two fall into the next-generation, self-service camp and are also Gartner’s two highest-ranked. Guess which one is often credited with reimagining the data-analysis experience and making a product people legitimately like using.

[dataset id=”914747″]

Narrowing it just to last year, Tableau’s revenue grew 92 percent between the first and fourth quarters, while Qlik’s grew 65 percent. Microstrategy stayed relatively flat and is trending downward. It’s fourth quarter was actually down year over year.

[dataset id=”914758″]

And what does Wall Street think about what’s happening? [company]Tableau[/company] has the least revenue for now, but probably not much longer, and has a market cap more than [company]Qlik[/company] and [company]Microstrategy[/company] combined.

Here are a few more data points that show how impressive’s Tableau’s ongoing coup really is. Tibco Software, another Gartner leader and formerly public company, recently sold to private equity firm Vista for $4.2 billion after disappointing shareholders with weak sales. Hitachi Data Systems is buying Pentaho, a BI vendor hanging just outside the border of Gartner’s “leader” category, for just more than $500 million, I’m told.

A screenshot from a sample PowerBI dashboard.

A screenshot from a sample PowerBI dashboard.

Although it’s worth noting that Tableau isn’t guaranteed anything. As we speak, startups such as Platfora, ClearStory and SiSense trying to match or outdo Tableau on simplicity while adding their own new features elsewhere. The multi-billion-dollar players are also stepping up their games in this space. [company]Microsoft[/company] and [company]IBM[/company] recently launched the natural-language-based PowerBI and Watson Analytics services that Microsoft says represent the third wave of BI software (Tableau is in the second wave, by its assessment), and [company]Salesforce.com[/company] invested a lot of resources to make its BI foray.

Whatever you want to call it — data discovery, self-service analytics, business intelligence — we’ll be talking more about it at our Structure Data conference next month. Speakers include Tableau Vice President of Analytics (and R&D leader) Jock Mackinlay, as well as Microsoft Corporate Vice President of Machine Learning Joseph Sirosh, who’ll be discussing self-service machine learning.

Hitachi Data Systems to buy Pentaho for $500-$600M

Storage vendor Hitachi Data Systems is set to buy analytics company Pentaho at a price rumored to be between $500 million and $600 million (closer to $500 million, from what I’ve heard). It’s an interesting deal because of its size and because Hitachi wants to move beyond enterprise storage and into analytics for the internet of things. Pentaho sells business intelligence software, and can transform even big data from stream-processing engines for real-time analysis. According to a Hitachi press release, “The result will be unique, comprehensive solutions to address specific challenges through a shared analytics platform.”

Microsoft throws down the gauntlet in business intelligence

[company]Microsoft[/company] is not content to let Excel define the company’s reputation among the world’s data analysts. That’s the message the company sent on Tuesday when it announced that its PowerBI product is now free. According to a company executive, the move could expand Microsoft’s reach in the business intelligence space by 10 times.

If you’re familiar with PowerBI, you might understand why Microsoft is pitching this as such a big deal. It’s a self-service data analysis tool that’s based on natural language queries and advanced visualization options. It already offers live connections to a handful of popular cloud services, such as [company]Salesforce.com[/company], [company]Marketo[/company] and GitHub. It’s delivered as a cloud service, although there’s a downloadable tool that lets users work with data on their laptops and publish the reports to a cloud dashboard.

James Phillips, Microsoft’s general manager for business intelligence, said the company has already had tens of thousands of organizations sign up for PowerBI since it became available in February 2014, and that CEO Satya Nadella opens up a PowerBI dashboard every morning to track certain metrics.

A screenshot from a sample PowerBI dashboard.

A screenshot from a sample PowerBI dashboard.

And Microsoft is giving it away — well, most of it. The preview version of the cloud service now available is free and those features will remain free when it hits general availability status. At that point, however, there will also be a “pro” tier that costs $9.99 per user per month and features more storage, as well as more support for streaming data and collaboration.

But on the whole, Phillips said, “We are eliminating any piece of friction that we can possibly find [between PowerBI and potential users].”

This isn’t free software for the sake of free software, though. Nadella might be making a lot of celebrated, if not surprising, choices around open source software, but he’s not in the business of altruism. No, the rationale behind making PowerBI free almost certainly has something to do with stealing business away from Microsoft’s neighbor on the other side of Lake Washington, Seattle-based [company]Tableau Software[/company].

Phillips said the business intelligence market is presently in its third wave. The first wave was technical and database-centric. The second wave was about self service, defined first by Excel and, over the past few years, by Tableau’s eponymous software. The third wave, he said, takes self service a step further in terms of ease of use and all but eliminates the need for individual employees to track down IT before they can get something done.

The natural language interface, using funding data from Crunchbase.

The natural language interface, using funding data from Crunchbase.

IBM’s Watson Analytics service, Phillips said, is about the only other “third wave” product available. I recently spent some time experimenting with the Watson Analytics preview, and was fairly impressed. Based on a quick test run of a preview version of PowerBI, I would say both products have their advantages over the other.

But IBM — a relative non-entity in the world of self-service software — is not Microsoft’s target. Nor, presumably, is analytics newcomer Salesforce.com. All of these companies, as well as a handful of other vendors that exist to sell business intelligence software, want a piece of the self-service analytics market that Tableau currently owns. Tableau’s revenues have been skyrocketing for the past couple years, and it’s on pace to hit a billion-dollar run rate in just over a year.

“I have never ever met a Tableau user who was not also a Microsoft Excel user,” Phillips said.

That might be true, but it also means Microsoft has been leaving money on the table by not offering anything akin to Tableau’s graphic interface and focus on visualizations. Presumably, it’s those Tableau users, and lots of other folks for whom Tableau (even its free Tableau Public version) is too complex, that Microsoft hopes it can reach with PowerBI. Tableau is trying to reach them, too.

“We think this really does 10x or more the size of the addressable business intelligence market,” Phillips said.

A former Microsoft executive told me that the company initially viewed Tableau as a partner and was careful not to cannibalize its business. Microsoft stuck to selling SharePoint and enterprise-wide SQL Server deals, while Tableau dealt in individual and departmental visualization deals. However, he noted, the new positioning of PowerBI does seem like a change in that strategy.

Analyzing data with more controls.

Analyzing data with more controls.

Ultimately, Microsoft’s vision is to use PowerBI as a gateway to other products within Microsoft’s data business, which Phillips characterized the the company’s fastest-growing segment. PowerBI can already connect to data sources such as Hadoop and SQL Server (and, in the case of the latter, can analyze data without transporting it), and eventually Microsoft wants to incorporate capabilities from its newly launched Azure Machine Learning service and the R statistical computing expertise it’s about to acquire, he said.

“I came to Microsoft largely because Satya convinced me that the company was all in behind data,” Phillips said. For every byte that customers store in a Microsoft product, he added, “we’ll help you wring … every drop of value out of that data.”

Joseph Sirosh, Microsoft’s corporate vice president for machine learning, will be speaking about this broader vision and the promise of easier-to-use machine learning at our Structure Data conference in March.

Microsoft CEO Satya Nadella.

Microsoft CEO Satya Nadella.

Given all of its assets, it’s not too difficult to see how the new, Nadella-led Microsoft could become a leader in an emerging data market that spans such a wide ranges of infrastructure and application software. Reports surfaced earlier this week, in fact, that Microsoft is readying its internal big data system, Cosmos, to be offered as a cloud service. And selling more data products could help Microsoft compete with another Seattle-based rival — [company]Amazon[/company] Web Services — in a cloud computing business where the company has much more at stake than it does selling business intelligence software.

If it were just selling virtual servers and storage on its Azure platform, Microsoft would likely never sniff market leader AWS in terms of users or revenue. But having good data products in place will boost subscription revenues, which count toward the cloud bottom line, and could give users an excuse to rent infrastructure from Microsoft, too.

Update: This post was updated at 10:15 a.m. to include additional information from a former Microsoft employee.

Sort-of-stealthy Domo gets ready for its closeup

Looks as if Domo, the business intelligence startup founded by former Omniture CEO Josh James, is getting ready to be more chatty about just what its product does and what it’s been doing with the $250 million it’s raised.

It announced plans for Domopalooza, a customer conference to take place in Salt Lake City in April. There’s not a ton known about the product except that it gathers data from all of a company’s existing applications — [company]Salesforce.com[/company] CRM, [company]SAP[/company] ERP, 300 of them in all — to help it react faster to changes or errors in its processes.

Domo CEO Josh James also talked a bit to Re/code. Per that story, Domo is

a big software platform that pulls in all of a company’s operational data, creating a live view of pretty much every aspect of its operations — inventory, manufacturing, the amount of needed supplies, who’s being paid and who’s paying, who the employees are and what they’re being paid.

Which, in truth is very little more than what James told the Wall Street Journal last February (paywall).

At that time, James said Domo can start by:

[showing] someone data about their company, bringing in data from Salesforce and Concur and SAP and looking at the data. The next thing they want is to ask questions, so you need social. Then they want to understand who people are … and it expands and grows from there. We feel like we’re doing seven or eight unique things, and I don’t think there’s a company that has more than one or two.

Domo, founded in 2010, claims 1,000 customers — all of which had to sign non-disclosure agreements about their use of its software. That’s a pretty impressive tally, although a  startup’s definition of “customer” can be pretty loosey-goosey.

Still, James has a lot of credibility. Omniture, which Adobe bought for $1.8 billion in 2009, offers a popular tool web analytics tool.

Domo, based in American Fork, Utah, is one of several new-look business intelligence startups — a cadre that also includes Thoughtspot, Looker, Chartio and others. Since it reportedly has 600 employees, it’s sort of hard to see it as a startup.

Hands on with Watson Analytics: Pretty useful when it’s working

Last month, [company]IBM[/company] made available the beta version of its Watson Analytics data analysis service, an offering first announced in September. It’s one of IBM’s only recent forays into anything resembling consumer software, and it’s supposed to make it easy for anyone to analyze data, relying on natural language processing (thus the Watson branding) to drive the query experience.

When the servers running Watson Analytics are working, it actually delivers on that goal.

Analytic power to the people

Because I was impressed that IBM decided to a cloud service using the freemium business model — and carrying the Watson branding, no less — I wanted to see firsthand how well Watson Analytics works. So I uploaded a CSV file including data from Crunchbase on all companies categorized as “big data,” and I got to work.

Seems like a good starting point.

watson14Choose one and get results. The little icon in the bottom left corner makes it easy to change chart type. Notice the various insights included in the bar at the top. Some are more useful than others.

watson15But which companies have raised the most money? Cloudera by a long shot.

watson18

I know Cloudera had a huge investment round in 2014. I wonder how that skews the results for 2014, so I filter it out.

watsonlast

And, voila! For what it’s worth, Cloudera also skews funding totals however you sort them — by year founded, city, month of funding, you name it.

watsonlast2

Watson analytics also includes tools for building dashboards and for predictive analysis. The latter could be particularly useful, although that might depend on the dataset. I analyzed Crunchbase data to try and determine what factors are most predictive of a company’s operating status (whether it has shut down, has been acquired or is still running), and the results were pretty obvious (if you can’t read the image, it lists “last funding” as a big predictor).

watsonpredict3

If I have one big complaint about Watson Analytics, it’s that it’s still a bit buggy — the tool to download charts as images doesn’t seem to work, for example, and I had to reload multiple pages because of server errors. I’d be pretty upset if I were using the paid version, which allows for more storage and larger files, and experienced the same issues. Adding variables to a view without starting over could be easier, too.

Regarding the cloud connection, I rather like what [company]Tableau[/company] did with its public version by pairing a locally hosted application with cloud-based storage. If you’re not going to ensure a consistent backend, it seems better to guarantee some level of performance by relying on the user’s machine.

All in all, though, Watson Analytics seems like a good start to a mass-market analytics service. The natural language aspect makes it at least as intuitive as other services I’ve used (a list that includes DataHero, Tableau Public and Google Fusion tables, among others) and it’s easy enough to run and visualize simple analyses. But Watson Analytics plays in a crowded space that includes the aforementioned products, as well as Microsoft Excel and PowerBI, and Salesforce Wave.

If IBM can work out some of the kinks and add some more business-friendly features — such as the upcoming abilities to refine datasets and connect to data sources — it could be onto something. Depending on how demand for mass-market analytics tools shapes up, there could be plenty of business to go around for everyone, or a couple companies that master the user experience could own the space.

Tableau hits the $100M revenue mark in third quarter

Tableau continues to grow at a fast pace, hitting the $100 million mark for the first time in the third quarter. Although there are plenty of startups willing to point out its weaknesses, Tableau has lots of room to grow and the money to do it.