Xen security issue prompts Amazon, Rackspace cloud reboots

Amazon Web Services and Rackspace are warning their customers of upcoming reboots they’re taking to address a new Xen hypervisor security issue.

In a premium support bulletin issued Thursday night, Amazon said fewer than 10 percent of all EC2 instances will require work but the affected instances must be updated by March 10. [company]Rackspace[/company] also notified customers of the issue, which will affect a subset of a portion of its First and Next Generation Cloud Servers, Thursday night. Later on Friday, Linode also warned users of an upcoming Xen-related reboot.

If you’re sensing a little bit of deja vu, it’s because the major cloud players were forced to reboot a bunch of their customers in September due to a Xen hypervisor issue, although the reason for the updates was not disclosed at first. Last time out, AWS also said 10 percent of its EC2 instances were affected.

Cloud vendors impacted by these security issues tread a tricky path. They have to address the vulnerability as fast as possible before the details of the flaw are made public, which can lead to a bit of a fire drill. In this case, more information about the flaw will be disclosed March 10.

In September, [company]Amazon[/company] was first out of the chute with notifications, followed by Rackspace and then IBM Softlayer made its disclosures the following week.

Note: This story was updated at 3:49 p.m. PST to note that Linode is also performing system updates.

CIA’s secret Amazon cloud ready to roll

The Amazon Web Services-based cloud for the CIA is near completion, according to the CIA’s chief information officer (CIO).

On Wednesday, Doug Wolfe told a Cloudera industry gathering that the agency’s long-awaited AWS cloud has hit “final operational capability”  after 18 months of work, EnterpriseTech reported.

[company]Amazon[/company] famously prevailed with its  bid for this CIA cloud contract in late 2012 when it beat out [company]IBM[/company] even though it bid a higher dollar amount.  This $600 million deal was huge for AWS, which wants to show security- and compliance-conscious industries that its public cloud infrastructure can meet their special needs.

Since 2011, Amazon had already fielded GovCloud, a separate cloud region that meets special criteria so it can be used by government agencies. The CIA has additional requirements above and beyond that.

Wolfe also said the new cloud would be rolled out across 17 different intelligence agencies (news to me), and that the agency would have Cloudera’s enterprise data hub, running on the AWS service within months, according to the story.

 

Cloud options mean decisions, decisions for IT buyers

Much has been written about cloud consolidation, with M&A roiling the cloudscape over the past few months: Cisco bought Metacloud, EMC bought CloudscalingHP snapped up Eucalyptus. Despite all that, cloud deployment options abound, and choice will be a big theme at the upcoming Structure 2015 event, this June in San Francisco.

First, there is more choice than ever in public cloud. Sure, Amazon Web Services leads the market-share race by a wide margin. But viable options are available — from Microsoft Azure to Google Cloud Platform to vCloud Air to Digital Ocean to CenturyLink. What many of us tend to forget is that, despite all the cloud talk, we’re still very early in the game when it comes to business deployment. There’s a ton of opportunity out there. Is it enough to float all boats? That’s the zillion-dollar question.

We will discuss those options, and how even the biggest enterprises — General ElectricWalmart — are deploying more of their IT on cloud. The question is no longer if, but when.

At this year’s event, we’ll welcome back [company]Amazon[/company] CTO Werner Vogels, Khosla Ventures founder Vinod Khosla, [company]Microsoft[/company] EVP Scott Guthrie, Google SVP Urs Hölzle, Battery Ventures technology fellow Adrian Cockcroft and DataGravity CEO Paula Long.

We’ll hear from first-timers, too: Canonical founder Mark Shuttleworth, Digital Ocean CEO Ben Uretsky, CoreOS CEO Alex Polvi. And, on the end user side, we’re really excited to bring on stage National Football League CIO Michelle McKenna-Doyle, FBI CISO Arlette Hart and Pinterest head of engineering Michael Lopp. More names to come.

For a refresher of last year’s event, here’s a sampling of some favorite sessions:

Google’s Urs Holzle:

[youtube https://www.youtube.com/watch?v=I9R4P0TLViA]

Facebook’s Jay Parikh:

[youtube https://www.youtube.com/watch?v=F9FYTbxWK1o]

Intel SVP Diane Bryant:

[youtube https://www.youtube.com/watch?v=HTXuwqLUw7M]

Amazon’s Werner Vogels:

[youtube https://www.youtube.com/watch?v=oZPlr2-KMnw]

Microsoft’s Scott Guthrie:

[youtube https://www.youtube.com/watch?v=TImzXnUaO0A]

Kubernetes comes to OpenStack this time thanks to Mirantis

For businesses wanting to run the Kubernetes cluster management framework for containers on OpenStack clouds, Google and Mirantis have teamed up to make that happen more easily.

The OpenStack Murano application catalog technology promises to ease deployment of Kubernetes clusters on OpenStack and then deploy Docker containers on those clusters.

Murano provides what Mirantis CEO Adrian Ionel (pictured above) described as a “seamless point-and-click experience” not only for deploying workloads to OpenStack, but also making sure they get there with associated automation, provisioning and security intact. “In this case we use it to automate the provisioning and life cycle management of containers,” he said.

Murano, he added, makes it easier for people to build application environments that can be container-only, or mix containers with bare metal and virtual machines in one big happy package. (I’m paraphrasing here.)

This is not the industry’s first attempt to bring Kubernetes technology, open sourced by Google last year, over to OpenStack. In August, [company] Hewlett-Packard[/company] announced its own Kubernetes setup utility for HP’s OpenStack-based Helion cloud, but I haven’t heard much about it since.

There is no exclusivity in this latest news. The work Mirantis and [company]Google[/company] have done here will, in theory, help customers deploy Kubernetes on any OpenStack distribution. Mirantis and Google will demonstrate the technology Thursday in San Francisco.

And in the grand scheme of things, nearly every cloud or wanna-be cloud vendor worth its salt (including SaltStack) Microsoft, IBM, Red Hat and others, have pledged or contributed actual support for Kubernetes.

This latest news is another indication that Google is indeed serious about providing cloud capabilities to business customers, many of whom still view public clouds like Google Cloud Platform with suspicion. OpenStack is the cloud framework usually mentioned when a company decides to deploy a private cloud that they deem more suited for mission-critical workloads.

“From a Google perspective, containerization is important and running container clusters is a great way to enable developers to be productive,” said Kit Merker, the Google product manager focusing on Google Container Engine and Kubernetes.

“We know that enterprises will take time to transition to cloud. Kubernetes is a way to optimize infrastructure so it can run workloads in private or public cloud or bare metal.”

kubernetes openstackSo this is about workload portability but not really hybrid cloud per se. “This means you can build an application that uses containers and then move it to a different environment. That is what Kubernetes is all about,” he said. That is not the same thing as seamlessly integrating public and private clouds into a hybrid scenario.

[company]Amazon[/company] Web Services still leads the world in public cloud but Google and [company]Microsoft[/company] are giving it a run for its money. Microsoft Azure, because of its business roots, is seen as an attractive public cloud for that company’s myriad business customers so both Google and AWS have to show that they “get” CIO concerns about cloud deployment and provide enterprise class features and functions.

This step by Google, along with other moves announced in the fall and more recent news that it’s bringing four Google services to VMware’s  vCloud Air, are meant to reassure the C-suite set that Google means business.

Note: This story was updated at 11:11 a.m. PST with a more complete list of Kubernetes contributors.

 

AWS maintains lead in public cloud, but Azure inches forward

Amazon Web Services continues to dominate public cloud usage across the board, but Microsoft Azure is making strides at least in business accounts, according to a new RightScale survey.

[company]Amazon[/company] cloud adoption leads the pack with 57 percent of respondents reporting use of AWS (up from 54 percent last year) while 12 percent said they run [company]Microsoft[/company] Azure Infrastructure as a Service, up 6 percent from last year’s survey.

Among business or enterprise users, though, while AWS still leads with 50 percent, up slightly from 49 percent, Azure IaaS scored 19 percent, up from 11 percent.  [company]Rackspace[/company] and [company]Google[/company] App Engine are the next most popular clouds in this category, while vCloud Air logged 7 percent adoption, down from 18 percent. (Could the rebranding of vCloud Hybrid Services to vCloud Air have been a factor here?)

The Rackspace callout is interesting since the company said Tuesday it will stop breaking out public cloud and private cloud revenue and report them together. Rackspace is now focusing on private, managed cloud, in what some say shows it is ceding public cloud to the big guys.

RightScale Enterprise Cloud 2014-2015

All of these numbers are based on RightScale’s survey (downloadable here) of 930 cloud users, 24 percent of whom are RightScale customers.

Private cloud boosters won’t like this part: The new numbers show overall adoption of private cloud pretty much holding steady compared to last year. [company]VMware[/company] vSphere virtualized environments led with 53 percent of enterprise customers who reported that they use it as a private cloud. (Another 13 percent said they use vCloud Director as cloud.) This echoes last year’s survey in which many customers equated their virtualized server rooms with private cloud.

While private cloud appears to be in a bit of a swoon, it’s no surprise that Docker usage is hot. Per the survey, that containerization technology, while relatively new, is already used by 13 percent of respondents, while more than a third of the rest (35 percent) said they are planning to implement it.

Rightscale Public Clouds 2014OpenStack showed the greatest traction this year, with 13 percent adoption, growing by three percent year over year and still garnering big interest from companies whether they use it or not. A full 30 percent of respondents said they were evaluating or interested in using OpenStack over time. Microsoft’s relatively new Azure Pack showed a respectable seven  percent usage. Azure Pack, which mirrors Microsoft’s internal Azure usage, can run in a company’s own data centers or server rooms to provide an Azure-on-Azure hybrid.

Overall, Santa Barbara, California–based RightScale concluded from its research that cloud adoption is “a given” and hybrid cloud is the preferred mode of adoption. Of course RightScale offers multi-cloud management tools so that works out nicely for them.

RightScale VP of Marketing Kim Weins was our Structure Show guest after last year’s survey and had some interesting insights that might be helpful to compare and contrast. Check out the podcast below.

[soundcloud url=”https://api.soundcloud.com/tracks/143987938?secret_token=s-6kZD6″ params=”color=ff5500&auto_play=false&hide_related=false&show_artwork=true” width=”100%” height=”166″ iframe=”true” /]

OpenStack comes up huge for Walmart

For those skeptics who still think OpenStack isn’t ready for prime time, here’s a tidbit: @WalmartLabs is now running in excess of 100,000 cores of OpenStack on its compute layer. And that’s growing by the day.

It’s also the technology that ran parent company Walmart’s prodigious Cyber Monday and holiday season sales operations. If that’s not production, I’m not sure what is.

San Bruno, California–based @WalmartLabs, which is the e-commerce innovation and development arm for the [company]Walmart[/company] retail colossus, started working with OpenStack about a year and a half ago, at first relying heavily on the usual vendors but increasingly building up its in-house talent pool, Amandeep Singh Juneja, senior director of cloud operations and engineering, said in an interview.

Building a private cloud at public cloud scale

@WalmartLabs has about 3,600 employees worldwide, 1,500 of whom are in the Bay Area. Juneja estimated the organization has hired about 1,000 engineers in the last year or so — no mean feat given that there are lots of companies, including the OpenStack vendors, in the market for this expertise.

“Traditionally, Walmart is vendor-heavy in its big technology investments — name a vendor and we’ve worked with it and that was also true with OpenStack,” Juneja noted. “We started about one and a half years ago with all the leading distribution vendors involved … we did our first release with Havana and [company]Rackspace[/company]. But then we invested internally in building our own engineering muscle. We attended all the meet-ups and summits.” Havana is the code name for the eighth OpenStack code release.

Amandeep Singh Juneja, @WalmartLabs

Amandeep Singh Juneja, @WalmartLabs

Nothing says big like Walmart. It has around $480 billion in annual revenue, more than 2 million employees, and more than 11,000 retail locations worldwide (including Sam’s Club and Walmart International venues). Walmart.com claims more than 140 million weekly visitors. So scale was clearly an issue from the get-go.

What @WalmartLabs loved about OpenStack was that it could be molded and modified to fit its specifications, without vendor lock-in.

AWS need not apply

This is a massive private cloud built on a public cloud scale. There are also some macro issues at play here. Since parent company Walmart competes tooth and nail with [company]Amazon.com[/company], the chances of Walmart using Amazon Web Services public cloud are nil. (I asked Juneja whether Walmart would ever use any public cloud capabilities and he politely responded that this question was above his pay grade.)

The beauty of open-source projects like OpenStack is that new capabilities continually come on line and there is a community of deeply technical people working on the code. Going forward, Juneja is particularly interested in Ironic, an OpenStack project to enable provisioning of bare metal (as opposed to virtual) machines, and in the Trove database-as-a-service project. Trove, he noted, has matured a bit and Walmart will be using more DbaaS going forward.

Another work in progress is the construction of a multi-petabyte object store using the OpenStack Swift technology, but there are also plans to bring more block storage in-house, possibly using OpenStack Cinder. And the team is looking at Neutron for software-defined network projects.

One thing Walmart must deal with is its brick-and-mortar roots. The ability to order online and pick up in the store means that what @WalmartLabs builds must interact with inventory and other systems already running the Walmart/Sam’s Club storefronts. Non-e-commerce-related IT projects are run by Walmart’s Information Services Division at the company’s Bentonville, Arkansas headquarters.

So the ability of the shiny new OpenStack systems to interface with infrastructure that’s been in place for decades or so — some for as much as 50 years — is critical. It also spells the full employment act for all those @WalmartLabs engineers.

Note: this story was updated at 11:30 a.m. PST to reflect that Walmart is running 100K+ cores, not nodes, of OpenStack

AWS suits up more enterprise perks

More AWS perks for business users

Amazon Web Services has beefed up its identity management and access control capabilities so that businesses can more easily apply permissions to users, groups and roles in a consistent way. As explained in a blog post,  these identity and access management (IAM) policies are now treated as “first-class AWS objects” so that they can be created, named, and attached to one or more IAM users, groups, or roles.

Since I was unclear about what a first-class AWS Object really is I reached out to someone who knows who said that these policies get their own unique Amazon Resource Name (ARN). And that, in turn means users can more easily reuse common managed policies without having to write,update and maintain permissions.

These managed policies can also be managed centrally and applied across IAM entities — the aforementioned users, groups, or roles. And, customers can subscribe to shared AWS Managed Policies, so that its easier for them to appy best security or other practcies.

 

That news came a few days after [company]Amazon[/company] announced general availability of its AWS Config, a configuration management database (CMDB) tool, announced in November, that keeps track of the cloud resources used and the connections between them. The goal is that it can then track changes made to those resources and make sure those changes are logged in AWS CloudTrail.  The data collected there can then be polled via Amazon’s own APIs

AWS Config, and AWS Service Catalog, were both announced in preview form AWS re:Invent in November. A Service Catalog is a tool used in enterprise accounts to shop for and manage authorized tools and applications and will be tied into IAM.  General availability for Service Catalog was promised for early 2015, so stay tuned.

All of these services — promised and delivered — are geared to make AWS more IT friendly in bigger enterprises — to help make sure that users can access only the resources they are authorized for and that those resources are the most updated versions.

It’s also interesting that AWS, which used to announce new services only when they were ready, is now fully in enterprise software mode, pre-announcing new products weeks and months before they are broadly available.

 

AWS Re:invent

AWS Re:invent

EMC Cloudscaling aims to bridge OpenStack-AWS divide

If you’re running an OpenStack private cloud and want it to talk to Amazon’s EC2 compute service, you may want to check out this a new “drop-in”API created by EMC/Cloudscaling and available from Stackforge.

https://gigaom.com/2015/02/13/heres-a-new-drop-in-ec2-api-for-openstackers-who-want-it/

Randy Bias, co-founder  of Cloudscaling and now VP of Technology for [company]EMC[/company], has long maintained that OpenStack needs to work with Amazon. He also pledged similar support for [company]Google[/company] Compute Engine APIs. Asked via email if that’s still the plan, Bias  said “yes but it’s a lower priority until we see traction.”

Structure Podcast: The biologic roots of deep learning

Deep learning, which enables a computer to learn — or program itself — to solve problems — is a hot topic that Enlitic CEO Jeremy Howard and Senior Data Scientist Ahna Girshick helped explain to mere mortals on this week’s podcast.   If you want to know why you don’t necessarily need a ton of data to do good work in deep learning and how the field is inspired by biology, if not the human brain,  check out this show. And, to hear more from Gisrshnick on this hot topic, you can also sign up for next month’s Structure Data event.

[soundcloud url=”https://api.soundcloud.com/tracks/190680894″ params=”secret_token=s-lutIw&color=ff5500&auto_play=false&hide_related=false&show_comments=true&show_user=true&show_reposts=false” width=”100%” height=”166″ iframe=”true” /]

SHOW NOTES

Hosts: Barb Darrow and Derrick Harris.

Download This Episode

Subscribe in iTunes

The Structure Show RSS Feed

 

This story was updated at 11:37 a.m. PST February 18 with more detail on what an AWS First-Class is.

Here’s a new “drop-in” EC2 API for OpenStackers who want it

Many news cycles have been burned on the debate over whether OpenStack-based cloud providers should or need to support the major Amazon Web Services APIs.

Cloudscaling and its co-founder Randy Bias have long advocated that such support is critical to the success of OpenStack and promised Cloudscaling support for [company]Amazon[/company] elastic compute cloud (EC2) APIs. AWS, after all, is by far the market leader in the public cloud arena.

As of this week, Cloudscaling, now part of [company]EMC[/company], has made available a “drop-in” replacement for the existing OpenStack Nova EC2 API. Nova, OpenStack’s compute module, already offered a degree of EC2 API compatibility that a vendor could expose, or not, in its own cloud offering.

Rackspace notably chose not to expose it. [company]Hewlett-Packard[/company] at first opted to support the EC2 API, then reversed course in late 2013 — but within a year bought Eucalyptus, a provider of private cloud technology noted for its AWS API support. And VMware’s cloud chief Bill Fathers made it pretty clear on the recent Structure podcast that he doesn’t give a fig about supporting AWS APIs.

Bias, now VP of technology at EMC, is unwavering in his belief that AWS API support will strengthen OpenStack’s chances of success in the market. Cloudscaling also has promised support for key Google Compute Platform APIs.

Per Bias’ blog post:

I’ll reiterate again, since folks still sometimes get confused, I’m not advocating dropping the OpenStack APIs in favor of AWS.  I’m advocating embracing the AWS APIs, making them a first class citizen, and viewing AWS as a partner, not an enemy.  A partner in making cloud big for everyone.

His plan is to improve upon the existing Nova EC2 API — actually build it from scratch — and ask the community to test it out and support it. His rationale? People are using Amazon’s cloud and OpenStack needs to attract those people.

Bias used a chart from the November OpenStack user survey (which had 669 respondents) to illustrate his point. Nearly half of users surveyed use the EC2 compatibility API in production, 38 percent use it in development/quality assurance and 38 percent use it in proof-of-concept projects. By contrast, just four percent said they used the Open Cloud Computing Interface in production, one percent in dev/QA and seven percent in proof of concept trials.

Compatibility APIs

If you want the back story of the great API kerfuffle, check out this YouTube video of a debate between Bias, Mirantis co-founder Boris Renski and others.

[youtube https://www.youtube.com/watch?v=W7H5zFWUSVI]

The poor private cloud gets no respect

Pity your private cloud, if you have one. If cloud analysts are to be believed, private cloud is losing ground as public cloud providers — chiefly Amazon Web Services, Google, and Microsoft — keep adding features and functions, many of which target enterprise IT buyers.

Last week, for example, Gartner analyst Thomas Bittman blogged that 95 percent of enterprise IT types he surveyed found something lacking in their own private clouds. Of course Bittman loaded the gun for them, distilling the reasons “your enterprise public cloud is failing”  into six key categories and then polling an audience about them at an event.

Part of the problem may be in definitions. Private cloud is not merely a highly virtualized data center. It needs to deliver on-demand services easily and offer the sort of scale-up-and-down-as-needed elasticity that is the hallmark of public clouds. In a response to one comment on his post Bittman defined private cloud as the

cloud computing style delivered with isolation. Fully private would be fully isolated. It doesn’t need to be owned and managed on-premises, but today it often is (I’d say, 90-95% of the time).

Of the 140 companies Bittman surveyed, the most common reason for dissatisfaction (noted by 31 percent of respondents) is that too much emphasis was placed on cost-cutting, not on providing agility in creating, spinning up and down capabilities as needed. The second most-cited complaint, for 19 percent of respondents, was that their private cloud doesn’t do enough. But check out the whole post, along with the comments.

In August Gigaom Research published its own analysis showing public cloud options outstripping private clouds (subscription required) for several reasons. Notably, even if you are running a real private cloud — not just a heavily virtualized server room — you are probably still buying, deploying and maintaining your own hardware and software.

Gigaom research analyst David Linthicum — who is also SVP at Cloud Technology Partners, which works with the big public cloud providers — noted in that report that security, or lack thereof, has been touted as a key private cloud selling point but is not necessarily a differentiator in the way most people expect. He wrote:

Private clouds, while they feel more secure since you can see the blinking servers in your data center, are as secure or less secure than public clouds, generally speaking. Enterprises are just discovering this fact, and are opting for public clouds as cloud projects come on-line.

Ouch. Private cloud purveyors, please feel free to comment below.

Philip Bertolini, CIO of Oakland County, Michigan, said to term private clouds as failures because there is not 100 percent satisfaction is unfair. In the Gartner blog post, he noted, Bittman discusses how 95 percent of the users have had problems but that doesn’t mean their efforts failed.

“Moving to the cloud is difficult and has to be planned out carefully. Any IT project requires good planning or the results can be less than desirable. I do believe that the is not the magic wand for everything that troubles us. Using the cloud wisely with good planning can be very successful,” Bertolini noted by email.

There is some merit to the private-cloud-doesn’t-meet-expectations argument. Vendors have fed into that by overselling the technology, for one thing. But, the notion that a small number of public cloud vendors (even vendors as huge as [company]Amazon[/company], [company]Google[/company] and [company]Microsoft[/company]) can fill every need is a stretch.

As more than a dozen vendors, many of them pitching OpenStack-based private clouds, duke it out, they need to counter this perception that public cloud is becoming the inevitable destination for many, many workloads going forward.

This story was updated on February 12 with quotes from Oakland County CIO Philip Bertolini and on February 13 with a note of David Linthicum’s affiliation with CTP.

Microsoft woos Y Combinator startups with big Azure credits

Microsoft wants to boost its cloud’s profile among startups so it’s making $500,000 in Azure credits available to Y Combinator-backed companies.

The credits start rolling with the Winter 2015 class and will continue after that, according to this Y Combinator blog post. This can be a good number of companies — there were 106 companies in the Spring and Winter 2014 classes, for example.

Cloud credits are ubiquitous — Y Combinator has special hosting offers from [company]Amazon[/company], [company]Google[/company], [company]Rackspace[/company] and now [company]Microsoft[/company], according to Y Combinator president Sam Altman. But, $500K is a big number. (Oh, and the startups will also get three years of Office 365 subscription, “access to Microsoft developer staff,” plus a year of CloudFlare and DataStax services.

Qualified startups can typically get $1,000 to $15,000 in Amazon Web Services (AWS) credits, and there are other freebies available. Then, in September, things started going a bit haywire. Google started offering $100,000 in Google Cloud Platform credits to qualified startups. Two months later [company]IBM[/company] upped the ante to  $120,000 in credit for SoftLayer infrastructure or BlueMix PaaS. Again all for “qualified” startups.

This is a strategic gambit for Microsoft, which wants to get more young companies — many of which are probably not Windows focused — to check out Azure. It’s also a way to chip away at [company]Amazon[/company] Web Services’ prodigious lead among startups. AWS is pretty much the default cloud selection for young companies.

This story was updated at 5:24 a.m. PST February 11 to reflect that AWS typically provides qualified startups with up to $15K in promotional funding.