AWS boosted cloud reliability in 2014, new stats show

Amazon Web Services is not an organization that rests on its laurels. New stats compiled by CloudEndure show that AWS, the public cloud market leader, boosted reliability significantly over the course of last year, based on the number of errors reported. It logged 41 percent fewer errors in the fourth quarter of 2014 than in the first quarter.

2014 AWS errors by quarter

CloudEndure, which offers disaster recovery and cloud migration services, based its findings on data from the [company]Amazon[/company] health dashboard.

Also tracked: Number of errors by AWS service, but there’s more data too, so check out the whole post. CloudEndure added suggestions right out of the AWS playbook, such as that workloads should be deployed across regions.

2014 AWS Errors by Service

CloudEndure, based in Israel with offices in New York City, also tracks [company]Microsoft[/company] Azure availability and posted this earlier this week. It does not follow [company]Google[/company] Cloud Platform.

Azure Service Errors 2014 by quarter

Looks like Microsoft is outsourcing a part of its cloud

CDN expert Dan Rayburn reported Monday that Microsoft is deep-sixing its home-grown content delivery network capabilities and is instead turning to Verizon EdgeCast to deliver video for Microsoft Azure customers. Verizon bought Edgecast for its media delivery expertise, in late 2013.

Reached for comment, a [company]Microsoft[/company] spokesman provided a limited confirmation: “[company]Microsoft[/company] licenses technology from many partners to complement our product offerings and to give customers complete solutions. We are happy to partner with EdgeCast to provide an integral component of the Azure Media Services workflow.”

Delivery of content of all kinds, including bandwidth-hungry video, has been a priority for the Azure forces. Microsoft trumpeted the use of Microsoft Azure Media Services to help live stream Winter Olympic events from Sochi, for example, but that effort relied on CDN market leader Akamai.

“While Azure did have some CDN services of its own before shutting them down, they were basic, Rayburn said via email.

“Partnering with [company]Verizon[/company]’s EdgeCast gives Azure more CDN functionality, greater reach and capacity and allows Azure to get all of the advantages of one of the best CDNs in the market, without any of the major capex or opex challenges. It’s a smart move on Azure’s part,” he said.

As Rayburn pointed out on his blog, [company]Amazon[/company] builds almost everything in its cloud from foundation to rooftop. Microsoft, on the other hand, is more partner-focused and thus more inclined to license or buy technology.

And, don’t forget, Microsoft is also playing cloud catch up to Amazon Web Services, which, having launched in 2006, has a multi-year head start over competitors. Azure, in its first PaaS-based incarnation launched in 2010, but the more AWS-comparable version kicked off in 2013.

When you’re behind in the race, buying in could be a way to make up for lost time.

 

Big, new AWS C4 instances are here — for real

Amazon Web Services pre-announced big, new C4 compute instances in November at AWS Re:invent, apparently re-announced them on January 2 in a disappearing blog post, and on Monday announced availability officially (with the same blog post). Talk about recycling.

In November, [company]Intel[/company] SVP and GM Diane Bryant helped [company]Amazon[/company] CTO Werner Vogels introduce the instances, which are based on Intel’s Haswell chip with a base speed of 2.9 GHz and achievable clock speeds of up to 3.5 GHz.

The latest additions to the AWS’s EC2 menu, the instances could be a boon for compute-intensive workloads in online gaming, risk analysis or graphics rendering.

This is the latest shot fired in the compute instance arms race. Last week, [company]Microsoft[/company] announced availability of its big G4 VMs (Microsoft speak for instances).

In case you’ve forgotten, here are the various now-available instances with pricing for U.S. East and West (Oregon) regions. (The instances are also available in other AWS regions, but prices may vary.)

aws c4 instance pricing

Cloud upgrades no big deal if deployment is done right

When Verizon said it might shutter its new Verizon Cloud for 48 hours this weekend for a major upgrade, users were shocked. Some promised to live-tweet the event which starts Saturday at 1 p.m. EST. Verizon’s subsequent qualifications — that 48 hours is the worst-case scenario, that customers received ample notice, that this work would make future upgrades less traumatic — eased the situation a bit.

Cloud vendors take these occasions — whether the issue is a planned-in-advance upgrade or more hastily put together patch  — to educate users about best deployment practices. They tick off a range of tips — that workloads should be deployed across availability zones and regions, for example. And third-party tools and management vendors likewise parlay these events to promote the use of their own products.

To mitigate downtime, plan, plan, plan

Cedexis, which offers a cross-cloud load balancing service, sent out an email Thursday with its own list of best practices. From the email:

Whatever your opinion on Verizon Cloud and the way they are rolling this upgrade out, what is true is that there is no reason for this type of system maintenance to impact the correctly configured enterprise. It is time for architects and designers to realize that cloud outages are a fact of life — just like Data Center outages.

In cloud, as in past IT deployment models,  disaster recovery “relies on the use of geographically diverse deployment of applications. Why would anyone adopting Cloud think single-homing an application is a reasonable practice?,” according to Cedexis.

Deploy across zones, regions and vendors

Cedexis goes further than the multi-availability zones mantra to say enterprises should use multiple cloud vendors as well. That’s something you probably won’t hear from [company]Amazon[/company] or [company]Microsoft[/company].

Vendor specific outages — whether the vendor is Amazon Web Services, Microsoft, Rackspace, IBM or Verizon — are “more common than ‘acts of God,'” according to Cedexis, so selecting multiple vendors with (of course) global load balancing protects the user from these events. [company]RightScale[/company] is another vendor who provides these cross-cloud management and deployment capabilities.

So go easy on [company]Verizon[/company], said Carl Brooks, an analyst with 451 Research. In a comment on Gigaom’s earlier story, Brooks wrote that the hubbub around Verizon’s planned upgrade was way overblown.

[company]Verizon[/company] is “telling a small subset of its customers to get the heck  out while they do some planned maintenance, which is miles better than the usual practices from cloud providers,” Brooks wrote.

“I’d bet real money they could do most of this upgrade without kicking users off … [company]Microsoft[/company] has planned monthly outages to patch and tells you if you’re in the target zone ahead of time. This isn’t so different, just a bit more ham handed. You know what AWS says to its [most of its] users before it upgrades and reboots a bunch of its platform? Nothing.” Brooks later amended his statement via email to say that most customers never notice routine AWS maintenance.

Update: A Verizon customer pointed out that at least one Verizon Cloud (compute and storage) shut down in September did, in deed, last two days. It kicked off at 11 a.m. CT on September 11 and services were not back on till 11:38 p.m. two days later, according to this Verizon support post.

Note: This story was updated at 1:43 p.m. PST with information on Verizon’s September 11 shut down and againa t 7:19 p.m. PST January 9 with Carl Brooks’ amended statement.

 

Microsoft unveils Azure Key Vault and big strong servers

Microsoft caused a bit of a stir in October when it announced plans for big cloud servers with up to 32 cores, 450 gigabytes of RAM and 6.5 terabytes of local SSD. As of Thursday, those G-series instances (or VMs, in Azure-speak) are available. See the chart below for versions and pricing.

Also new for the [company]Microsoft[/company] public cloud: a preview version of Azure Key Vault, which will let customers control their encryption keys and passwords without having to resort to an on-premises Hardware Security Module (HSM) appliance.

As we’ve seen over and over, Microsoft is racing to add features and services to its cloud to make it more competitive with [company]Amazon[/company] Web Services. (As an interesting aside, the new AWS C4 Family of instances, pre-announced in November, were apparently ready to roll last week when an errant blog post slipped out, but are still not officially available.)

Azure Key Vault pricing

Cloud security is key

One leg of that race is adding ever more powerful instances or servers along with price cuts the longer the instances are on the market. Another leg, in the wake of Sony and other publicized data breaches, is bulletproofing public clouds as those vendors know that security is the primary concern for coveted enterprise accounts.

A recent Piper Jaffray survey of 112 CIOs found that a majority (73 percent) plan to budget dollars for private, public or hybrid cloud projects this year. But 35 percent of them also said security concerns would keep their data on premises, according to the Wall Street Journal (paywall).

“In light of the increasing number of security breaches, we believe the movement of data and workloads to a public cloud provider will likely remain muted in 2015,” the report said.

That’s the rationale behind Azure Key Vault, the AWS Key Management Service Amazon announced in November and an array of third-party tools that integrate with cloud compute and storage offerings.

One more thing

Finally, in its effort to embrace popular Docker container technology, Microsoft company has added a Docker engine — as part of an Ubuntu Linux image — to its Azure marketplace.

“This is a simple, fast way to get started,” Corey Sanders, partner director of program management for Azure, told me in an interview. Users can pull the Ubuntu image down and just get going. “It gets rid of roadblocks so you can get up and running in minutes with Docker.”

Microsoft Azure G-Series pricing

Amazon Web Services tops list of most reliable public clouds

Google and Microsoft now host public clouds meant to give Amazon Web Services a run for its money, but they still have a way to go to rival good ol’ AWS in terms of uptime, according to cloud reliability ratings by CloudHarmony.

The Laguna Beach, California company tracks status of more than 30 clouds from AWS to Zettagrid, including a couple I’ve never heard of, so take a look.

Filtering worldwide downtime over the past year, the [company]Amazon[/company] S3 storage service, for example, registered 23 outages for a total of 2.69 hours of downtime, while EC2 compute logged 20 outages accounting for 2.41 hours out, according to CloudHarmony’s CloudSquare status metrics.

By contrast, the [company]Google[/company] Compute Engine component of Google Cloud Platform showed 72 outages for 4.42 hours downtime and Google Cloud Storage reported 8 outages adding up to 14.23 minutes down.

And [company]Microsoft[/company] Azure Virtual machines fell more than 92 times for nearly 40 hours of downtime over 12 months — including a significant outage in November; Azure Object Storage was off 141 times for 10.97 hours.

Some caveats

Jason Read, co-founder of CloudHarmony, acknowledged that the service can’t and won’t record every glitch. For example, it monitors only relatively recent M3 large EC2 instances running in one Availability Zone per AWS Region. So if there were glitches in older AWS instances during the great Xen reboot of 2014, they wouldn’t show up. And CloudHarmony only “watches” Linux instances running in Microsoft Azure.

So take a look at the numbers, but remember they represent a subset of total cloud resources from each provider.

cloudharmony AWS

CloudHarmony Google

Microsoft Cloud Harmony

After rough year for Amazon stock, will AWS feel the pain?

People who watch Amazon Web Services tend to be cloud oriented and don’t necessarily pay a ton of attention to the Amazon Inc. mothership. Maybe they should broaden their focus a bit.

In 2014, that mothership had a rough year  — Amazon’s stock price fell 18 percent over a period during which the Nasdaq overall was up 14 percent. That put [company]Amazon[/company] founder and CEO Jeff Bezos on the receiving end of a $7.4 billion paper loss for that period. (His 18 percent ownership is still worth just north of $26 billion.)

The company’s third quarter was particularly worrisome. For the period ending September 30, 2014, Amazon posted a $544 million loss on revenue of $20 billion, provoking much consternation on Wall Street, which appeared to be losing patience with Bezos’ growth-at-all-costs strategy. At that time, I wondered whether, given all that angst,  the company would keep investing in AWS to the degree it had previously. I’ve reached out to Amazon for comment and will update this post as needed.

Some people see AWS as a profit generator for Amazon Inc. Despite the steady drumbeat of price cuts, it’s been clear for some time that AWS isn’t the loss leader most experts once deemed it to be. But as of 2014, AWS was no longer the sole arbiter of public cloud IaaS pricing. Google and Microsoft sliced the prices of public cloud services on their own, sometimes beating AWS to the punch — which means that even as AWS adds scale and features, it is in a much more competitive market than it was for the first seven years of its existence.

AMZN Chart

AMZN data by YCharts

More big fish in the pond

Over the past year or so, big competitors have come online with many comparable services for the first time. And two of those competitors — [company]Microsoft[/company] and [company]Google[/company] — seem prepared to spend what it takes to keep scaling up, cutting prices and adding higher-level services.

The thinking seems to be that AWS (and Amazon itself) sacrifices profits for market share growth, and then, when the growth spurt stops, it can start reaping the rewards.

That strategy worked well for Amazon’s retail operations a decade ago.  But at that time, there weren’t e-commerce competitors capable of competing at Amazon scale. That’s changed. Now there’s Chinese retail giant Alibaba, Target.com and Walmart.com. While Bezos’s net worth got a haircut in 2014, Alibaba Group co-founder Jack Ma added another $25.1 billion  to his stockpile, thanks to his company’s September IPO.

Just as those retail competitors got with the program, tech competitors have gotten the cloud memo. Microsoft now fields Azure, Google has Google Cloud Platform, [company]IBM[/company] has SoftLayer.

No one doubts Bezos’ financial acumen, and he isn’t standing still. In early December, the company sold $6 billion in a debt offering, a move that met mixed reviews by various bond rating agencies. Moody’s pretty much immediately downgraded its outlook on Amazon to negative; Standard & Poor’s, on the other hand, reaffirmed its positive AA rating on the debt.

In November, AWS SVP Andy Jassy was asked whether Amazon’s cloud could compete with cash-rich rivals and he expressed confidence that the company can continue to “fund this business to its potential” and reiterated that with its head-start it still offers far more services than anyone else.

Still, it helps to remember that even companies that dominate markets can’t hold the top slot forever. The public cloud marketplace has been validated, so competitors flooded in. The niche players won’t all be able to play at mass scale IaaS, but it’s clear that Microsoft and Google are here to stay. AWS isn’t alone anymore.

Note: This story was updated at 10:13 a.m. to include Andy Jassy’s comments from November for context.

Yup, 2014 was a big year in cloud

2014 was the year in which both Microsoft and Google got serious about their public cloud options and taking on Amazon Web Services directly with their own Infrastructure as a Service and associated services.

That both of these companies have extremely deep pockets is not lost on the market leader AWS which continues to roll out new, and higher-level services frequently. If you’re a cloud deployer or would-be cloud deployer, AWS Re:Invent is a must-attend event.

Long story short: [company]Google[/company] and [company]Microsoft[/company] have made huge strides, but AWS, with its 8-year head start, remains the cloud to beat. It’s a good time to be a cloud customer provided you can track the dueling product releases and price cuts and can manage to keep yourself out of the vendor lock-in that afflicted many IT shops in the past few decades.

Cybersecurity fears grow

The counterpoint to all of the above is that 2014 was also a year that saw scary security breaches including the latest: Anonymous is claiming credit for filching 13,000 passwords and credit card data of users with [company]Sony[/company] PlayStation, Microsoft Xbox LIVE, and [company]Amazon[/company] accounts. And, if you don’t think these attacks don’t put even more fear of God (and cloud) in corporate IT buyers, you have another think coming.

Data security concerns remain the biggest inhibitor to cloud adoption. This is true even though most IT folks who, if they’re being honest, would admit privately that their own on-premises server rooms, are hardly paragons of security. But perception is reality and people are wary of putting valuable data in a cloud they can’t control. This is a problem that will only grow with the new year.

As [company]General Catalyst[/company] Managing Partner Steve Herrod wrote recently:

… As bad as 2014 has been, and it has been bad, we’re just seeing the tip of the iceberg. Given the steady increase in value going through our systems (credit cards numbers, personal information, IP), organized crime and nation-sponsored attacks will continue to rise in quantity and sophistication.

Cloud turf war

This is worrisome for Jane Q. Consumer, but even more so for big IT vendors. all of whom are trying to woo corporate customers to their respective clouds. Legacy giants [company]IBM[/company], [company]HP[/company], [company]Oracle[/company], [company]VMware[/company], [company]Dell[/company], Microsoft all want to keep their existing customers in house and (dare they hope?) win new customers as well. Their well-founded fear — other than that security fiascos will keep people away from cloud altogether — is that a ton of those jobs are flowing to AWS which is somehow both an IT upstart and the industry leader in cloud.

[company]Google[/company] is the wild card here. The company which knows a little something about building massively scaled services, is showing itself to be serious about wooing enterprise customers to its  cloud as well — although there was some skepticism on that front which it has allayed somewhat with new peering agreements and VPN options; the hiring of enterprise savvy execs most notably former Red Hat CTO Brian Stevens; and a Microsoftian-looking partner program.

Brian Johnson onstage at Google Cloud Platform.

Brian Johnson onstage at Google Cloud Platform Live.

The biggest personnel move in cloud this year was the ascension of Satya Nadella to Microsoft CEO after a very public and somewhat painful 6-month search. Now even some Microsoft haters see the company as an up-and-comer in cloud. To be fair most of that hard work was accomplished on former CEO Steve Ballmer’s watch but Nadella is seen as more pragmatic and much less doctrinaire than his predecessor, who exhibited an almost pathological hatred of all things Apple or Google. Nadella, after all, broke tradition to bring Office to non Windows devices, a huge departure for the company.

Microsoft CEO Satya Nadella

Microsoft CEO Satya Nadella

Microsoft already has enterprise customers and partners in spades which could help in its hybrid cloud push. Ditto VMware, HP, IBM. But public cloud kingpin AWS is making a push into that hybrid scenario with products targeting VMware admins their Windows counterparts.

Structure Show: Docker, Docker, Docker

We didn’t do a show on this holiday week, but check out our last podcast with Docker CEO Ben Golub if you haven’t already. Golub addresses how the competitive landscape has shifted with the CoreOS decision to launch Rocket, a container of its own.

[soundcloud url=”https://api.soundcloud.com/tracks/182147043″ params=”color=ff5500&auto_play=false&hide_related=false&show_comments=true&show_user=true&show_reposts=false” width=”100%” height=”166″ iframe=”true” /]

Where is enterprise infrastructure headed in 2015?

The enterprise industry is another year older … and hopefully somewhat wiser. Here’s what enterprise watchers should expect to see in 2015.

More cyber attacks

Sadly, this is an easy one. As bad as 2014 has been, and it has been bad, we’re just seeing the tip of the iceberg. Given the steady increase in value going through our systems (credit cards numbers, personal information, IP), organized crime and nation-sponsored attacks will continue to rise in quantity and sophistication. The current approaches to security clearly aren’t cutting it, which is why the security space is one of my biggest personal focus areas. (Full disclosure I’m an investor in Illumio, Menlo Security and ThreatStream, all companies in this space.)

AWS pushes further into the enterprise

Almost every startup that we fund today is using [company]Amazon[/company] Web Services, but it’s interesting to see AWS creep further into the medium and larger companies that dominate IT spending. At this year’s AWS Re:Invent, there were lots of compelling enterprise anecdotes, plenty of “all-in” stories, and, most importantly, the arrival of an ecosystem. There were startups and large companies alike announcing integrations with AWS with particular focus on the “-ities” — predictability, manageability, security, and availability. These are good signs of increased adoption in mainstream businesses where it’s now not “if” but “when” a company adopts cloud.

AWS: Reinvent

The rise of IaaS competitors

AWS continues with its strong lead, but 2014 also showed that there’s going to be a bigger fight than ever. In particular, [company]Google[/company] Compute Engine and [company]Microsoft[/company] Azure are rapidly improving their services and have the pocketbooks to fight this for the long term. Throw in Rackspace, IBM, vCloud Air, HP and the many other regional or vertically oriented offerings, and it’s going to be a major battle — with customers as the likely winners.

Containers get down to work

The rise of Docker has been one of the true success stories of 2014. However, it has also created a deluge of competitors (CoreOSRed HatUbuntu) and interesting co-opters (AmazonGoogle and VMware). Quite a year for a previously unheralded technology. The rise is real, but I believe that some of the hype will subside in 2015 as the some of the real work of making containers usable by enterprises begins in earnest.

Docker and the money container

Converged/hyper-converged infrastructure grabs the limelight

2014 saw continued excitement over “converged infrastructure,” pre-configured hardware/software bundles that are powerful and easy to adopt. Nutanix, VCE and Cisco UCS get most of the attention, but there’s lots of interesting competition on the way, especially as the latter two vendors update their relationship status to “It’s complicated.” Latest offerings include VMware’s EVO designs to new products from the big system vendors (Dell and HP are particularly aggressive). And I can personally attest to a slew of startups heading into this converged world with a variety of technologies and approaches.

APIs on the mind

I wrote about “mobile-first infrastructure” earlier this year and continue to think it will drive several longer-term infrastructure changes. In 2015, I think it will manifest itself most as the rise of APIs in enterprise development, as companies both produce and consume APIs like never before. Look for increased conversations, companies and challenges arising over this shift. (Full disclosure: I’m a backer of RunScope, which makes developer tools for this “API economy.”)

Network virtualization gets its legs

There has been much discussion of the arrival of software-defined networking (SDN). However, the term itself has been polluted to a point where it means different things to almost anyone you ask. I prefer the term network virtualization to speak more holistically about the advancement in separating out the logical network from the physical network. Cisco ACI and VMware NSX appear to have the lead, and 2014 saw significant movement from proof-of-concepts toward significant paid usage. Anecdotally, most of the adoption is in service providers, financial services and tech-heavy IT companies. 2015 should see further progress in the adoption, including by a broader set of consumers.

Big deals for big data

In 2014 there was nonstop talk about big data, analytics, and the opportunities and challenges of each. 2014 funding for companies has been unprecedented, ranging from Intel’s huge bet on Cloudera to substantial private investments in DataStax (driver of Cassandra)Databricks (driver of Spark)PlatforaAltiScaleDataGravity and numerous others. (My company, General Catalyst, invested in AltiScale and DataGravity.) Next year these companies will all focus on revenue — and we’ll see how the public markets respond to at least one Hadoop vendor, as Hortonworks is now a public company.

That’s one person’s cut at developments in enterprise infrastructure for 2015, and I’m sure I’ve omitted others that will be even bigger. That’s what is so fun about this space these days: We’re in a modern-day renaissance driven by the convergence of new technologies, new expectations and new challenges, all of which point toward more and bigger changes happening each year than may have taken place in prior decades. Here’s to the fun ride ahead.

Dr. Steve Herrod is a managing director at General Catalyst Partners and was CTO and senior vice president of R&D at VMware.

Note: This story was updated at 5:24 p.m. PST to correct the reference to Cisco ACI (application-centric infrastructure) not ACE.

Mo’ money, mo’ data, mo’ cloud on the Structure Show

[soundcloud url=”https://api.soundcloud.com/tracks/180964822?secret_token=s-MTCRX” params=”color=ff5500&auto_play=false&hide_related=false&show_comments=true&show_user=true&show_reposts=false” width=”100%” height=”166″ iframe=”true” /]

 

If you want an informed opinion on the state of the cloud and the relative merits of the players, Sebastian Stadil’s a good person to ask. Founder and CEO of Scalr, a multi-cloud management company, he keeps his finger on the pulse of all the players and — perhaps more importantly — their customers.

On this week’s Structure Show he handicaps how [company]Google[/company] and [company]Microsoft[/company] are doing in public cloud not just technologically but in terms of their sales strategies which, when it comes to enterprise accounts, may be just as important as technology. And of course the company everyone is measuring by is [company]Amazon[/company] Web Services which leads the pack.  Oh, and he’s got lots to say about the OpenStack ecosystem as well; cloud technologies from [company]Oracle[/company], [company]Joyent[/company] and more.

It was a busy week in the funding arena with Nginx, DataGravity, and Mesopshere all getting substantial VC rounds ($20 million, $50 million and $36 million respectively.) That ain’t chicken feed and we talk that out.

And, of course, this week’s Hortonworks IPO puts the spotlight back on Hadoop and big data in a  big way, which gives us a chance to tout Gigaom’s upcoming Structure Data event which will feature talks from Hortonworks CEO Rob Bearden, Cloudera CEO Tom Reilly and MapR CEO John Schroeder.

 

Sebastian Stadil, CEO of Scalr

Sebastian Stadil, CEO of Scalr

 

SHOW NOTES

Hosts: Barbara Darrow and Derrick Harris

Download This Episode

Subscribe in iTunes

The Structure Show RSS Feed

PREVIOUS EPISODES:

Why CoreOS went its own way on containers

More from Facebook on its new networking architecture 

Do you find OSS hard to deploy? Say hey to ZerotoDocker

All about AWS Re:Invent and oh some Hortonworks and Microsoft news too

Ok, we admit it. Google may be serious about cloud after all