Report: Docker and the Linux container ecosystem

Our library of 1700 research reports is available only to our subscribers. We occasionally release ones for our larger audience to benefit from. This is one such report. If you would like access to our entire library, please subscribe here. Subscribers will have access to our 2017 editorial calendar, archived reports and video coverage from our 2016 and 2017 events.
Image 1 for post Navicron- Linux emerges as clear winner in mobile applications( 2008-02-07 22:25:59)
Docker and the Linux container ecosystem by Janakiram MSV:
Linux container technology is experiencing tremendous momentum in 2014. The ability to create multiple lightweight, self-contained execution environments on the same Linux host simplifies application deployment and management. By improving collaboration between developers and system administrators, container technology encourages a DevOps culture of continuous deployment and hyperscale, which is essential to meet current user demands for mobility, application availability, and performance.
Many developers interchange the terms “container” and “Docker,” sometimes making it difficult to distinguish between the two, but there is a very important distinction. Docker, Inc. is a key contributor to the container ecosystem in the development of orchestration tools and APIs. While container technology has existed for decades, the company’s open-source platform, Docker, makes that technology more accessible by creating simpler and more powerful tools. Using Docker, developers and system administrators can efficiently manage the lifecycle of tens of thousands of containers.
This report provides a detailed overview of the Linux container ecosystem. It explains the various components of container technology and analyzes the ecosystem contributions from companies to accelerate the adoption of Linux-based containers.
To read the full report click here.

Cloud options mean decisions, decisions for IT buyers

Much has been written about cloud consolidation, with M&A roiling the cloudscape over the past few months: Cisco bought Metacloud, EMC bought CloudscalingHP snapped up Eucalyptus. Despite all that, cloud deployment options abound, and choice will be a big theme at the upcoming Structure 2015 event, this June in San Francisco.

First, there is more choice than ever in public cloud. Sure, Amazon Web Services leads the market-share race by a wide margin. But viable options are available — from Microsoft Azure to Google Cloud Platform to vCloud Air to Digital Ocean to CenturyLink. What many of us tend to forget is that, despite all the cloud talk, we’re still very early in the game when it comes to business deployment. There’s a ton of opportunity out there. Is it enough to float all boats? That’s the zillion-dollar question.

We will discuss those options, and how even the biggest enterprises — General ElectricWalmart — are deploying more of their IT on cloud. The question is no longer if, but when.

At this year’s event, we’ll welcome back [company]Amazon[/company] CTO Werner Vogels, Khosla Ventures founder Vinod Khosla, [company]Microsoft[/company] EVP Scott Guthrie, Google SVP Urs Hölzle, Battery Ventures technology fellow Adrian Cockcroft and DataGravity CEO Paula Long.

We’ll hear from first-timers, too: Canonical founder Mark Shuttleworth, Digital Ocean CEO Ben Uretsky, CoreOS CEO Alex Polvi. And, on the end user side, we’re really excited to bring on stage National Football League CIO Michelle McKenna-Doyle, FBI CISO Arlette Hart and Pinterest head of engineering Michael Lopp. More names to come.

For a refresher of last year’s event, here’s a sampling of some favorite sessions:

Google’s Urs Holzle:

[youtube https://www.youtube.com/watch?v=I9R4P0TLViA]

Facebook’s Jay Parikh:

[youtube https://www.youtube.com/watch?v=F9FYTbxWK1o]

Intel SVP Diane Bryant:

[youtube https://www.youtube.com/watch?v=HTXuwqLUw7M]

Amazon’s Werner Vogels:

[youtube https://www.youtube.com/watch?v=oZPlr2-KMnw]

Microsoft’s Scott Guthrie:

[youtube https://www.youtube.com/watch?v=TImzXnUaO0A]

Amazon Web Services adds to its enterprise war chest

This week Amazon Web Services continued its enterprise-focused feature push with new resource groups and a tag editor for EC2 instances, which means that IT people at big companies (or even smaller organizations) will have an easier time isolating the resources they use.

The new [company]Amazon[/company] Tag Editor makes it easier for admins to group resources on a logical basis whereas before they had to tag them “service-by-service, region-by-region” according to the AWS blog post.

Admins can also then allocate a group of resources that share one tag or more and the group can span regions and services. This, said AWS, creates a “custom console that organizes and consolidates the information you need on a per-project basis.” There’s more here from IDG news.

Over the past six months or so, AWS has launched features to appeal to Windows admins, their counterparts in the VMware  world; directory services, and a service catalog for its cloud products that some think will soon encompass third-party services in the cloud and on premises as well.

New Google SDK for data heads

Also this week in cloud,  Google said it is open-sourcing the software development kit (SDK) for Cloud Dataflow, which itself was announced (in alpha form) at Google I/O in June. Dataflow’s goal is to make it easier for analysts, data scientists and developers to access large data sets for their work.

Per the [company]Google[/company] Cloud Platform blog post, the SDK

… introduces a unified model for batch and stream data processing. Our approach to temporal based aggregations provides a rich set of windowing primitives allowing the same computations to be used with batch or stream based data sources. We will continue to innovate on new programming primitives and welcome the community to participate in this process.

This SDK sort of compares to the AWS Kinesis Java client while Cloud Dataflow overall will contend with Amazon’s Kinesis service, which, as one developer pointed out “is available now, fully supported and not in alpha or beta.”

Google Cloud Platform logo

Docker boss responds to CoreOS fracas

Who knew that containers could stir so much passion? This week on the Structure Show, [company]Docker[/company] CEO Ben Golub talks about the dustup CoreOS started with its Rocket container news and touches on other hot topics as well. So check out Derrick Harris’ and Jonathan Vanian’s chat with Golub about halfway through, but if you want to hear about our take on how [company]Hortonworks[/company] and [company]New Relic[/company] were faring in their first week as publicly held companies, start at the beginning.

Ben Golub, CEO of Docker

Ben Golub, CEO of Docker

[soundcloud url=”https://api.soundcloud.com/tracks/182147043″ params=”color=ff5500&auto_play=false&hide_related=false&show_comments=true&show_user=true&show_reposts=false” width=”100%” height=”166″ iframe=”true” /]

 

Download This Episode

Subscribe in iTunes

The Structure Show RSS Feed

CoreOS CEO: We’re not out to replace Docker, just its containers

There was a major shakeup in the world of container-based computing this week when operating system provider CoreOS decided to get into the container space with a new open source project called Rocket. It’s a container runtime environment as well as a set of specifications for how App Containers — what CoreOS calls its container images — are built and function. But the bigger news industry-wide was the suggestion from CoreOS that it built Rocket because developer darling Docker isn’t living up to expectations.

CoreOS Co-founder and CEO Alex Polvi came on the Structure Show podcast this week to clarify that message and to explain the rationale behind Rocket and everything CoreOS does. If you’re interested in the future of containers, distributed systems and even cloud computing, both business-wise and technologically, it’s a must-listen interview. Here are some highlights, but there’s a lot more good stuff.

[soundcloud url=”https://api.soundcloud.com/tracks/179884272?secret_token=s-mb5PC” params=”color=ff5500&auto_play=false&hide_related=false&show_comments=true&show_user=true&show_reposts=false” width=”100%” height=”166″ iframe=”true” /]

Download This Episode

Subscribe in iTunes

The Structure Show RSS Feed

We’re fine with Docker, really!

If there’s one point that Polvi really wants to get across, it’s that CoreOS didn’t build Rocket because it doesn’t like Docker — either the technology or the company. He called that notion — expressed by the media, as well as, in numerous fora, Docker founder and CTO Solomon Hykes — “fundamentally flawed.”

The rationale behind Rocket is simple, Polvi explained. Docker is turning into more of a platform, adding in features around cluster management, networking and booting cloud servers, and CoreOS wanted to make sure that the original, simple container component didn’t get lost to the world as that happens. In fact, he says he’s fine with the idea of a Docker platform:

[blockquote person=”” attribution=””]”That’s a fine product, the private cloud is an open territory right now still. So the Docker platform is a product that needs to exist. We just want the simple composable building block to also exist for people that have their own platforms or they’re trying to build their own platform to use as a reusable component.”[/blockquote]

Although, below the surface, it might not be the mutual respect society the companies would like everyone to think it is. Later, while comparing Docker’s move away from containers to VMware’s move away from virtual machines, Polvi noted, “There is a debate as to whether the technology warrants another company like VMware to emerge.”

CoreOS CEO Alex Polvi

CoreOS CEO Alex Polvi

We build what we have to

When you consider the CoreOS business strategy, the reasons for Rocket begin to look a little more clear. Polvi calls the CoreOS lineup of technologies, which also includes a database, registry service, cluster management and other pieces, “a platform for platform builders.” It’s building the “primitives” that people need to build next-generation distributed systems and platforms, as opposed to actually building the platforms (think Heroku or CoreOS partner Deis) where people ultimately deploy applications.

“We are never trying to just take somebody else’s solution and build it,” Polvi said. “We’re trying to fill in the white space and build something that’s technically sound in an area we think is an open problem.”

He contrasts this with Docker, which he says is now becoming more akin to cluster (and container) management plays such as Mesosphere and the Kubernetes project, or VMware. Those technologies might use containers and let users move them around and manage them, but they’re far more about the management aspect than about the containers, or any other pieces of infrastructure, themselves.

Kubernetes works levels above the container, which isn't mentioned on this diagram from Microsoft.

Kubernetes works levels above the container, which isn’t mentioned on this diagram from Microsoft.

In fact, despite the fact that CoreOS has its own cluster-management tool, called Fleet, Polvi said the company actually contributes quite a bit to the Google-led Kubernetes project because it really likes the technology and the trajectory the project is on.

“Docker was a similar thing early on,” he added. “We used it for a year, we collaborated heavily with that community, but then it became clear they were on a trajectory that was no longer what we needed — and what a lot of people needed, not just us.”

Still, Polvi noted, technically, there’s no reason why Docker containers and Rocket can’t coexist provided Docker is willing to work within CoreOS’s container specifications or collaborate with CoreOS to develop a standard container format.

Structure 2010: Sebastian Stadil – CEO, Scalr; William “Skip” Bacon – VP of Products and CTO, Virtual Instruments; Michael A.Jackson – Co-Founder, President, and COO, Adaptive Computing; Jagan Jagannathan – Founder and CTO, Xangati; Alex Polvi – CEO and Co-Founder, Cloudkick; Javier Soltero – CTO for Management Products, SpringSource

A younger Polvi (far left) talking cloud a Structure 2010.

A quick thought on the cloud

We also asked Polvi about the world of cloud computing, where he used to work after Rackspace acquired his last startup, CloudKick, and where many CoreOS workloads will likely run. Maybe old allegiances just die hard, but Polvi thinks Rackspace is actually in a pretty good position as bigger cloud providers such as Amazon Web Services, Google and Microsoft continue to drive down prices.

“Now, because of the competitive pressure of the cloud providers, compute on infrastructure will go asymptotically to free over time, as well,” he said. “If you think about it, what’s left after the hard parts of software are free and the compute itself is relatively free, or free enough? … I think it’s service, that’s how you do it. You help people use all this stuff.”

Amazon’s Jeff Bezos talks succession plan (sorta)

The week in cloud

Last week, Amazon CEO Jeff Bezos talked a bit on stage about what Amazon would be without him. At a Business Insider event, Bezos was asked the zillion-dollar succession question. After the slow-mo Microsoft CEO search we endured last year, that topic resonates.

And the answer was: Yes, [company]Amazon[/company] does have a CEO successor lined up should something happen to Bezos, as it does with all senior managers, presumably including the Amazon Web Services brain trust of Andy Jassy and Werner Vogels.

So who is that successor? Bezos put down the hammer: “It’s a secret.”

Also interesting is Bezos’ take on how Seattle-based Amazon differs from stereotypical Silicon Valley tech companies. There are no free gourmet-lunches and no suburban Montessori-school-like headquarters. Amazon’s culture revolves around its urban HQ and a definitive butt-in-chair work ethic.

And, in Bezos’ opinion, taking a job with the company that offers the best free massages may not be the optimal career choice for a young engineer. More here from Gigaom’s Jeff Roberts on the event..

The full segment is here.

Container-on-container violence

If you were on a tech media news blackout last week, you missed the dustup touched off when CoreOS launched its own Rocket container, in a move seen as a direct shot at Docker.

CoreOS’s contention is that while Docker adds orchestration and other trimmings to its container it’s sort of neglecting the container itself and that its process model is “fundamentally flawed.”

Docker cried foul. If a company wants to just use just the Docker container, it can do so although Docker’s “batteries included but removeable” slogan isn’t helping its cause.

It makes total sense for CoreOS to glom onto what it sees as a great thing to get a piece of the action, which is how many people see what’s transpiring. “Every time some piece of technology gains traction you see all these prospectors staking claims,” said one long time observer who thinks CoreOS hasn’t done itself any favors the way it went about this, blasting out a blog post on the eve of Dockercon.

This gold rush happened with Rails, it happened with Node.js, it happened with Linux, and now it’s happening with Docker, he said.

But it also looks like there’s fear that the Docker “platform” will encroach on stuff offered by third parties, which Docker CTO and Founder Solomon Hykes addressed in comments to this blog post by CloudCredo CEO Colin Humphreys. Hykes wrote:

“The fact that Docker is building a platform does not at all mean that ‘docker’, the command-line tool, will become bloated and monolithic. Quite the contrary! The major theme of Docker is in fact the exact opposite: to make it more modular, and make it easier to use one part without the other. In fact, the very features mentioned by the CoreOS blog post (machine management, clustering) will not, in fact, be incorporated into the Docker binary.”

Below check out a couple of relevant Structure Show podcasts. First, last week’s show with CoreOS CEO Alex Polvi in which he says there’s nothing wrong with Docker going the platform route, but people need just plain containers. Then one from July in which Hykes discusses (presciently) how tricky Docker governance was going to be going forward.

[soundcloud url=”https://api.soundcloud.com/tracks/179884272?secret_token=s-mb5PC” params=”color=ff5500&auto_play=false&hide_related=false&show_comments=true&show_user=true&show_reposts=false” width=”100%” height=”166″ iframe=”true” /]

CoreOS CEO Alex Polvi

CoreOS CEO Alex Polvi

 

[soundcloud url=”https://api.soundcloud.com/tracks/158076928″ params=”color=ff5500&auto_play=false&hide_related=false&show_comments=true&show_user=true&show_reposts=false” width=”100%” height=”166″ iframe=”true” /]

 

Docker Founder and CTO Solomon Hykes at DockerCon 2014

Docker Founder and CTO Solomon Hykes at DockerCon 2014

 

 

 

SHOW NOTES

Hosts: Barbara Darrow and Derrick Harris

Download This Episode

Subscribe in iTunes

The Structure Show RSS Feed

PREVIOUS EPISODES:

More from Facebook on its new networking architecture 

Do you find OSS hard to deploy? Say hey to ZerotoDocker

All about AWS Re:Invent and oh some Hortonworks and Microsoft news too

Ok, we admit it. Google may be serious about cloud after all

EMC cloud really isn’t all about EMC hardware, says EMC product chief