Red Hat’s new operating system will power up your containers

Open-source software giant Red Hat said on Thursday that its new operating system custom made to power Linux containers is now available to download. Red Hat has been a big proponent of Docker and its container packing technology going back as far as last summer touting its support of the startup and making sure its Enterprise Linux 7 product was compatible with Docker’s technology.

Container technology has generated a lot of buzz over the past year by promising a type of virtualization that’s lighter weight than your typical virtual machine. In order for a container to actually run, it needs to be connected to a host Linux OS that can distribute the necessary system resources to that container.

While you could use a typical Linux-based OS to power up your containers, as CoreOS CEO Alex Polvi (whose own startup offers a competing container-focussed OS) told me last summer, these kinds of operating systems merely get the job done and don’t take full advantage of what containers have to offer.

Red Hat’s new OS supposedly comes packed with features designed to make running containerized applications less of a chore to manage. These features include an easier way to update the operating system (OS updates can often be a pain for IT admins) and an integration with Google’s Kubernetes container-orchestration service for spinning up and managing multiple containers.

The new OS is also promising better security for those Docker containers — which has been an issue that Docker’s team has been addressing in various updates — with a supposed stronger way of isolating containers from each other when they are dispersed in a distributed environment.

Of course, [company]Red Hat[/company] has some competition when it comes to becoming the preferred OS of container-based applications. CoreOS has its own container-centric OS and Ubuntu has its Snappy Ubuntu Core system for powering Docker containers. Additionally, a couple of the former Cloud.com veterans who recently departed Citrix in September have started their own startup called Rancher Labs that just released RancherOS, which the startup describes as a “minimalist Linux distribution that was perfect for running Docker containers.”

It will be worth keeping an eye on which OS gains traction in the container marketplace and whether we will see some of these new operating systems starting to offer support for CoreOS’s new Rocket-container technology as opposed to just the Docker platform.

A Red Hat spokesperson wrote to me in an email that “Red Hat Enterprise Linux-based containers are not supported on CoreOS and rocket is not supported with Atomic Host. We are, as always, continuing to evaluate new additions in the world of containers, including Rocket, with respect to our customer needs.”

Chromebooks can now run Linux in a Chrome OS window

This is cool: Chromebook users can now run their favorite Linux distribution within a window right on their Chrome OS desktop. Google’s own happiness evangelist François Beaufort revealed with a Google+ post Tuesday that Chromebook oners who have set their device in developer mode can download special Crouton Chrome extension to run Linux without being forced to switch back and forth between the two operating systems.

Screenshot 2014-12-29 at 11.09.09 AM

Running Linux on a Chromebook is not a new thing. Chrome OS is based on the Linux kernel, and there are a number of ways to run both Chrome OS and Linux on the device. My colleague Kevin Tofel highlighted three ways of accessing Linux (and other operating systems as well) a while back, and he even recorded a video of using Crouton to run Chrome OS and Linux simultaneously, which you can watch below. However, the new Crouton Chrome extension makes it possible for the first time to run Linux in a window.

[youtube=http://www.youtube.com/watch?v=v031udlfY5E]

Mesosphere’s new data center mother brain will blow your mind

Mesosphere has been making a name for itself in the the world of data centers and cloud computing since 2013 with its distributed-system smarts and various introductions of open-source technologies, each designed to tackle the challenges of running tons of workloads across multiple machines. On Monday, the startup plans to announce that its much-anticipated data center operating system — the culmination of its many technologies — has been released as a private beta and will be available to the public in early 2015.

As part of the new operating system’s launch, [company]Mesosphere[/company] also plans to announce that it has raised a $36 million Series B investment round, which brings its total funding to $50 million. Khosla Ventures, a new investor, drove the financing along with Andreessen Horowitz, Fuel Capital, SV Angel and other unnamed entities.

Mesosphere’s new data center operating system, dubbed DCOS, tackles the complexity behind trying to read all of the machines inside a data center as one giant computer. Similar to how an operating system on a personal computer can distribute the necessary resources to all the installed applications, DCOS can supposedly do the same thing across the data center.

The idea comes from the fact that today’s powerful data-crunching applications and services — like Kafka, Spark and Cassandra — span multiple servers, unlike more old-school applications like [company]Microsoft[/company] Excel. Asking developers and operations staff to configure and maintain each individual machine to accommodate the new distributed applications is quite a lot, as Apache Mesos co-creator and new Mesosphere hire Benjamin Hindman explained in an essay earlier this week.

Mesosphere CEO Florian Leibert

Mesosphere CEO Florian Leibert – Source: Mesosphere

Because of this complexity, the machines are nowhere near running full steam, said Mesosphere’s senior vice president of marketing and business development Matt Trifiro.

“85 percent of a data center’s capacity is typically wasted,” said Trifiro. Although developers and operations staff have come a long way to tether pieces of the underlying system together, there hasn’t yet been a nucleus of sorts that successfully links and controls everything.

“We’ve always been talking about it — this vision,” said Mesosphere CEO Florian Leibert. “Slowly but surely the pieces came together; now is the first time we are showing the total picture.”

Building an OS

The new DCOS is essentially a bundle of all of the components Mesosphere has been rolling out — including the Mesos resource management system, the Marathon framework and Chronos job scheduler — as well as third-party applications like the Hadoop file system and YARN.

The DCOS also includes common OS features one would would find in Linux or Windows, like a graphical user interface, command-line interface and a software-development kit.

These types of interfaces and extras are important for DCOS to be a true operating system, explained Leibert. While Mesos can automate the allocation of all the data center resources to many applications, the additional features provide coders and operations staff a centralized hub from which they can monitor their data center as a whole and even program.

“We took the core [Mesos] kernel and built the consumable systems around it,” said Trifiro. “[We] added Marathon, added Chronos and added the easy install of the entire package.”

To get DCOS up and running in a data center, Mesosphere installs a small agent on all Linux OS-based machines, which in turn allows them to be read as an “uber operating system,” explained Leibert. With all of the machines’ operating systems linked up, it’s supposedly easier for distributed applications, like Google’s Kubernetes, to function and receive what they needs.

The new graphical interface and command-line interface allows an organization to see a visual representation of all of their data center machines, all the installed distributed applications and how system resources like CPU and memory are being shared.

If a developer wants to install an application in the data center, he or she simply has to enter install commands in the command-line interface and the DCOS should automatically load it up. A visual representation of the app should then appear along with indicating which machine nodes are allocating the right resources.

DCOS interface

DCOS interface

The same process goes for installing a distributed database like Cassandra; you can now “have it running in a minute or so,” said Leibert.

Installing Cassandra on DCOS

Installing Cassandra on DCOS

A scheduler is built into DCOS that takes in account certain variables a developer might want to include in order to decide which machine should deliver resources to what application; this is helpful because it allows the developer to set up the configurations and the DCOS will automatically follow through with the orders.

“We basically turn the software developer into a data center programmer,” said Leibert.

And because DCOS makes it easier for a coder to program against, it’s possible that new distributed applications could be made faster than before because the developer can now write software to a fleet of machines rather than only one.

As of today, DCOS can run on on-premise environments like bare metal and OpenStack, major cloud providers — like [company]Amazon[/company], [company]Google[/company] and [company]Microsoft[/company] — and it supports Linux variants like CoreOS and Redhat.

Changing the notion of a data center

Leibert wouldn’t name which organizations are currently trying out DCOS in beta, but it’s hard not to think that companies like Twitter, Netflix or Airbnb — all users of Mesos — haven’t considered giving it a test drive. Leibert was a former engineer at Twitter and Airbnb, after all.

Beyond the top webscale companies, Mesosphere wants to court legacy enterprises like those in the financial-services industry who have existing data centers that aren’t nearly as efficient as those seen at Google.

Banks, for example, typically use “tens of thousands of machines” in their data centers to perform risk analysis, Leibert said. With DCOS, Leibert claims that banks can run the type of complex workloads they require in a more streamlined manner if they were to link up all those machines.

And for these companies that are under tight regulation, Leibert said that Mesosphere has taken security into account.

“We built a security product into this operating system that is above and beyond any open-source system, even as a commercial plugin,” said Leibert.

As for what lies ahead for DCOS, Leibert said that his team is working on new features like distributed checkpointing, which is basically the ability to take a snapshot of a running application so that you can pause your work; the next time you start it up, the data center remembers where it left off and can deliver the right resources as if there wasn’t a break. This method is apparently good for developers working on activities like genome sequencing, he said.

Support for containers is also something Mesosphere will continue to tout, as the startup has been a believer in the technology “even before the hype of [company]Docker[/company],” said Leibert. Containers, with their ability to isolate workloads even on the same machine, are fundamental to DCOS, he said.

Mesosphere believes there will be new container technology emerging, not just the recently announced CoreOS Rocket container technology, explained Trifiro, but as of now, Docker and native Linux cgroup containers are what customers are calling for. If Rocket gains momentum in the market place, Trifiro said, Mesosphere will “absolutely implement it.”

If DCOS ultimately lives up to what it promises it can deliver, managing data centers could be a way less difficult task. With a giant pool of resources at your disposal and an easier way to write new applications to a tethered-together cluster of computers, it’s possible that next-generation applications could be developed and managed far easier than they use to be.

Correction: This post was updated at 8:30 a.m. to correctly state Leibert’s previous employers. He worked at Airbnb, not Netflix.

Benjamin Hindman, co-creator of Apache Mesos, joins Mesosphere

Apache Mesos co-creator and former Twitter lead engineer Benjamin Hindman is joining the startup Mesosphere, which provides commercial backing for the open-source Mesos resource management system. Hindman will help Mesosphere build out its first distributed operating system (OS) that will use Mesos as the OS kernel, which is essentially the part of the OS that’s responsible for managing system resources. Twitter, Airbnb, PayPal and Netflix are all companies that use Mesos to help manage their infrastructure. In June, Mesosphere announced a $10.5 million series A round.