Google has published a paper about its latest big data system, a globally distributed data warehouse called Mesa that can ingest millions of rows in minutes and even survive a data center failure.
Marathon is a new framework that turns Mesos — a favorite of Twitter — into a more dynamic tool for running different applications on a single set of machines. Marathon comes from a startup called Mesosphere, founded by two former Airbnb engineers who know Mesos cold.
Netflix has forged a cottage industry building tools to fill gaps in Amazon’s cloud infrastructure or otherwise add value to it. Now the company is open sourcing its Hystrix libraries, which manage interactions between the myriad distributed services that power its applications.
Modern web-scale applications like Google, Twitter, Netflix, LinkedIn, and others are implemented as distributed systems as opposed to single monolithic codebases. Surprisingly, modern startups are generally built the same way. So what can startups learn from web apps?
Amazon DynamoDB brings the power of the cloud to NoSQL databases, but the storage looks expensive for big data apps and most developers are still unclear about what “eventual consistency” means.