It’s Black Friday, so who has the lowest cloud prices?

Who’s really got the cheapest cloud? The answer, alas, is “it depends.”

In today’s brave new IT world, there are few things more downright confusing than public cloud pricing. Every provider has a different way of T-shirt sizing infrastructure as well as pricing compute, memory, storage, bandwidth, OS, and support.

RBC Capital Markets recently advanced the cause of comparative cloud pricing by showing how a single metric — spend per GB of RAM — could be used to compare cloud providers. Though convenient, this method is not enough to figure out which is the lowest cost cloud provider for your particular application and workload needs.

To get to a more accurate pricing comparison, you would need to use a technique similar to the market basket pricing approach used in economics — i.e., a collection of components in a basket (e.g. storage, networking, RAM, CPU, support). But there is no one-size-fits-all cloud market basket when it comes to applications. This is because each app requires a different mix of infrastructure and services to meet its needs. For example, a typical medium-sized collaboration application might need eight app servers, four database servers, 5 TB of storage and 2,400 GB/month, while a medium-sized CRM app might need eight web servers, four app servers, four database servers, 3 TB of storage and 800 GB/month.

Since each cloud provider prices the individual infrastructure and services components differently (and changes price on each component differently and asynchronously), an accurate comparison of cloud provider pricing would need to be application-specific, workload-specific, and time-stamped.

Using an application- and workload-specific approach, we compared four major cloud providers (AWS, Google, IBM and Microsoft Azure) across nine applications. We assumed the application operated on a light workload and on optimized infrastructure. Infrastructure sizing was based on what we have seen as typical in customer deployments. We also included support with less than 1-hour SLA (anything else wouldn’t be enterprise-class).

We ran two scenarios: First, a medium-sized application infrastructure across these applications and, second, a large-sized infrastructure. All prices were current as of November 26, 2014 and included the most recent Google price cuts as well as the September introduction of new (D class) instance sizes at Microsoft Azure.

The results, showing approximate monthly pricing, are shown in the tables below. Note that the prices derived are for specific application deployments — your own mileage will vary depending on the specifics of your application and workload and the size of infrastructure needed.

Table 1: Provider comparison — Medium sized application infrastructure

Application Lowest 2nd Lowest 3rd Lowest
Business Intelligence Amazon $1,616 Microsoft $2,929 IBM $4,570
CDN Web App IBM $4,866 Amazon $5,071 Google $6,075
Collaboration Google $10,360 IBM $12,346 Amazon $12,713
Disaster Recovery IBM $1,126 Amazon $1,284 Google $1,924
Enterprise Portal App IBM $5,295 Amazon $5,429 Google $5,836
Enterprise Web App IBM $13,051 Microsoft $13,545 Amazon $13,828
HPC Amazon $3,291 Microsoft $6,232 IBM $8,493
Mobile Development Amazon $2,422 Google $2,480 IBM $2,655
OLTP IBM $2,543 Amazon $2,772 Google $3,397

We see that the lowest provider depends by application and there is no single provider that is lowest for all applications. The mean price difference between lowest and second lowest price was 25 percent,with a standard deviation of 33 points.

If we change the size of the application infrastructure, we get different comparison results:

Table 2: Provider comparison — Large sized application infrastructure

Application Lowest 2nd Lowest 3rd Lowest
Business Intelligence Amazon $6,469 Microsoft $8,867 IBM $18,531
CDN Web App Amazon $15,642 IBM $16,464 Microsoft $17,678
Collaboration Google $27,358 IBM $33,156 Amazon $33,495
Disaster Recovery IBM $4,756 Amazon $5,120 Google $6,496
Enterprise Portal App IBM $21,181 Amazon $21,417 Microsoft $22,138
Enterprise Web App Microsoft $44,652 IBM $45,252 Amazon $47,565
HPC Amazon $13,109 Microsoft $21,878 IBM $34,221
Mobile Development Google $8,163 Amazon $9,004 IBM $9,912
OLTP IBM $10,421 Microsoft $10,785 Amazon $11,016

What drives the difference in results between table 1 and table 2? In part, the answer is how each provider prices components (such as memory) as they scale in size, but it is also because of offering granularity. Granularity refers to the resolution of instance sizes available by a particular provider. The more granular the offering, the easier it is to match to your application sizing need without having to step up to the next highest instance level.

Granularity also helps explain the large differences in pricing between the lowest and second-lowest provider for some of the applications (e.g. HPC and BI). Both Amazon and Microsoft offer low-priced high compute and memory instances (lower than IBM). But Amazon wins because Microsoft’s lower instance resolution (i.e. less granular offering) requires customers to bump up to the next instance level to meet minimum performance requirements, thus raising the price.

IBM comes out ahead in many applications because it allows users to independently choose an instance’s CPU and memory at a granular level, making it easier to match the application needs.

Google comes out ahead in Collaboration applications because its high memory instances are a strong match for such applications and are competitively priced against Amazon and IBM.

However, price isn’t everything. We caution cloud users from just choosing the lowest cost provider for any given application. Cost is only one decision variable. Other important criteria are performance, location and features (e.g. security, VM scalability, file based storage, etc). A customer may decide to choose a more expensive cloud provider because they get higher performance or desired support, location, network security, etc.

At the end of the day, the real question should not be “which is the cheapest cloud?” but “which is the best-fit cloud for my application and business need?”

Praveen Asthana is chief marketing and strategy officer at Gravitant, a software company that enables IT organizations to become cloud service brokers. Thanks to Aaron Yan, pricing analyst at Gravitant, for crunching the numbers.