Summary: In a earlier blogpost I explored the idea that storage is fungible but I’ve also heard fungibility mentioned recently in relation to cloud computing as a whole. If cloud computing is becoming a commodity (which is another argument) why shouldn’t it be traded like any other commodity, with a marketplace, brokers, futures trading etc? Are we going to see cloud compute traded much like gas or electricity?
Strategic Blue’s presentation on ‘Cloud brokers’ at CloudCamp London back in October 2012 centered around this exact idea and generated plenty of animated discussion on the night. Some felt that this was a pipe dream whereas others felt it was inevitable. My wife’s a commodity trader so after returning home I had various discussions with her trying to understand the concepts of commodity trading. It’s harder than I thought! The more technically minded of us immediately started thinking about issues like compatibility, interoperability, and service maturity but apparently (and somewhat surprisingly) these are all irrelevant when it comes to a true marketplace. It’s not the current IT providers that will define and run the cloud computing markets which is an idea that takes a bit of getting used to!
In a fascinating article, What’s required for a utility market to develop?, Jack Clark identified & scored the various criteria which need to be satisfied before cloud computing can be considered a utility. He gave it 7/10 (which is probably higher than I would have expected) but two of the requirements in particular struck a chord with me;
- a transmission and distribution capability – represented in the cloud by datacentres and networks
- a market mechanism – typically an exchange (like the FTSE or NASDAQ)
Let’s investigate these two criteria.
Transmission and distribution
For a commodity to be tradeable you need to be able to move it around to where it’s needed in a cost effective manner. In my mind the transmission and distribution of compute capacity has only been viable since the introduction of virtualisation which has massively increased the portability of workloads. It’s the ability to move a workload from one provider to another (via encapsulation) that opens up the potential for a cloud computing market – prior to that the effort and time required to either rebuild the application or physically relocate hardware would likely have defeated any economic gain. Knowing that another cloud provider can host your workload cheaper than your current provider is no use if you can’t move your workload across.
That’s not to say distribution and transmission are perfect yet. At the IaaS level there are still plenty of technical challenges with different hypervisors and myriad incompatible standards, and the application tier is little better. Do enterprise applications need to be rewritten to be effective in the cloud? Some think they do. In a thought provoking article titled ‘The Importance of being fungible‘ analyst Dana Gardner refers to ‘application fungibility’ to describe the portability of applications (or lack of) across infrastructures. He feels this is a necessary for a market to develop;
“Application fungibility would, I believe, create a real marketplace for cloud services, something very much in the best interest of enterprises, small and medium businesses (SMBs), independent software vendors (ISVs), and developers.”
These are all technical issues however – does that mean they shouldn’t interfere with development of a utility market? Answers on a postcard please!
A market mechanism
Before a commodity can be traded on an exchange it must be fungible – this allows a broker to buy capacity from multiple providers and provide it as necessary to the consumers. For cloud computing to be considered http://premier-pharmacy.com fungible there must be a standard – something to allows ‘comparison’ between disparate providers offerings, thus providing a pricing framework. Referring to this second criteria, a market mechanism, Jack Clark concluded;
“There is not yet a clear market mechanism for homogenising compute and storage from different providers making them truly interchangeable. There have been some attempts to create a true market for this – like Spotcloud – though there is little evidence these services have caught on.”
In other words the lack of fungibility is currently holding back the development of cloud computing as a utility.
Dr James Mitchell from Strategic Blue argued much the same at CloudCamp – today there is no agreed ‘standard’ by which a cloud provider can be evaluated and without it no way of comparing one providers offerings to another and understanding their relative value in the market. Many commodities aren’t fungible by nature so commodity markets often ‘invent’ an arbitrary measurement which allows them to specify the nature of the commodity (e.g the purity of a metal being traded) sufficiently tightly for it to be treated as fungible. For example coal is traded using ‘benchmark coal’ which doesn’t exist in real life – nobody digs up benchmark coal yet it’s used as a ‘standard’ to enable trade. I’d strongly recommend listening to Brian Gracely’s Cloudcast with Dr James Mitchell as they discuss the development of a cloud computing ‘standard’ and some of the challenges around it.
Even if the industry as a whole haven’t created or adopted such a standard there’s no saying someone can’t step it to bridge the gap and that’s exactly what at least one company has done. 6Fusion provide a ‘standard’ for measuring resource consumption which makes it possible to compare providers who may themselves use different metrics. Called ‘the Workload Allocation Cube‘, or WAC for short, it comprises the six resources consumed by cloud compute;
- CPU
- Memory
- Storage
- Disk I/O
- LAN I/O
- WAN I/O
By providing this layer of abstraction they’re attempting to make cloud computing fungible and hence allow a market to develop. For a good overview of what 6Fusion do check out their video below;
I’ve no idea how much traction 6Fusion or Strategic Blue are getting in the marketplace today but as I noted in my container shipping analogy it took over a decade for the container to reach a level of standardisation where it was considered fungible and that’s surely a simpler construct. No market is going to develop around a ‘standard’ that’s owned by a single company so while 6Fusion may offer a valuable service I can’t see their solution enabling a trading marketplace.
If you’re interested there’s a GigaOM webinar tonight (23rd May 2013), “IaaS marketplaces – the impact of utility computing” discussing many of these issues;
- What lessons can we learn from previous marketplace failures and successes?
- What is the role of IaaS brokers and what new products will they develop?
- What is the impact of potential government regulation and oversight?
- How will IaaS marketplaces affect staffing, operations and cost?
Further Reading
UPDATE: 19th May 2015 – another good article at Forbes.com (via Reuben Cohen, creator of SpotCloud). Includes estimated size of market ($8Bn) compared to other commodities.
Cloud Computing’s utility future gets closer
A market for computing power (The Economist)
Evolution begets Genesis begets Evolution
Developing the world’s first IaaS marketplace (Zimory)
Cloud spot markets – what they are and why you should care
Spotcloud – a Cloud Capacity clearing house
SpotCloud aims to create online spot market for cloud
The value and utility of cloud computing
Cloud broker or exchange – what’s the difference?