Microsoft has committed to modular containerised data centers in a big way. It will put 200 of them under one roof in a Chicago facility. Mike Manos, the man responsible, tells Ambrose McNevin why and how
It took a little under a decade. From the 1950s container shipping accelerated the world economy, setting off a chain reaction which led to globalised manufacturing and products that were specifi cally designed to fi t neatly into standard 40`metal boxes. Cost of transport was pushed ever lower and goods fl owed across the globe at an unprecedented scale. A strategic infl ection point had occurred and the world economy changed forever.
Now almost sixty years on from first use, containers are back in the news, this time to house data centers.
Some believe containers are a step on the path to industrialisation of the data center. If the components of a data center become commoditised to fit into a standard metal box and the metal box itself becomes the commodity then someone puts several hundred under one roof with standardised power and cooling, will we have a standardised, scalable industrialised data center?
In the Spotlight
Mike Manos, general manager for Microsoft's data center operations doesn't claim to be attempting to standardise the data center through building a major facility using connected modularised data centers housed inside shipping containers.
Nor is Manos claiming that his plan to put up to 200 server-packed shipping containers into a single Chicago data center heralds a sea change in all data center design and operation. He does however admit that it has touched a nerve of those currently investing in traditional set ups and he knows his project is being as closely watched as any build ever undertaken.
"What containers represent is a scale unit. Each container has servers or storage, depending on applications, pluged in to the power, water and the network. That is the reason that this is an interesting business case that data center operaters should take notice of. They should be thinking "if I'm going to build a data center should I get involved in the religious wars, about water and air, air cooled versus water cooled. What containers bring is a conversation change. I've got a central spine of services. Is it AC, is it DC, I don't care,' he says.
By rolling 200 containers into a single facility which it will manage as a single operation, Microsoft's site will be the largest publicly acknowledged container-based data center in the world. Google is rumoured to have built at least some of its facilities using containers but (at time of going to press) the search giant has stuck to its usual stance of not commenting about the infrastructure set up within its data centers.
Some Difference Data centers inside shipping containers have been around for a while and have been used to good effect as overspill capacity in crowded sites or to bring data centers to remote locations (see 'Contain your excitement'). The reason for nervousness among traditional data center operators, believes Manos, is the scale at which Microsoft is deploying its containers. By very publicly putting so many of them inside a single facility Manos is challenging the orthodoxy of data center set up of raised floors, continuous racks, hot aisle, cold aisle confi guration and AC or DC power in and air or liquid cooling out.
Manos is less concerned with what is happening inside container than what happens when they start to work together. In containerised data center vendor terms Manos says it is hard for him to say which manufacturer has the lead. "People think about the container - the block and tackle, server optimisations in bins and cooling. From a commodity level, all are commodities that are packed differently. That's one of the things we were stuggling with. Containers have been out for a little while but it is still diffi cult to differentiate from one vendor or another.'
For Manos, using containers at scale was what was lacking and tackling the connectivity issues means seeing how containers behave at massive scale and he believes that brings him closer to standardising a specifi cation where there hadn't been one before.
"There is an optimum set up - containers aren't for everyone - we are using them specifically to address our scale problem. If you are deploying 10,000 to 20,000 servers per month it can't be done efficiently if every new server needs to be hand crafted by a server artisan. You need to get it out of the box.'
"From a container perspective - and that's what we see as the technology set - there is measurement of total return on investment and total cost of ownership of going to that model. A big component is driven by our scale - to be able to deploy that much technology is key. We wouldn't invest in it if we didn't have the applicability. You could find 100 people to say why it [large scale containerised data centers] won't work but none of them have built one."
Manos explains the thinking behind his management choices while admitting that Microsoft may be uniquely positioned. The company's investments in software automation and management capabilities and the knowledge existing within the company means he can, for example, plug in a server and see instantly its function, is it a hotmail box, is it an exchange box?
To the operators who are finding comfort in their traditional way of thinking and are happy to stick to their religion Manos makes the point that using his method of data center building means that now he has a process where he can easily pull out a container and upgrade his data center without any disruption. And he is not stuck in the same data center infrastructure for the next 20 years. "I have the ability to upgrade on the go with the minimum of distruption. What I can do can be measured by choice,' he said.
Efficiency Measures Because containers are operating at 1,000 watts per square foot they allow many more servers to be powered in a given area meaning better power usage effectiveness (PUE) numbers.
On the topic of PUE Manos believes his data center is very well positioned when compared with others. (see Sum of all things?).
When asked: How granular is your PUE measurement? Can you start measuring power drawn by each container? He says: "[We are] unbelievably insane in methodology. We have a million points of measurement in all of our facilities. We believe that containers and containment in general allow us to drive a more significant PUE number than traditional environments.' Manos says he has measured PUE's of 1.3 saying "we load them up and went to 100% utilisation at disk and CPU and found real time monitoring PUE ranging between 1.3 and 1.4.'
Microsoft has aggressive PUE targets, with the crossover point to a PUE of below 1.2 expected to happen sometime in 2010-11 and to continue toward 1 in 2012. But for the moment we remain in a world were reliability and redundancy are the natural enemies of performance.
How Much?
Manos is more reticent when it comes to discussing total cost of ownership of running a containerised data center. As it is not up and running yet, he is not willing to comment on exactly the level or type of running costs he expects from his facility - though he says that he has run the numbers and it will save him money compared with running a traditional data center. (We agree that it is a subject worth returning to six months into the facility being fully operational.) Once operational, Manos' data center will have something else in common with a container ship. It will be a lonely place work. Including "janitors and gardeners' it will never employ more than 35 people. Even by the sparsely populated data center operations of today, the number is low.
That Strategic Inflection Point Manos is an engineer. A practical man dealing with practical data center problems. But he is not blind to why he has been tasked with building a huge modular data center.
Chicago will be key to managing the growth in Microsoft's online products and services - services such as hosted Exchange along with the full Microsoft Live online product suite. Having 500 million unique users (of Microsoft Live products) means a lot of time is spent adding more features and functionality and all that breaks down to infrastructure requirements, he says.
But ask him how many more data centers will be built to support Microsoft's cloud applications or if they will be built using the container model and he won't say. A data center based global infrastructure is being built and Microsoft clearly believes that one effective and efficient way to build this is through a modular approach. (See Cloud Computing and the Virtualised Data Center)
"We are at the information utility dawn. A little over 100 years ago society realised that electricity was pretty cool but that the problem was how to get into the hands of the users. The question was how do I distribute this? This was governed by two aspects, one was products and features and the other was infrastructure reliability. Data centers are the electricity sub-stations of tomorrow. Infrastructure is key. You have to have a global footprint. For cloud computing you need a global scale infrastructure The telephone is useless with out the live dial tone. The expectation is that the information source will come through the cloud and that it will always be on.' Wagons roll.
