7076162
story
Comments: 135 +- IT: Cooling Bags Could Cut Server Cooling Costs By 93% on Tuesday November 17, @11:13AM
background: url(//a.fsdn.com/sd/topics/topicstorage.gif); width:48px; height:51px;
storage
background: url(//a.fsdn.com/sd/topics/topicit.gif); width:75px; height:61px;
it
judgecorp writes "UK company Iceotope has launched liquid-cooling technology which it says surpasses what can be done with water or air-cooling and can cut data centre cooling costs by up to 93 percent. Announced at Supercomputing 2009 in Portland, Oregon, the 'modular Liquid-Immersion Cooled Server' technology wraps each server in a cool-bag-like device, which cools components inside a server, rather than cooling the whole data centre, or even a traditional 'hot aisle.' Earlier this year, IBM predicted that in ten years all data centre servers might be water-cooled." Adds reader 1sockchuck, "The Hot Aisle has additional photos and diagrams of the new system."
Related Stories
Executive ability is prominent in your make-up.
All trademarks and copyrights on this page are owned by their respective owners. Comments are owned by the Poster. The Rest © 1997-2010 Geeknet, Inc.
Yes, but how much does it cost? (Score:3, Insightful)
Re:Yes, but how much does it cost? (Score:4, Funny)
Parent
Re: (Score:3, Informative)
That's really nifty, and I'm sure it works ok and everything, but... how much does it cost?
Figures cited by Iceotope show that the average air-cooled data centre with around 1000 servers costs around $788,400 (£469,446) to cool over three years. The Iceotope system claims to eliminate the need for CRAC units and chillers by connecting the servers in the synthetic cool bags to a channel of warm water that transfers the heat outside the facility. This so-called “end to end liquid” cooling means that a data centre, fully equipped with Iceotope-cooled servers, could cut cooling costs to just $52,560 - a 93 percent reduction, the company states.
taking the above figures into account as long as the cost to install is under the 200k figure theres an incentive to switch
Re:Yes, but how much does it cost? (Score:5, Insightful)
The idea that the mainboard components are sealed inside a liquid-filled compartment seems like a major point against the system. Extra proprietary vendor lock-in components mean extra costs of owning and operating, which probably offset any savings from cooling... if any.
I'm skeptical that it will significantly reduce cooling costs (Compared to, say, a chilled cabinet system) because the total cooling load stays the same. If you're generating a billion BTUs of heat you still need to remove a billion BTUs of heat. Any savings will only be from the higher energy densities water allows versus air and maybe initial installation.
Plus, based on their exploded view, there is no less than three heat exchanges before it even gets out of the cabinet: Chip to liquid (via heat sink), submersion liquid to module liquid, module liquid to system liquid. Each time to go through an exchange your temperature gradient goes up.
What they need is a system that is compatible with commodity components to leverage low cost hardware against lower cost cooling. Why not fit water blocks directly to existing mainboard layouts and circulate chilled water from the main loop directly through them via manifolds and pump at each rack? You can still enclose the mainbaord and cooling block in a sealed, insulated compartment to eliminate condensation problems, but not being submerged means you can actually repair/upgrade the modules.
=Smidge=
Parent
Re:Yes, but how much does it cost? (Score:4, Interesting)
As for water blocks, I suspect that all the various minor chips in the system would be problematic. Even if your cooling of the CPU, or even the top 3-5 chips, by thermal output, is perfect, there are loads of other components that will heat up and die without airflow. CPU voltage regulators, northbridge, RAM, RAID controllers, ethernet, etc. You can't waterblock them all(at least without a serious redesign that makes using commodity components impossible, or a plumbing scheme that would make Escher wince). Either you go with a hybrid waterblock/conventional air cooling system; which gives you the vices of both; or you have to go with the fluid bath as in this setup.
Parent
Re: (Score:2)
Well what I had in mind is a flat plate (say, aluminum) with water channels in it. On this plate there are two or three protrusions that match the main chip locations that need cooling that are milled to physically contact the chips just like discreet heat sinks would.
You attach the mainboard to this plate like you would attach it to the inside of a normal computer case, only backwards. eg; the screws go through from the back side instead of the component side. This puts the components very close to, if not
Re: (Score:3, Funny)
And mutated sea bass patrolling the aisles to maintain security?
Ugh. (Score:3, Interesting)
For some reason, the filters at work won't let me view the article. Does it happen to mention how much the upfront cost for these bags are?
Re: (Score:2, Insightful)
Like the unpriced bottle of wine at Applebees. If you have to ask...
Re: (Score:2)
It was more of a curiosity thing :-)
I'm wondering about the upfront costs vs. money saved over time after the initial investment.
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Applebees has its purpose...the Monterey Grilled Chicken that the location around the corner from my house serves is amazing (although the rest of their menu is indeed questionable.)
Re: (Score:2)
So: sealed l
Re: (Score:2)
Sounds good to me.
I'm still waiting for the day when it is feasable (physically and economically) to lay down small pipes for coolant directly onto a PCB or between PCB layers. That will bring along the true cooling revolution!
Do I get at least a pair of rubber gloves? (Score:5, Insightful)
Re:Do I get at least a pair of rubber gloves? (Score:4, Insightful)
You pull that server out of the farm and let other servers pick up the slack while you make repairs.
It's hype, based on the assumption that every server on the planet will be virtualized by 2019, and that the separation of hardware from the software that runs on it will allow IT departments ample time to offload work into "the cloud" while they swap out RAM.
Either that or it's made for large datacenters with multiple redundancies and enormous cooling costs. :)
Parent
Re:Do I get at least a pair of rubber gloves? (Score:5, Funny)
What, they won't? Oh man, this virtualization thing is brilliant.
So you virtualize a box, so that, if there's a hardware failure, the box can be brought back up on another machine, with minimal downtime! Also, you can run multiple systems on a box saving money!
We virtualized all our servers around here, went from about 200 servers to 8 machines, each with 16 CPU cores. It went well. So we decided to repeat the process. We then had 4 machines, each taking two VM hosts! It was great, more savings, more vodka for my drawer... So I thought, how could I make this even better...?
That's right, I put all four of THOSE VM hosts on a 486 in the back room that doesn't even need special cooling. Let me tell you, in terms of Vodka, this virtualization thing has been *quite* productive.
How could it not be all pervasive by 2019? I'm sure everyone will be virtualizing all of their VM hosts on VM hosts running on 486s by 2019!
Parent
A few questions (Score:5, Interesting)
Re: (Score:2)
Coming back full circle (Score:2)
Grandma would be proud of her cold compress technology.
Re:Coming back full circle (Score:4, Funny)
Parent
Quick Release (Score:5, Informative)
The problem with all this is you need a good piping and plumbing system in place, complete with quick release valves to ensure you can disconnect or connect hardware without having to do a whole bunch piping and water routing in the process. Part of the beauty of racks is you just slide in the computer, screw it in, and plug in the plugs at the back and you're done.
I'm not saying it's impossible, but just building a new case, or blade, or whatever isn't going to do it - you need a new rack system with built in pipes and pumps, and probably a data center with even more plumbing with outlets at the appropriate places to supply each rack with water. This is no small task for trying to retrofit an existing data center.
Not to mention that you have to make sure you have enough pressure to ensure each server is supplied water from the 'source', you cannot just daisy chain computers because the water would get hotter and hotter the further down the chain you go. This means a dual piping system (one for 'cool or room temperature' water and one for 'hot' water). And it means adjusting the pressure to each rack depending on how many computers are in it and such.
The issues of water cooling a data center go WAY beyond the case, which is why nobody has really done it yet - sure, the cost savings are potentially huge, but it's a LOT more complicated that sticking a bunch of servers with fans in racks that can move around and such, and then turning on the A/C. And there is a lot less room for error (as someone else mentioned, what if a leak occurs? or a plumbing joint fails, or whatever. Hell, if a pump fails you could be out a whole rack!).
Re: (Score:3, Interesting)
Nospill valves and other modern water cooling tech (Score:3, Informative)
No spill (as in "almost insignificant", not as in "not too much, won't empty the whole system, but you better have some towel nearby just in case"), quick disconnect, low resistance valves for watercooled system have been already available for quite some time for enthusiasts.
(Koolance is an example of compagny producing such thing in the US, Aquatuning is an example of shop selling similar implements in the EU - no links to avoid gratuitous advertising to web spiders, but you can easily google the names).
An
Water cooling on that size is no small feat... (Score:4, Interesting)
The ES/9000 that I had contact with was a series of cabinets that were all water-cooled from the outside in...it was a maze of copper pipes all around the edges and back and looked like a fridge. When you opened a cabinet, you could feel a blast of cold air hit you.
It was no trivial feat to do this, they had to install a separate water tank, some generators (I remember one of the operations guys pointing to a Detroit Diesel generator outside in the alley and saying it was just for the computer's water system), moved a bathroom (only water they wanted around the computer was the special chilled stuff), and I can distinctly remember seeing the manuals(!)... 3-inch thick binders with the IBM logo on them, and all they were for was the planning and maintenance of the water system.
No wonder it took almost a year to install the machine.
Water is a hassle (Score:5, Informative)
I work with particle accelerators that draw enough power that we don't have much choice but to use water cooling, and even though we have major radiation sources, high voltage running across the entire place, liquid helium cooled magnets, high power klystrons that feed microwaves to the accelerator cavities etc... the only thing that typically requires me to place an emergency call during a night shift is still water leaks.
Water is just that much of a hassle around electronics. Even an absolutely minor leak can raise the humidity in a place you really don't want humidity, it evaporates and then condenses on the colder parts of the system where even a single drop can cause a short circuit and fry some piece of equipment. After it absorbs dirt and dust from the surroundings it starts attacking most materials corrosively, which may not be noticed at first but gives sudden unexpected problems after a few years. If you don't keep the cooling system itself in perfect condition valves and taps will start corroding and you get blockages. Maintenance is a pain because you have to power everything down if you want to move just 1 pipe etc...
I just don't see why you would go through the hassle with water cooling unless you actually have to, and quite frankly if your servers draw enough power to force you to use water for cooling then you're doing something weird.
Re: (Score:2)
Re: (Score:3, Insightful)
Water (and liquid coolants, even metals) can be a hassle if not deigned correctly. I have had my experiences with water cooled systems but mainly the "over efficiency", well, one burst which shouldn't have happened (LOL).
One thing I have learned (from my son) - in cars, everything replaced with military and/or airplane grade fittings, valves, tubes, etc - makes life much easier. Not much more expensive but very fast pays back. If I would have known that (much) earlier instead of accepting engineering (good
Prior Concepts (Score:2)
Cray-2 (Score:4, Insightful)
Hmm... The Cray-2 [wikipedia.org] was cooled via complete immersion in Fluorinert [wikipedia.org] way back in circa 1988. I was an admin on one (Ya, I'm old). So, this is a bit different, but certainly not ground-breaking.
Re: (Score:2)
Oh it's groundbreaking. It's unique. I know, because they have patents on it. 1988 doesn't exist. It's all in your head. They invented something new and innovative so they patented it! Duh.
*cough*
Too early yet for that much sarcasm?
Re: (Score:3, Interesting)
The crays full immersion coolant model hit a big problem - the coanda effect.
This is where layer of fluid near the actual component flows much slower than actual flow - in layers slowing down exponentially as it gets closer to the stationary components.
For air this is not too much of a problem - only a very fine layer of stationary air over compenents that does not affect cooling. But with liquids the effect is both noticable and severely impacts coolant flow over hot surfaces - with some then "next gen" cr
Re: (Score:3, Informative)
The crays full immersion coolant model hit a big problem - the coanda effect.
You did not describe the Coanda effect, you described boundary layer issues. I don't know enough about the story to know whether boundary layers were the real issue: It's pretty routine to take into account the fact that friction causes the fluid velocity to approach zero at the surface of a stationary object, and to account for a lack of turbulence in the laminar part of the boundary layer that reduces heat transfer. It could just be that they needed to get a phase change at the surface in order to pull
Re: (Score:2)
Everything *trickles* from Supercomputers/Mainframes eventually.
Doesn't look practical (Score:2)
Look at the cross section photo. This dispenses completely with convection (air flow) and instead designs the system for direct physical contact from the heat sink to the components. Then the water flows behind the heat sink to take the heat away from that.
The problem is that means that you have to make a heat sink with varying height "fingers" on it to meet every component that produces heat (which is all of them), which means every time you change a component you have to redo the heat sink. And of course
Re: (Score:2)
That was my thought too.
I can see heat sinks with liquid pipes in them in the future. Plus regular air cooling. ie a hybrid solution.
weight? (Score:5, Interesting)
How much does a rack full of water-cooled blades weigh?
Never thought I'd see the UPS become the lightest thing in the server room.
Hmmm, so what happens when internals break? (Score:2)
With all those layers it doesnt seem that sliding one of these out and quickly swapping some RAM or any other part is
going to happen.
As well do these Iceotope guys actually make server hardware or just the cooling specs. Who do they get there guts from or are they just advertising and hoping the guys like HP, IBM or SUN (well maybe not SUN) decide to design there next generation of servers with this in mind?
I'd like to see how easy it is for replacement. doesnt look like there is a lot of room for other bit
night time freezing of liquid would save more (Score:3, Interesting)
Direct cooling makes far more sense than cooling rooms like I keep seeing around now.
LoB
I thought we'd finally learned... (Score:2)
Among the many great things the internet has brought us (*cough*porn*cough*), "location-independence" ranks pretty high up there. Your servers don't need to all fit in one cargo container that runs so hot it requires LN cooling. For all it matters, you could put them in a single line of half-racks on a mountain ridge, cooled naturally by the wind (with some care to ke
OK so then how do you explain this? (Score:2)
"Intel set up a proof-of-concept using 900 production servers in a 1,000 square foot trailer in New Mexico, which it divided into two equal sections using low-cost direct-expansion (DX) air conditioning equipment. Recirculated air was used to cool servers in one half of the facility, while the other used air-side economization, expelling all hot waste air outside the data center, and drawing in exterior air to cool the servers. It ran the experiment over a 10-month period, from Oc
Server standardization... (Score:2)
The problem with this is that it requires server manufacturers to standardize their designs. There was talk a few years ago about standardizing Bladeservers. I don't see this happening as there's too much control in the bladecenter chassis, switch interfaces, management abilities etc. Plus why would IBM want to sell an empty chassis and then let the customer fill it with HP C-Class blades?
Even racks themselves from IBM/HP/Dell/EMC/netapp/Sun aren't standardized, other than they are 19" wide. This is why
Re:Excess Heat (Score:4, Insightful)
Parent
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Very little, since you're dealing with very low quality heat. The hottest temp in your system is going to be the hardware itself (unless you're expending energy to pump it - then what's the point of trying to generate power from it?)
So if your max hardware temp is, say, 38C (100F) that's not good enough to generate any appreciable power from.
On the other hand, you probably will be pumping the heat to chill the system, and the rejected heat temp may be quite a bit higher - maybe as high as 75C. You can use t
Re: (Score:3, Funny)
"Very little, since you're dealing with very low quality heat."
And that's why in the really good systems, the only acceptable option is Monster Heat, the finest quality heat available, and its even gold plated.
Re: (Score:2)
we all know what happens when you mix water and server rooms http://www.youtube.com/watch?v=1M_QTBENR1Q [youtube.com] better call up Noah
"The Iceotope approach takes liquid – in the form of an inert synthetic coolant, rather than water – directly down to the component level," the company said.
Re: (Score:2)
Oh, that's listed under hobbies, specifically "cave diving".