Google, Oracle Cloud Servers Wilt in UK Heatwave, Take Down Websites (theregister.com) 61
Cloud services and servers hosted by Google and Oracle in the UK have dropped offline due to cooling issues as the nation experiences a record-breaking heatwave. From a report: When the mercury hit 40.3C (104.5F) in eastern England, the highest ever registered by a country not used to these conditions, datacenters couldn't take the heat. Selected machines were powered off to avoid long-term damage, causing some resources, services, and virtual machines to became unavailable, taking down unlucky websites and the like.
Multiple Oracle Cloud Infrastructure resources are offline, including networking, storage, and compute provided by its servers in the south of UK. Cooling systems were blamed, and techies switched off equipment in a bid to prevent hardware burning out, according to a status update from Team Oracle. "As a result of unseasonal temperatures in the region, a subset of cooling infrastructure within the UK South (London) Data Centre has experienced an issue," Oracle said on Tuesday at 1638 UTC. "As a result some customers may be unable to access or use Oracle Cloud Infrastructure resources hosted in the region.
Multiple Oracle Cloud Infrastructure resources are offline, including networking, storage, and compute provided by its servers in the south of UK. Cooling systems were blamed, and techies switched off equipment in a bid to prevent hardware burning out, according to a status update from Team Oracle. "As a result of unseasonal temperatures in the region, a subset of cooling infrastructure within the UK South (London) Data Centre has experienced an issue," Oracle said on Tuesday at 1638 UTC. "As a result some customers may be unable to access or use Oracle Cloud Infrastructure resources hosted in the region.
OOF. (Score:2)
Might as well go with someone more reliable and competent. Like HostGator or GoDaddy.
Re: (Score:1, Informative)
Using the cloud is already a recipe for disaster.
FTFY-
Re: (Score:2)
Using the cloud is already a recipe for disaster.
FTFY-
Not necessarily. A private cloud in your own DC may be reliable, secure and cost-effective. If you do not do things on the cheap.
Re: (Score:1)
Using the cloud is already a recipe for disaster.
FTFY-
Not necessarily. A private cloud in your own DC may be reliable, secure and cost-effective. If you do not do things on the cheap.
In my experience, people that claim that "the cloud" is holistically bad are coming from one of two camps:
- employee of some massive tech company that already runs their own large datacenters, so of course off-loading into someone else's DC is just "bad"
- employee of some small company that's trying to save a nickel by running their own physical servers within a leased cabinet, and those servers are most likely long past their expiration date
Re: OOF. (Score:2)
If you think cloud datacenters donâ(TM)t run stuff that is long overdue for replacement, you havenâ(TM)t run on bare metal cloud. As long as it doesnâ(TM)t cost more to run than it brings in, Amazon and co will definitely continue to run on ancient Xeon. Most systems donâ(TM)t need 100% CPU anyway, RAM and disk are consumed a lot more quickly.
Re: (Score:1)
For most of what we run we're using the AMD Epyc 7763 hosts, which for what we're paying is a lot better than trying to build it ourselves then upgrade it every
Re: (Score:1)
Rugged servers (Score:2)
Re: (Score:2)
and how well does that work in an BIG DC with racks and racks of servers?
Re: Rugged servers (Score:1)
Re: (Score:3)
Good luck with that. You still need to get a certain amount of heat out and if you are not prepared to do that at 40C outside air temperature, no amount of "rugged" will help.
Re: Rugged servers (Score:1)
Re: (Score:2)
You are looking at the wrong component. The closer the outside temperature gets to the cooling target, the more air you have to pump through (if outside temperature is _below_ the cooling target). There are limits to what works and even a "Ultra High Temperature DC at 50C" (https://www.intel.com/content/dam/doc/technology-brief/efficient-datacenter-high-ambient-temperature-operation-brief.pdf) may not get sufficient cooling anymore at 40C outside temperature without using heat-pumping.
Re: Rugged servers (Score:1)
Re: (Score:2)
I have no issue with that and I know such hardware exists. It is not DC standard though, because of cost, access (50C is already a problem with people going in there, you may be required to have a rescue-team at standby outside in some countries and you need protective equipment), performance and other factors.
That said, at 55C (or 70C) max operating temperature you obviously can do air-cooling with outside air at 40C without much of a problem.
Re: (Score:2)
The smaller the differential is between inside and outside, the easier it is to move the heat. Also by the time the DC hits 40C, the racks of non-heat tolerant servers will shut down.
No idea about the cost of those servers, so no way to determine if it's worth it or not.
Re: (Score:2)
That only applies if it is not a "hot" DC: https://www.intel.com/content/... [intel.com]
A hot DC may not even have heat pumping but just uses outside air directly.
For a "cold" DC (these days often 25C or even a bit warmer) this does apply only if the outside temperature is above the inside temperature and heat-pumping has to be used.
Re: (Score:2)
None of that makes any sense in this context. TFA is about high outside temperatures overwhelming the DC cooling systems. So the DCs HAVE cooling systems and the outside temperature is clearly above 25C.
Re: (Score:2)
It makes a lot of sense. There are 2 options for "cooling systems"
1. Just blow in air from outside. That will be a "hot" DC, with server temperatures up to 50C.
2. As 1, but blow that air into a heat-pump and use the other side of that to cool. This can be a "cold" DC or an additional option in a "hot" DC to be used when outside air gets to hot.
Both options can get overwhelmed in serval ways.
Re: (Score:2)
If you're in a "Hot" DC, you already had to buy a rugged server just for it to survive.
Re: (Score:2)
If you're in a "Hot" DC, you already had to buy a rugged server just for it to survive.
Yes. I do not quite understand what you think does not make sense here.
Re: (Score:2)
Your apparent argument against the utility of a rugged server.
Re: (Score:2)
That seems to be a misunderstanding. A rugged server is not immune to this problem. Depends all on the circumstances.
Re: (Score:2)
Oh, I'm sure NASA has some radiation-hardened 80486s sitting around that could probably survive. Mind you, they probably run at 16mhz, so it may not quite have the horsepower a data center is accustomed to.
Re: Rugged servers (Score:3)
Re: Rugged servers (Score:1)
Re: (Score:2)
Re: (Score:2)
I'm not sure why you assume the datacenter temperature would have stayed under 55C. If part of the cooling system has failed and there already was barely enough for those temperatures, the temperature will start climbing fast. Don't forget there are 10s or 100s of MW worth of power generating heat in there. Even shutting systems down, they were probably still above normal operating temperatures and just trying to prevent catastrophic failures.
We need to lower CO2 emissions, but... (Score:2, Interesting)
We need to lower CO2 emissions or face more and more extreme weather events but we can't do that with the energy source that produces the least CO2 per unit of energy, that being nuclear fission.
https://en.wikipedia.org/wiki/... [wikipedia.org]
At least that's the message given by the global warming alarmist. So, global warming is bad but I guess nuclear power is worse. So then just how bad can nuclear power be? How bad can global warming be if we have to do without nuclear power to avoid it? Nuclear power is the safest
Re:We need to lower CO2 emissions, but... (Score:4, Informative)
Re:We need to lower CO2 emissions, but... (Score:4, Interesting)
It's funny that your own sources disagree with your statement...
Nuclear fanatic. These days these people are on the level of flat-earthers with regards to their connection to reality.
Re: (Score:2)
Think about how much effort MacMan and his alt accounts go to just to promote nuclear. It's not just the posts, he farms mod points to mod himself up.
He's got to be getting paid. At least I hope so.
Re: (Score:2)
Think about how much effort MacMan and his alt accounts go to just to promote nuclear. It's not just the posts, he farms mod points to mod himself up.
He's got to be getting paid. At least I hope so.
Yes, looks like it. Although I doubt Slashdot is a relevant propaganda target these days. He is probably just an useful idiot hard at work doing virtue signaling.
Re: (Score:2)
Maybe. He forgot to mention Dr. McKay, so he probably lost his bonus on this one if he was getting paid for it.
Re: (Score:1)
Sure, you have Chernobyl and Fukushima, but those are exceptions.
Sure, they are expensive, but these are national interest things so state support is always required.
Sure, they produce waste. Tell me of something that doesn't produce waste. Countries like Sweden and Finland are working in safe ways to deal with them.
And here we are. Countries like Germany shutting down almost all of its nuclear reactors without not
Re: (Score:2)
We have solar/wind/hydro + battery tech in addition to Nuclear, nut at the end of the day, what will "win" the battle isn't going to be a single "silver bullet" but rather a "new gun" and "bullets".. a different way of thinking about the problem, a reduction/change in consumption, and a change in generation. addressing only ONE of these problems simply pushes the problem further out rather than fixing the problem.
We need to change and/or reduce our consumption
Re: (Score:2)
> We need to make a choice, nuclear fission, global warming, or energy poverty
Don't worry, we'll choose all three.
As a European, I can only observe that we're screwed first.
Nukes take a decade to build. Once gas is gone, that's it. lights out.
Even Poland, our coal powerhouse, is importing coal.
Germans have mainly lignite left. Burning that is insanity. They're also very much lying on their current electricity carbon intensity. (when the sun comes out, they disconnect coal plants, but don't spin them dow
Re: (Score:2)
It's almost like we knew a half a century ago that we were headed towards catastrophe, but instead of starting the transition in the 1970s, we just thought "Oh well, I'll be dead or too old to care, so fuck it, let's burn more oil and gas!"
The chickens are coming home to roost, or more appropriately roast. At least you'll be able to fry up your meat on the sidewalk, so there will be some energy savings there.
Re: (Score:1)
Re: (Score:2)
Choosing to cut off supplies of natural gas from Russia and closing all the nuclear power plants has resulted in Europe mining for coal like never before to keep the lights on.
Replace "never before" with "slightly more than last year, and still massively down from its all time high" and your post would actually make some sense.
We should absolutely build nuclear power. But stop pretending it's the solution to global warming. You can't build it in time, even if you start today. Let the fantasy go and join us with actual workable solutions.
You're like a person standing on a train track watching the oncoming train, and hoping there's a project to divert the train tracks around you th
That's the cloud for you (Score:3, Insightful)
Your ability to work at the mercy of a heatwave, and which of their servers Oracle decides to take down.
Re: (Score:2)
Re: (Score:2)
Well, IT is not cheap. You need to always keep that in mind. Interestingly, European regulators now require a prepared re-insourcing strategy to be part of any IT outsourcing (including cloud) from banks and insurances. Apparently the wonders of the cloud are quite widely known these days.
Re: (Score:2)
Re: (Score:2)
Actually the second. Hosting outside of Europe is basically illegal these days for these companies.
Re: (Score:2)
Well, if you have a handful of servers, even in a cold climate, you already probably need cooling for them since they can get very hot. Even for normal PCs (not laptops) in an office, if the A/C goes out in the office the temperature can quickly rise, something I have seen happen a few times.
Re: (Score:2)
Implying that your own self managed server is capable of surviving?
Note they closed our office yesterday and sent everyone home because our AC system failed under the load.
Those "other people" running "other people's computer" are typically much better at it than most in-sourced IT. The only reason you think otherwise is because every small company's IT problems doesn't make the news.
Re: (Score:2)
Just remember that even if their website says "99.999% uptime", unless your contract actually says "99.999% uptime or we pay out $$$/minute", it's just marketing wank.
Re: (Score:2)
You can just move your applications to a different server on a different continent. Can't do that with a rack in your office though.
Clouch (Score:2)
Can a cloud go up in smoke, or is that redundant?
Oracle Licensing fees always comes first... (Score:2)
Of course the root cause is probably a telecom closet whose cooling system shutdown due to a fault sensor, but let's not waste this opportunity to egg on Oracle for another outage =o)
Re: (Score:2)
Oracle: A database company named after an ancient woman who got buzzed out of her gourd on methane and then spouted whatever came into her head...
Re: (Score:2)
Oracle OCI is also one of the most expensive on-demand cloud services there is. They don't have any serverless technologies that don't eventually require you to create some VMs, and their VMs are more expensive than anyone elses. I also note VMs were some of the last services to be restored as they recovered services early this morning. They had something like 16 hours of disruption - all for want of some better chillers.
I'd never advise my clients to use OCI *unless* I ever work for Zoom, TikTik, etc - the
Re: (Score:2)
Let's be honest for a moment -- Oracle Cloud probably allocates most revenue towards Oracle Enterprises' exorbitant and recurring licensing fees
Maybe so. But you get SO MUCH VALUE FROM IT - Oracle's UNSTOPPABLE CLOD ..errr.. CLOUD.
Brexit (Score:2)
They exited Europe, and the only landmass willing to take them was the middle east. Enjoy roasting!
Penny wise, pound foolish (Score:1)
Seeing these outfits can do whatever they want with the hardware design, the obvious answer is to do negative pressure liqu