Met Office and Microsoft To Build Climate Supercomputer (bbc.com) 27
The Met Office is working with Microsoft to build a weather forecasting supercomputer in the UK. From a report: They say it will provide more accurate weather forecasting and a better understanding of climate change. The UK government said in February 2020 it would invest $1.6bn in the project. It is expected to be one of the top 25 supercomputers in the world when it is up and running in the summer of 2022. Microsoft plans to update it over the next decade as computing improves. "This partnership is an impressive public investment in the basic and applied sciences of weather and climate," said Morgan O'Neill, assistant professor at Stanford University, who is independent of the project. "Such a major investment in a state-of-the-art weather and climate prediction system by the UK is great news globally, and I look forward to the scientific advances that will follow." The Met Office said the technology would increase their understanding of the weather -- and will allow people to better plan activities, prepare for inclement weather and get a better understanding of climate change.
Define supercomputer (Score:2)
"It will use Microsoft Azure's cloud computing services"
By that definition every cloud server farm is a supercomputer then. Not really in the spirit of the terminology though even though they say they'll integrate some Crays too.
Also on a side note, Azure and other cloud setups don't exactly have a stellar reputation for reliability and weather forecasts can be life and death for aviators and sailors. Should they really be relying on Azure (or AWS or any of the others) for this?
Re: (Score:3)
Cloud is good for parallel but low bandwidth processing. A super computer is often good for fast serial processing and parallel processing with a high bandwidth of data.
So for example if I am taking data from hundred thousand weather stations on wind speed and direction. I may have the cloud calculate the vectors for every 10 square miles by passing each cloud node a few weather station data. but because I am sending it to thousands of computers in the could, they all just kinda process that small amount of
Re: Define supercomputer (Score:3)
Depends on what "cloud" means and what bandwidth counts as high.
A small number of operations common in scientific computing, like for instance finding the eigenvalues of a million x million quadruple precision complex matrix, pretty much require sharing memory directly to utilize the full benefit of multiple processors. But if the matrix is sparse or is a blockwise matrix, memory sharing isn't necessary and the operation can proceed just as quickly with distributed machines with skinny links to eachother.
Re:Define supercomputer (Score:5, Informative)
Note that a modern Cray looks pretty much identical to a cloud server farm with a good network.
Supercomputers used to be fundamentally 'exotic', then became pretty mundane servers that sometimes (not always) have slightly exotic networking (e.g. infiniband). However more and more the features exclusive to that exotic networking are constructed by cloud providers out of ethernet (using openflow to make an 'ethernet subnet manager' or, more commonly, constructing similar features using layer 3 features), and the supercomputer networking is more and more ethernet like (e.g. same cables and layer-1 signalling for infiniband and ethernet). In fact, every Cray sold now is now ethernet based, after divesting themselves of their proprietary interconnect to Intel, who let it be spun off into yet another company after they went to give up on it.
The gold standard for weather is extremely and expensively redundant systems to make sure your real-time obligations are met. Climate actually is less critical, since you don't have real-time requirements for climate work. In fact the normally surplus capacity of a redundant weather system is commonly used for climate, with pre-emption to kill off climate work if a loss of redundancy demands it.
Re: (Score:2)
Cray is "special" Ethernet with 32 byte packets and special switches, but yes it's basically Ethernet. However doing the math 100Gbps RDMA Ethernet is preposterously cheap and good enough for the majority of work loads. You are better off spending the money on more nodes when you look at flops per dollar over the system life. Cloud data centers are really pushing the envelopes of ethernet, both in terms of speed and price that makes fancy interconnect less and less worth while.
The other point I would make i
Re: (Score:2)
We just did some analysis of that and 75% of the core hours on our system are single node jobs (it's 90% of all jobs), with another 14% being two node jobs
Yeah, there is plenty of use for compute capacity, but people have generally figured out that bothering to scale running a solve of one problem to multi-system is usually less productive than just running multiple instances of single-system jobs. Sure each individual job takes longer than you could have taken, but almost always you have loosely coupled problems that you are just as happy to solve independently as you would have solving a job quicker.
There are some segments that need larger jobs more than ot
Re: (Score:2)
Note that a modern Cray looks pretty much identical to a cloud server farm with a good network.
s/good network/exotic network/
Supercomputers used to be fundamentally 'exotic', then became pretty mundane servers that sometimes (not always) have slightly exotic networking (e.g. infiniband).
They're pretty exotic. The #1 supercomputer (Fujitsu) has a custom interconnect on the same die as the CPU in order to minimize latency.
Sure many use infiniband. But by the time you're in the market for the type of infiniba
Microsoft? (Score:1)
Re: (Score:2)
You find no paradox that the weather is as unstable as a Microsoft instance, yet Microsoft will try to predict the weather? Will the predictions be good for a week until the next Patch Tuesday discovers zero-day hailstorms and hurricanes?
Re: Microsoft? (Score:2)
It's not "bad" but it is outside their comfort zone. IBM has been building scientific computers since before the electronics age. Microsoft has been making consumer and light business software for desktop PCs and web servers for almost the entirety of its 40 year existence.
Re: (Score:2)
FTFY
Microsoft will find the cure for climate change! (Score:4, Funny)
If Microsoft was responsible for the coronavirus vaccine, you'd have to get a booster shot every month or your immunity would disappear.
Oh what fun (Score:3)
2. Everyone goes for weather status.
3. Service throws BSOD. It some how won't be a normal stop error screen. Like the error message will literally be able to kill certain things. Like with most Linux command if spoken out loud.
4. PANIC!
5. Galveston 2.0
Re: Oh what fun (Score:2)
Neal Stephenson sequels... (Score:2)
Forecast (Score:5, Funny)
Tomorrow calls for a blue screen of death followed by multiple reboots. And now to John for sports.
Re: (Score:2)
First forecast (Score:5, Funny)
Heavy licensing costs across the planet with a chance of BSOD
HPC failure brought to you by M$ (Score:1)
Will probably still run Windows 10 (Score:4, Funny)
Re: (Score:2)
What platform will the application run on? (Score:2)
" It will use Microsoft Azure's cloud computing services and integrate Hewlett Packard Enterprise (HPE) Cray supercomputers."
So, presumably, this will ultimately be a giant cloud host for Linux instances, on which the actual application s/w runs on.
OTOH, this is the UK, which has been Microsoft's bitch for many years, so perhaps not.
What exactly Microsoft will provide? (Score:1)
I hear all supercomputers run Linux these days.
What is Microsoft going to deliver then? Their expertise in subscription services?