Over 45,000 VMware ESXi Servers Just Reached End-of-Life (bleepingcomputer.com) 57
An anonymous reader quotes a report from BleepingComputer: Over 45,000 VMware ESXi servers inventoried by Lansweeper just reached end-of-life (EOL), with VMware no longer providing software and security updates unless companies purchase an extended support contract. Lansweeper develops asset management and discovery software that allows customers to track what hardware and software they are running on their network. As of October 15, 2022, VMware ESXi 6.5 and VMware ESXi 6.7 reached end-of-life and will only receive technical support but no security updates, putting the software at risk of vulnerabilities.
The company analyzed data from 6,000 customers and found 79,000 installed VMware ESXi servers. Of those servers, 36.5% (28,835) run version 6.7.0, released in April 2018, and 21.3% (16,830) are on version 6.5.0, released in November 2016. In total, there are 45,654 VMware ESXi servers reaching End of Life as of today. The findings of Lansweeper are alarming because apart from the 57% that enter a period of elevated risk, there are also another 15.8% installations that run even older versions, ranging from 3.5.0 to 5.5.0, which reached EOL quite some time ago.
In summary, right now, only about one out of four ESXi servers (26.4%) inventoried by Lansweeper are still supported and will continue to receive regular security updates until April 02, 2025. However, in reality, the number of VMware servers reaching EOL today, is likely far greater, as this report is based only on Lansweeper's customers. The technical guidance for ESXi 6.5 and 6.7 will carry on until November 15, 2023, but this concerns implementation issues, not including security risk mitigation. The only way to ensure you can continue to use older versions securely is to apply for the two-year extended support, which needs to be purchased separately. However, this does not include updates for third-party software packages. For more details about EOL dates on all VMware software products, check out this webpage.
The company analyzed data from 6,000 customers and found 79,000 installed VMware ESXi servers. Of those servers, 36.5% (28,835) run version 6.7.0, released in April 2018, and 21.3% (16,830) are on version 6.5.0, released in November 2016. In total, there are 45,654 VMware ESXi servers reaching End of Life as of today. The findings of Lansweeper are alarming because apart from the 57% that enter a period of elevated risk, there are also another 15.8% installations that run even older versions, ranging from 3.5.0 to 5.5.0, which reached EOL quite some time ago.
In summary, right now, only about one out of four ESXi servers (26.4%) inventoried by Lansweeper are still supported and will continue to receive regular security updates until April 02, 2025. However, in reality, the number of VMware servers reaching EOL today, is likely far greater, as this report is based only on Lansweeper's customers. The technical guidance for ESXi 6.5 and 6.7 will carry on until November 15, 2023, but this concerns implementation issues, not including security risk mitigation. The only way to ensure you can continue to use older versions securely is to apply for the two-year extended support, which needs to be purchased separately. However, this does not include updates for third-party software packages. For more details about EOL dates on all VMware software products, check out this webpage.
Good time to switch (Score:5, Informative)
Plenty of other players in the market that offer more, better and cheaper features, open source based like KVM and no longer beholden to Broadcom that doesn't care about its customers.
Re: (Score:2)
Plenty of other players in the market that offer more, better and cheaper features, open source based like KVM and no longer beholden to Broadcom that doesn't care about its customers.
Management knows that just not getting updates is even cheaper* than having to switch! Management knows best!
* Until the company is hacked, the entire floor falls out, they spend a ton of money on new VMWare software, suddenly we need to have meetings about how to deal with EOL software, and all responsibility will fall upon development for not bringing this to managements attention even though that's exactly what they did.
Re: (Score:2)
Re:Good time to switch (Score:4, Informative)
As a Vmware system engineer currently forced to work on Microsoft virtualization I can wholeheartedly say that no, right now there isn't a viable enterprise level alternative.
Re: (Score:2)
As a Vmware system engineer currently forced to work on Microsoft virtualization I can wholeheartedly say that no, right now there isn't a viable enterprise level alternative.
A thousand times this.
And it's not just VMWare's excellent and unrivalled management platform but ESXi's own robustness.
The robustness of ESXi is it's own worst enemy in this regard. Because a 6.5 or even 5.5 server will keep trucking along a lot of clients become complacent about upgrading.
Re: (Score:1)
You're comparing VMWare with an equally poor system that isn't even seriously competing in the market. HyperV is a technology to run Windows XP or 7 programs on Windows 10/11 desktops, not a serious server-grade hypervisor system like KVM.
They are trying hard with Azure HCI, but they don't even support nVIDIA vGPU and way too much shit is still locked away behind PowerShell. Red Hat, OpenStack, Proxmox, Nutanix are stable and offer the same if not more features than VMWare.
Virtualization does not fix shoddy practices (Score:4, Interesting)
Even letting such an installation reach EoL is more than shoddy. It means nobody ever put any working provisions in place for the update requirement that was absolutely sure to come. Well, expect these to get hacked soon. I have absolutely no compassion for the morons that let this happen and think they should instead be held liable for any and all damage attackers will do using these systems.
Re:Virtualization does not fix shoddy practices (Score:5, Insightful)
What's shoddy is dropping support for fully functional hardware.
I've got ESXi servers that are going to stay on 6.7, because 7 won't fucking install on the hardware.
I don't need new servers, because what I have are plenty fast for what they're doing. This is artificial obsolescence, and I'm not playing the fucking game.
Re: (Score:2)
You chose to play the game when you chose to use vmware. Or maybe you've been saddled with someone else's decisions.
vmware in general doesn't give a shit about who wants what supported. you can't even run workstation properly on a modern kernel because they don't bother to keep up, and the OSS replacements are flaky.
Re: (Score:3, Informative)
You chose to play the game when you chose to use vmware.
They simply don't have to upgrade and they know it. Upgrading might turn out to be beneficial for the organization and might not. If they are behind a firewall they can and probably will skate by for a long time or damn near forever (I have many clients running ESX 4.x and 5.x) before upgrading if they keep things pretty clinical. Nothing says you have to face the Internet with it or leave it unfirewalled for anyone to access your vCenter. Every day they run on old hardware with old VMware is a day they did
Re: (Score:1)
I have no problem with your decision, as long as as you accept full liability for all damage to others it causes. The current state-of-the-art requires you to not use software that is unsupported by security patches. You violate that, liability goes to you.
Re: (Score:2)
I have no problem with your decision, as long as as you accept full liability for all damage to others it causes.
Ultimately, as a systems manager or operator you are probably on the hook already in a very real sense that you could lose your job or be reprimanded for being irresponsible should any of your managed systems get compromised. Management will be looking for someone to blame no matter how much you patched or how current your support contracts were. You can claim "best practices" and still be fired. If your money is best spent elsewhere, then it just is. If ring-fencing a big VMware environment is the only opt
Re: (Score:2)
Well, even people like you will eventually get dragged kicking and screaming into taking responsibility for their work, like every engineer must do. The price for society for continuing in this half-assed, egotistical and irresponsible way is just too high.
Re: (Score:2)
If it's not my company, it's not ultimately my problem. As you said, he might get fired. So what. This is especially true if he made a recommendation and it was ignored. Worker bee has done all they could but management knows best.
Taking responsibility when you get to make the calls is one thing and expected. Taking responsibility when management decided to ignore good advice doesn't make any sense. You'll be fired either way so why protect a manager that is not doing the same for you.
Also, the company is l
Re: (Score:1)
Well, even people like you will eventually get dragged kicking and screaming into taking responsibility for their work, like every engineer must do.
People like me keep the world operational. I'm sorry to break it to you, but plenty of engineers simply walk away from work they never had to be responsible for. Then guys like me come in and fix it properly. You seem to be upset because the world doesn't conform to your standards. As a Stoic, I'd advise you that's a losing game, but I doubt you'd listen.
The price for society for continuing in this half-assed, egotistical and irresponsible way is just too high.
The price for continuing to use software that's no longer supported is pretty low considering you get to keep the money. That's the "price". That's the poi
Re: (Score:2)
I have no problem with your decision, as long as as you accept full liability for all damage to others it causes. The current state-of-the-art requires you to not use software that is unsupported by security patches. You violate that, liability goes to you.
I have a problem with this formulation. It ignores the vendors failure to produce a product free of safety defects.
Re: (Score:2)
Speak the true words, brother.
The "hardware updates" I'm supposed to be afraid of not getting, are those like when there was that log4j problem a while back and VMWare was just like Â\_(ãf)_/Â for four or five months?
All my VMWare installs are on internal networks not reachable by the corporate LAN. If you're on my management LAN or you're deep enough into a virtual guest where you can be taking meaningful cracks at my vCenter or VMWare gear, I have _much_ bigger problems then whether those
Re: (Score:2, Interesting)
That is not "shoddy". That is scummy. If you are not prepared to follow whatever strategy the vendor has for the product _or_ are not prepared to move to a different vendor or platform, that is shoddy on your side. Sure, VMWare is screwing their customers over, no argument. Sure, that is wasteful and malicious and greedy. But it is on you to be prepared for a vendor doing that and obviously you are not prepared. And that is just unacceptable.
Re: (Score:2, Informative)
That is not "shoddy". That is scummy. If you are not prepared to follow whatever strategy the vendor has for the product _or_ are not prepared to move to a different vendor or platform, that is shoddy on your side. Sure, VMWare is screwing their customers over, no argument. Sure, that is wasteful and malicious and greedy. But it is on you to be prepared for a vendor doing [unreasonable and unexpected things like declaring still-under-warranty hardware suddenly obsolete] and obviously you are not prepared. And that is just unacceptable.
The only winning move is not to play. But just because one has started playing, doesn't mean you get to pretend you had prescient knowledge that VMWare would screw customers over.
Re: (Score:2)
But just because one has started playing, doesn't mean you get to pretend you had prescient knowledge that VMWare would screw customers over.
Unlike you, I have a working mind. No "prescient knowledge" involved.
Re: (Score:2)
No sane system administrator would leave their VMWare Hypervisors Internet facing. These things should be behind multiple firewalls and require VPN or local network/console access to administer.
Re: (Score:2)
No sane system administrator would leave their VMWare Hypervisors Internet facing. These things should be behind multiple firewalls and require VPN or local network/console access to administer.
Yes, pretty much. But many system administrators are not sane or do not care. And it is soooo convenient being able to access the hypervisor over the Internet!
The other thing is that you have an Internet-facing VM in there and a breakout-vulnerability, just protecting the Hypervisor interface may not do much.
Re: (Score:2)
You would think that the dumb sysadmins who left their hypervisors Internet-facing would have learned their lesson after getting pwned by the various major CVE's for VSphere in the past.
I guess that's the thing about dumb sysadmins... we get a new batch of them every year who need to learn things the hard way.
Re: (Score:2)
That's right. We should all rebuild or replace our perfectly-fine-working servers because VMware snaps their fingers, otherwise we should be held liable "for any and all damage" if we get hacked. Yep, that's putting the blame in the right place.
Re: (Score:2)
Thanks, I'm with you, and in a similar tight spot. Too much to detail, but like AC posted, perfectly good very reliable hardware that's being orphaned due to big-money big company decisions. There's no way to know the whims and capricious "business" decisions that these big companies will make. So I'm trying to somehow tell the future (where is Nostradamus when you need him!) and not get vendor lock-in, forced major upgrades in hardware and/or software.
I used to think IBM was a pretty cool company, but w
Looks like FAILURE to me (Score:5, Interesting)
One of the key goals of virtualization was to add a new layer of abstraction. It should be easy to move a work load to a newer hypervisor, or a least a later hypervisor version in the same family.
Honestly if I was VMWare looking at these stats, I would be asking why people are staying on down level versions of ESXi. I assume the stats include the free tier because of the way they are likely gathered but even that should be considered.
Do people like some feature/function/behavior of the older releases - even something silly like UI?
Are there compatibility problems, new ESXis dropping legacy hardware support to quickly?
Are there compatibility problems, newer ESXis failing to import/run existing guests?
Is the upgrade path - simple one host in someones lab guests offline during the update all the way to enterprise cloud five nines situations to hard / risky to execute?
License changes bothering people?
I have been out of the ESXi game since about when 6.5 was released, but upgrading was never a problem. So I have to asume something has change and not for the better. With all the competitors now and competing technologies like containers and namespaced execution, not to mention the cloud... VMWare better address it and fast or they will lose the market quickly
Re: (Score:2)
My guess would be either management software incompatibilities or cost of upgrade in most cases.
Re: (Score:2)
Re:Looks like FAILURE to me (Score:5, Informative)
Going from ESXi 6.x -> 7.0 meant loss of Linux driver support. There was a heap of drivers from the Linux kernel available through the VMKLinux module that weren't VMware native, they all got killed off, so a lot of hardware became unsupported.
The original release of 6.0 was in March of 2015, and people have been able to re-use their existing licenses (And hardware) within the 6.x branch, so for their original purchase they've had 7 years of licensing. Now to go to v7, they've had to buy new licenses, and probably, new hardware. In the 7.x branch, they've had 2 years of really buggy releases.
The 7.x train was buggy as anything. Patch releases on patch releases. I'm sure that VMware just called this one 8.0 to get away with the horrible release cycle that was 7.x, however, that means that anyone who bought 7.x licenses has only got 2 years of releases. Even Microsoft gives you 5 years of mainstream support and security fixes for another 5, so a total of 10 years of support. People have to be looking at VMware and going "What the hell?"
Further to this, everyone waits for 7.1 or more before deployment, ESPECIALLY with all the buggy releases of 7.0. Now VMware has dropped it. Anyone who thought they were going to upgrade to 7.1 has never had the chance as they never released a 7.1 or above.
I think a lot of management will be leaning on their VMware / resellers regarding licensing costs come renewal time, the 7.x branch has left VMware with black eye. A lot of people won't leave the platform immediately as they have too much invested into it, support contracts, hardware support, staff training, backups and business processes. When it comes time for a hardware refresh though, a lot of people are going to be looking over the fence at Nutanix and others. With Dell and VMware now separated, if Dell starts pushing another vendor, it could be the beginning of the end for VMware. If Dell and HPE start taking business elsewhere, they're going to be completely screwed.
Re: (Score:3)
7 dropped a ton of older and not-even-that-old hardware that runs 6.7 perfectly fine. Probably the biggest cutoff ever. I forget the exact details of the cutoff (also out of the ESXi game aside from home stuff, which I don't care that much about), but even relatively recent servers by corporate standards won't run it. Since migrating to other VM solutions is tricky and non-trivial it means there's a ton of 6.7 still out there.
Re: (Score:3)
There is (at least) server gear from HPE that was first available _less than five years ago_ which doesn't run 7.0. And it's not like people buying it in 2017/2018 knew that the version of VMWare that came out in 2020 wouldn't be supporting it.
That's BS, plain and simple.
Re: (Score:2)
"Are there compatibility problems, new ESXis dropping legacy hardware support to quickly?"
There are probably other reasons in this list, but the #1 reason is going to be the above. I'm sure it's a cahoots issue with Intel, HP, Dell, etc, but basically the assumption is the minute your gear is off the books (in terms of fully depreciated) you forklift it all out and replace it with new shiny.
It's conceivable if they're using other older gear there's some "works with the fat or Flash client only" type proble
Re: (Score:2)
"cahoots" implies something nefarious, but it's not that bad...
Anything beyond 5 year old hardware (While I'm sure it's running perfectly fine) is in extended support by HPE / Dell etc. The RAID cards and whatever else that Dell / HPE are using are probably even older than that, and those manufacturers (Funnily enough, Broadcom is one of the larger vendors here having acquired MegaRAID and Adaptec) who isn't writing a native VMware driver because the RAID cards are End Of Life. A lot of these cards worked o
Re: (Score:2)
Re: (Score:2)
Isn't it ironic?
VMWare in the Cloud Costs (Score:2)
Re: (Score:1)
we use OVH sddc ( vmware as a service) and they have enacted a similar lifecycle policy, without telling their users. Well, they did tell us, by updating a wiki page....https://docs.ovh.com/ca/en/private-cloud/lifecycle-policy/
I wonder how cloud migration is factoring in here (Score:2)
A lot of CTO's out there seem to be wanting to get rid of their data centers and move their infrastructure to cloud hosting like AWS and Azure. Once that's done, they don't have to worry about upgrading hypervisors, because Amazon and Microsoft are doing it for you behind the scenes. They are also charging you dearly for that privilege, but that's another topic entirely.
I know that we didn't bother upgrading our VMWare hypervisors from EOL versions during the 18 months (or so) when we were migrating everyth
Re: (Score:2)
How many didn't have hardware support or even if they did, saw the amount of pulled releases and stability issues with v7 even on qualified hardware and went "Nah, there's no way I'm running my business on that"
Bad hardware support (Score:3, Informative)
eBay (Score:2)
Good. I've been meaning to upgrade my internal network and consolidate a few things.
Re: (Score:2)
Ebay's great (I like it anyway). Tiny IT situation I help with- guy just bought a great server on craigslist. Seems good to me- HP, dual Xeon 3 GHz CPUs, 8 cores each (plus "hyperthreading", so 32 virtual cores total), 192 GB RAM, 8 1 TB SSDs, quad-port Ethernet card, all for $500.
Someone had installed XCP-ng, some Windows Server 2019 VMs, and xen-orchestra. No passwords, but I got into dom0 and xen-orchestra, but it's the free version, and so much is disabled that it's pretty useless. So I'm continuing
Re: (Score:1)
I love proxmox, well used to. haven't tried it in a while. Supports clustering too.
Safety and software (Score:2)
Still don't quite understand how this behavior is even legal. If I produce a faulty toaster that ensures you'll get burned when you press the release lever it doesn't matter if 10 or 20 years have passed I'm still on the hook for fixing it.
If I produce a faulty software product that ensures you'll get hacked and your business harmed even if I know of the problem and have fixed it I can just laugh in your face and refuse to fix it for you or fix it only after demanding a kings ransom.
Why should this behavio
Re: (Score:3)
Still don't quite understand how this behavior is even legal. If I produce a faulty toaster that ensures you'll get burned when you press the release lever it doesn't matter if 10 or 20 years have passed I'm still on the hook for fixing it.
If I produce a faulty software product that ensures you'll get hacked and your business harmed even if I know of the problem and have fixed it I can just laugh in your face and refuse to fix it for you or fix it only after demanding a kings ransom.
The operating environments for toasters don't really change over time, neither do the threat models. For software, the threat models and operating environments change over time. Toasters are also much simpler devices. The only external environmental inputs that a toaster has is electrical power and that really only has two things that could change - voltage and frequency. If a hacker changed the power entering your house from 240 volts to 400 volts and that caused your 20 year old toaster to start a fire or
Re: (Score:2)
The operating environments for toasters don't really change over time, neither do the threat models.
For software, the threat models and operating environments change over time.
I don't think the issue is presence or absence of security features but rather programming defects causing vulnerabilities that should never have existed in the first place.
The only external environmental inputs that a toaster has is electrical power and that really only has two things that could change - voltage and frequency.
There are lots of things that can change over the lifetime of the toaster that have to be considered in its design. For example environmental changes from loading of toaster with gunk and crap, exposure to moisture and component wear with time.
If a hacker changed the power entering your house from 240 volts to 400 volts and that caused your 20 year old toaster to start a fire or burn you, no one would blame the manufacturer.
This is one way "surge suppressors" get fried and catch on fire. God only knows how many of t
Re: (Score:2)
Did you have to accept an End User License Agreement for your toaster? If so, what did it say?
Because the EULA for your software says that there is no warranty and that the software does not even have to work.
I can tell you that I will never purchase any product from Symantec after they sold me a faulty product and then refused to provide a solution without me buying the product again. This was boxed commercial software that advertised the feature I was after right on the box, that feature simply did not w
Re: (Score:2)
IANAL, but since your question involves legal, it comes down to the difference between "civil" and "criminal".
I won't rant about how that frustrates me, especially because I suffered great personal and financial loss that the local DA would not touch until I spent tens of thousands of dollars doing investigation and taking it to civil court, then they'd look into criminal charges. WTF!! If I steal some money from you, DA will prosecute with a little evidence, right? You don't have to sue me in civil cour
Re: (Score:1)
Maybe try an alternative? (Score:2)
I've never used VMWare myself, but having seen the comments here, you do wonder why companies keep using it when there's support issues like this, along with a high cost too. For SMBs, something like Proxmox where you can run it for free or pay a relatively small server fee for support/earlier access to updates might be an alternative.