QNAP NAS Users Locked Out After Firmware Update Snafu (theregister.com) 46
A firmware update has left QNAP network-attached storage device owners unable to access their systems, with standard reset procedures failing to resolve the issue.
The problematic update, QTS 5.2.2.2950 build 20241114, was released last week before being partially withdrawn, according to user reports on QNAP's community forums. QNAP, the Taiwan-based storage manufacturer, has not specified which models are affected by the faulty firmware.
The problematic update, QTS 5.2.2.2950 build 20241114, was released last week before being partially withdrawn, according to user reports on QNAP's community forums. QNAP, the Taiwan-based storage manufacturer, has not specified which models are affected by the faulty firmware.
Re:Field updates ... (Score:4, Interesting)
Re: (Score:1, Troll)
Glad I keep two QNAP NAS devices. Primary one rsyncs to the Secondary one overnight. That way if some shit like this happens i'm not locked out of my data. When it comes to firmware updates I always apply them to the Secondary one 1st, give it about a week to make sure there are no issues before updating the Primary one.
Now there’s the capitalist solution. Why buy one when you can just buy two at twice the price.
Especially with vendor pre-rollout patch validation being as mythical as Bigfoot. /s
Re: (Score:2)
Now there’s the capitalist solution. Why buy one when you can just buy two at twice the price.
Especially with vendor pre-rollout patch validation being as mythical as Bigfoot. /s
Alright, let's hear your solution. I'm sure it's gonna be a doozy because you seem to actually be arguing against something that most people know to be true... If you've got critical data, HAVE A BACKUP.
I suspect you've also got a problem with RAID arrays, yes? Why have 1 drive when you can have 5 at five times the price, yeah?
Re: (Score:2)
Now there’s the capitalist solution. Why buy one when you can just buy two at twice the price.
Especially with vendor pre-rollout patch validation being as mythical as Bigfoot. /s
Alright, let's hear your solution. I'm sure it's gonna be a doozy because you seem to actually be arguing against something that most people know to be true... If you've got critical data, HAVE A BACKUP.
Calm down. I value data protection. I was more referring to the costly notion that the consumer-grade answer for this, is to simply buy another one.
And yes. I do have an answer for my data protection. It’s called a sync to differential hardware stored both (warm) onsite and (cold) offsite that is much less expensive because it’s the backup. This problem targeted entry-level hardware. That’s home consumers. Also known as the customer group not used to having to buy two of anything oth
Re: (Score:1)
I have a car that might not start tomorrow due to bad gas. The answer to that isn’t a second car. Unless you’re the one selling cars.
Of course it is, if that car is critical to your life, income, or general well-being. If you live in some remote location where a non-working vehicle could result in your death, then you better have two.
Yeah, if you're 5-minutes from civilization, you don't need a second car.. But if the car failing to start could result in a critical loss of income, and there isn't a rental place you can visit in a reasonable amount of time, you better have a backup.
Re: (Score:2)
Please pass that crack pipe around NO-ONE owns a back up car that just sets there in case your other one doesn't start unless your P-Diddler.
Your just making his point even more dramatically.
Re: (Score:2)
You're wrong. A lot of families have a vehicle that gets used only occasionally. The third or fourth vehicle on a policy is cheap to insure.
Re: (Score:1)
Please pass that crack pipe around NO-ONE owns a back up car that just sets there in case your other one doesn't start unless your P-Diddler.
Your just making his point even more dramatically.
That's pretty common around here but not in quite the way you say it. We use cars for the daily commute and keep 4x4 trucks for everything else. We don't use the latter much, but you can't pull a tandem axle trailer with a car. Nor are they much good in deep snow. The highways are plowed, but the roads leading to them are not.
This morning the wife's car had a flat tire so she took the Tahoe. As the car is due for factory service anyway I'll probably loaded it onto the trailer and drop it off at the dea
Re: (Score:2)
Using the word 'simply' implies, to me, either that you do not well understand data management, or that you haven't actually done so very carefully or very well.
There are few 'simple' solutions to operating and managing your own data server(s). At least, to clarify, do it well and minimize risk of data loss. And expense should not be considered a 'simple' matter either, it need not be onerously expensive, but cheap is also usually not good.
Re: (Score:2)
Just looking over the logic used when your car has bad gas....
comparing that to the previous discussion, is it possible to temporarily rent a NAS, as one can do with a car? I mean, if my car has to go to the dealership, said dealership offers a rental, sometimes for free (if you bought the car that's in the shop at that dealership). If I can rent a NAS that has all of my data on it, then I need to rethink my entire IT mentality.
Re: (Score:2)
I have a cheaper solution... I put OpenMediaVault onto my Qnap NAS. Now I get decent updates at a time of my choosing. I don't have a myriad of confusing menus and other cruft to administer it, I can run containers really cleanly and I don't have that stupid robot thing blinking at me the whole time. I actually don't think there's been a CVE that put me at any risk since I've had it - there have been a few for sure, but they never seem to be too serious (and usually require console access to the OS).
Qnap ju
Re: (Score:2)
A lot of Android phones and some routers have a builtin dual boot system, such a firmware flash always writes to the partition that is not currently running. There's always a backup. It's mostly thanks to this system that I have never bricked a device.
Re: (Score:2)
qnap is special in that there really is no firmware on device, its saved to the hdds in a special partition. I see no reason why they couldn't do this too. have a little grub menu that lets you choose which firmware to boot. Qnap has always been a little trash with their updates in my experience.
Re: (Score:2)
Indeed, if I recall, the drives are formatted ext4 and put in a raid configuration (if there are multiples). As long as they're not striped, it should be possible to just connect them to a PC running Linux and mount them to get the files off. If they're not in a mirrored config then it could be trickier since you'd probably need to get them all hooked up to the computer and get them mounted together (probably using mdadm).
I only ever set mine up in mirrored raid - but I also block their IPs from accessing
Re: (Score:2)
Qnap has always been a trashy little SOHO prostitute. Anyone thinking they make anything to a commercial grade needs to be fired on the spot.
Re: (Score:2)
There is a spot for QNAP and Synology. Their NAS solution may not have dual controllers. battery backed up RAM cache, and other items, but a basic dual-drive NAS is far better than nothing for a home backup solution, especially if the NAS can sync data offsite, so people can have 3-2-1 backups. For a lot of people a NAS with an external USB drive for local backups and some offsite cloud solution like Wasabi or Backblaze can provide a good amount of protection.
Scaling up from there, their RAID appliances a
Re: (Score:2)
Hell even running SOHO IT myself (a podunk k12 school) I'd never trust one of these small 4 drive NAS boxes as anything other than an expendable backup unit. I've had unusually high drive failures, and even a controller failure with these things (Not just QNAP, whoever made the Terastation too)
Install TrueNAS on the QNAP... (Score:2)
The firmware is stored on a disk-on-module. On the Intel machines with a HDMI controller, you can boot into BIOS, boot from a Linux distro and install Linux on an internal or USB drive. Just make sure you dd off the stuff on the disk-on-module to a safe location, because finding the original dd image can be tricky.
Once you have the machine booting Linux, treat it as anything else doing that. I bought a relatively low-end machine, maxed its RAM, and now it is doing an excellent job at slinging samba files
Re: (Score:2)
I'd actually never known what QNAP was before this article. Is it a NAS that's capable of functioning as a computer? I'm thinking about building a highly customizable NAS (OpenMediaVault, luks, btrfs, and bcachefs). Are these the ticket, rather than a so-called mini-computer (which has more CPU than I'd ever use)?
Re: (Score:2)
The Intel ones are PCs, but sans GPUs. You often have a PCIe slot that you can put a basic GPU in and use it as a PC. Just remember to get an Intel 64 bit CPU with a HDMI port, so you can use a local console with it. The CPU is sort of slow, so it wouldn't make a great desktop machine unless you buy one of the more expensive ones, but it can work as a Linux machine.
I like the NAS form factor, as it handles the drive bays well enough, and usually has two slots for NVMe drives for the smaller ones. For OS
Re: (Score:2)
That sounds like basically what I'm looking for. I'll investigate these, thanks!
Re: (Score:2)
> It's mostly thanks to this system that I have never bricked a device.
QNAP would do it for you, for free.
Re: (Score:2)
At the minimum, have four boot methods:
The latest firmware.
The previous firmware.
Firmware installed, but yet has to be booted from, so if it botches, it will just go back to the previous firmware, without destroying earlier versions.
A bootloader which can slurp an install image from a USB drive or SD card, validate the image's signature, then flash that.
This way, regardless of issue, the machine has a good chance of booting.
Works for me since it came out. (Score:1)
This is not the first time (Score:4, Interesting)
QNAP seem to have a history of pushing out bad firmware updates.
After the Deadbolt ransomware, QNAP started enabling automatic updates by default. If you updated your firmware, QNAP enabled automatic updating regardless of whether you had it enabled or disabled prior. They didn't tell anyone. You just had to *know* that you needed to manually disable automatic updates each time you did an update. I do update my stuff, but I need to know an update is reliable before I go live with it. That is not much to ask.
About three years ago QNAP pushed a dodgy update that failed and corrupted my RAID array. I wasn't the only one affected. Luckily I was able to recover most of my data. I ditched the QNAP OS pretty quickly after that. Luckily I was able to get Unraid up and running on the hardware, although the CPU was a little too under-powered for much more than that. I wouldn't touch QNAP again, even if they paid me.
Re: (Score:2)
I was lucky -- Deadbolt passed me by because I had firewalling enabled, blocking everything but incoming connections on the local subnet. That way, if QNAP's connection broker got something outside of the IP range, it wouldn't be allowed to touch the NAS. They finally got some sense and don't have that on by default, but a lot of people want to use their NAS on the Internet as a "hardware GDrive", so this was probably set as a default for people who didn't understand the difference between a NAS and a clo
Software update borks storage device (Score:2)
Re: (Score:1)
Of course it is but that would cost a few extra cents (storage for A/B images or similar), and shave 0.001% off profits, so that would be a hard "no" then.
Re: (Score:2)
It would also consume a bit of memory, since you need to keep a duplicate copy of the unupgraded system around. Switching to the older one would be easy, but keeping the older one around eats memory. (Sometimes it's done anyway, of course. See "recovery partition".)
Re: (Score:2)
No really. all you need to do is create an archive of any system files the upgrade replaces. One time, I remember using a utility that recorder the steps in a standard installer to create an unattended installer. It would have been trivial to add the roll-back facility.
Re: (Score:2)
Other commenters say their OS code is stored on customer disks.
Reflexively anticapitalist sentiment may not apply.
Re: (Score:2)
Possible? Yes. Did they do it? No.
Re: (Score:2)
Absolutely not because Qnap is not a commercial company and has never made a firmware chip in there life.
It's just a junk raid backup device with no brains.
Re: (Score:3)
here's the story I tell everyone in embedded, who is new to the field.
you want 3 boot partitions. 2 for flash and 1 for ROM.
your update toggles between partA and partB. you cant write to partC, its minimal and in rom. its there in case both A and B are not bootable (watchdog based).
I worked at a company that made radios that were for ISP's to use, tower to tower. a tower climb can cost $10k. you do NOT want to have to hire someone just to reboot or fix bad firmware. we learned that the cost of 3 parti
acronym (Score:1)
Quality Not APplicable
It's fixable thankfully (Score:4, Informative)
Re: (Score:2)
Non trivial with hardware using NVME SSDs. Guess it will be time to retire one of my older appliances.
Re: (Score:2)
I hadn't yet thought of hotplugging NVME drives. Well, another wrinkle to consider in my own NAS upgrade project.
Re-released with the same build identifier - WTF?! (Score:5, Insightful)
OP writes "The problematic update, QTS 5.2.2.2950 build 20241114, was released last week before being partially withdrawn " (my emphasis) but according to QNAP, the fixed version has the SAME BUILD IDENTIFIER:
"In response, QNAP promptly withdrew the operating system update, conducted a comprehensive investigation, and re-released a stable version of QTS 5.2.2.2950 build 20241114 within 24 hours."
For the love of all that is good and proper, you don't re-use a build number!
The release notes for this build [qnap.com] make no mention of the latest fix. This is sloppy sloppy sloppy release management. It may not be as globally disastrous as CrowdStrike but it demonstrates just as much negligence on the part of the vendor.
I'm waiting for the next release and am going to let it soak out in the wild before I apply it.
Re: (Score:2)
Look at it from the bright side: They don't hide the fact that they are selling garbage.
Not through any honesty on their part I just think their incompetence is all-encompassing :D.