Follow-up on EVE's Boot.ini Issue

Krinsath writes "CCP, publishers of Eve Online, have posted a Dev Blog detailing the circumstances leading up to the deletion of XP's boot.ini file, which was earlier discussed on Slashdot. The blog post has intimate details about how the mistake occurred (a new installer from their normal one), how they responded and what CCP has learned from it. While fairly dry, it is to the company's credit that they're being open about one of the more serious bugs to crop up in gaming's recent history."

That's actually a really straightforward response.

[Silverlancer (786390)]

Now if only more businesses acted this way.

Re:That's actually a really straightforward respon

[Harmonious Botch (921977)]

But they should delete greater percentages of XP...

Straightforward, sure.. but... | also, the bug

[Animaether (411575)]

what of the users who did lose valuable computer time due to this problem? The proverbial kid handing in their homework (or dissertation paper or whatever), for example. Apologizing and willing to pay for a third party tech support service (e.g. Geek Squad) is nice and all, but does that cover damages incurred? doubtful. Perhaps that EULA will finally get a test.

As for the bug itself... the installer code is NSIS script; quite powerful, but you do need to know what you're doing. Especially with a command such as "Delete", I can't help but wonder who failed to RTFM (TFM reads, as they point out, that "Delete" requires a full path to be safe or else it expects the path to be root) and instead made an -assumption- on how it would work.

Now, to their defense, NSIS is also a little inconsistent (RMDir needs /r to be recursive, but DeleteRegKey needs /ifempty to NOT be recursive; whatthe.) and I've wiped my entire root myself while developing an installer with it, although via a more complex bug.. NSIS simply doesn't have any built-in "you dumbass"-protection like most commercial installers.

Although I think it's nice of them to say that they're not blaming Windows for their own mistake, I do honestly think that Windows should protect such vital files at all cost - including against Administrator level process (e.g. a prompt "you dumbass - are you sure?" will do).

Re:Straightforward, sure.. but... | also, the bug

[TooMuchToDo (882796)]

what of the users who did lose valuable computer time due to this problem? The proverbial kid handing in their homework (or dissertation paper or whatever), for example. Apologizing and willing to pay for a third party tech support service (e.g. Geek Squad) is nice and all, but does that cover damages incurred? doubtful. Perhaps that EULA will finally get a test.

Almost never will damages be covered. Come to think of it, I think in this case I can say "Damages will never be covered." You have to show value and proof of destruction of that value. Your homework being destroyed? Your dissertation being destroyed? While it may have a large amount of value to you, monetarily it has very little value.

Re:Straightforward, sure.. but... | also, the bug

[zippthorne (748122)]

what of the users who did lose valuable computer time due to this problem?

That's a good point. And generates some good advice for future student/gamers: Do not install any new software of any kind a week or two before a paper is due*

*at least, not without having some kind of back-up which can be read and worked on on another computer and which you regular test.

TFM reads, as they point out, that "Delete" requires a full path to be safe or else it expects the path to be root

That sounds like the the opposite of a good way for delete to fail.

Re:

[LarsG (31008)]

Do not install any new software of any kind a week or two before a paper is due

ITYM "Don't install an addictive game before a paper is due."

Re:

[Atlantis-Rising (857278)]

Although I think it's nice of them to say that they're not blaming Windows for their own mistake, I do honestly think that Windows should protect such vital files at all cost - including against Administrator level process (e.g. a prompt "you dumbass - are you sure?" will do).

Isn't the first thing most people do on Vista is turn off the Administrator "Are you sure?" prompts?

(I know that personally I do not- I don't get them more than a half-dozen times a day, if that, so it's really not that big a deal.)

Re:

[DeadChobi (740395)]

Actually I've found that EVE provides such a nuanced experience that it would be totally lost on anyone not older than 16. The game isn't the GAME, it's the little political games that we play within the rules of the game. For example, having Band of Brothers, the largest alliance in the game, put in an appearance when we're fighting a smaller alliance indicates that that smaller alliance is allied with BOB.

There are more games to be played against other players than just fleet battles. You can fight allian

Re:

[Ecuador (740021)]

Yeah, if only more businesses did not test products enough before deployment, or read TFM when using delete commands...

AND THEN they send GEEK SQUAD to "fix" your computer. Talk about adding insult to injury!

How is that even possible

[AndrewBuck (1120597)]

From the article...

"Why doesn't Windows protect its system startup files? That's a good question, one that I have asked myself in these last few days and wish I knew the answer. But of course I'm not going to blame Microsoft for our mistake. Windows doesn't protect those files and therefore software developers must take care not to touch them. We should have been more careful."

That is a good question. I am not an EVE player myself so I don't know if this update had to be run with admin privileges but it doesn't appear to be that way from the question and reply. If you are not running as admin then how is it even possible to remove a system file that is necessary to boot the system. Unlike the EVE representative making this statement I am going to blame Microsoft, it should not be the developers responsibility to make sure they don't break the OS, it is the OS developers responsibility to make sure that it cannot be broken without admin/system/root access.

-Buck

Re:How is that even possible

[Osty (16825)]

That is a good question. I am not an EVE player myself so I don't know if this update had to be run with admin privileges but it doesn't appear to be that way from the question and reply. If you are not running as admin then how is it even possible to remove a system file that is necessary to boot the system. Unlike the EVE representative making this statement I am going to blame Microsoft, it should not be the developers responsibility to make sure they don't break the OS, it is the OS developers responsibility to make sure that it cannot be broken without admin/system/root access.

Two things to note:

1. This was an XP problem. Technically it could've happened on Vista, but I haven't seen anything that said it did. As such, this falls into the same category of problems that Microsoft attempted to fix in Vista with UAC -- nearly everybody ran XP as admin, and many apps expected you to be running as admin.
2. This was a problem with an installer/uninstaller. Since nearly everything on Windows installs into %programfiles% and that's a shared location, installers need admin access (installers that ask if you want to install for "Just this user" or "Everyone" are not going to install in %userprofile% if you choose "Just this user". They're just looking to see if the Start Menu shortcuts should go into "%appdata%\Microsoft\Windows\Start Menu" or "%allusersprofile%\Microsoft\Windows\Start Menu"). Vista will elevate your privleges when you try to run an installer (you'll get a UAC prompt), after which a misbehaving installer could screw up boot.ini. Regardless of operating systems, you almost always install applications as administrator. Yes, you can install apps in $HOME on *nix systems, but 9 times out of 10 you'll use sudo on the installer (sudo apt-get install foo). Therefore this is technically a bug that could happen on any OS. It's not difficult to imagine an application install that deletes your kernel image, for example.

The real WTF here is that they have an important game file named "boot.ini". That's an exceedingly poor choice of filename. Think of it like having a game file called "autoexec.bat" or "vmlinuz" that actually has nothing to do with the DOS boot process or the Linux kernel. The only defense they give for that is "legacy".

Re:

[mav[LAG] (31387)]

Technically it could've happened on Vista, but I haven't seen anything that said it did.

Well, that would require a group of people who have Vista installed.

Re:How is that even possible

[RulerOf (975607)]

Didn't Quake have an autoexec.bat file as a startup script?

Quake 3, and I assume for 2 and 1, contained a file called "autoexec.cfg." I always thought it was aptly named, being a DOS veteran myself, because it contains game configs like default keybindings (e.g. bind w +move) and such that actually allow you to control the game in the first place, and it's always called during game startup. Very similar in function to the file that it is named after.

Re:

[Bellum Aeternus (891584)]

I did patch EVE as a non-privileged user on my XP Pro system and the problem didn't happen to me. It does seem to be that since most people basically have to run as an administrative account to make XP "work", CCP was able to damage the OS as they did.

This is a multi-part failure. One part Microsoft for making an OS that almost requires standard users to run a privileged account all the time to make basic applications work. One part CCP for developing software that damaged the underlying OS.

My only hope i

Re:

[Deviate_X (578495)]

"how is it even possible to remove a system file that is necessary to boot the system. Unlike the EVE representative making this statement I am going to blame Microsoft,"

The boot.ini file is actually protected. It is specifically marked as a System File, Read-Only and Hidden. This means that to modify the file you need to remove these attributes in a specific order first before you can modify or delete it, even if you are logged in as Administrator.

The only way to prevent software from doing bad things

Re:

[dangitman (862676)]

Why the hell does a game need a producer?

Uh, to produce it?

Re:

[Perseid (660451)]

That's one of the flaws in XP - chances are you can't get away with not being administrator. Some things won't install. A few things won't even run. Not being Admin will lead to an unpleasant experience.

It's nice when companies are honest

[_Shad0w_ (127912)]

If only more companies were so honest and straight forward when they cockup. It almost makes me feel like playing EVE again. CCP can consider themselves as being given a virtual karma bonus.

Although I can't help but wonder if the "honesty is the best policy" choice was because of their handling of the last PR cockup.

Alright!

[Cheezymadman (1083175)]

Finally, a benefit to Vista! Vista users like myself were 100% unaffected by this. It was awesome.

I cant help but wonder...

[Nezer (92629)]

I have never been into MMOs. I just didn't get them. However, in the last week things have changed and it's due, in part, to this bug.

You see, until this bug happened EVE was totally off my RADAR screen. When I read about the bug on /. last week I went to the companies website and found myself intrigued. Further discovery that they didn't charge $50 for the box on top of the monthly fee was also appealing. Further, I see client software for Macs and Linux. Intrigued I download the Mac client and create the trial account. Two days later I'm hooked and sending them my CC #.

If it hadn't been for this bug, I probably would have never bought their product! They say that any publicity is good publicity and I think this is true. Sure the SNAFU was pretty bad yet the product was still compelling enough to buy it despite a pretty bad QA miss. This latest response from the company will only help further get their name out there and is truly an opportunity to make lemonade from lemons.

That was a very, very good analysis of the problem

[Antique Geekmeister (740220)]

Having violated /. policies and actually Read The Fine Article, it was a good analysis. I wish more people would write their bug reports this well, and explain how they're going to address the problem.

I also wonder if they wouldn't benefit from a nice virtual environment system to do QA testing of new releases with? Capturing the full graphical behavior of an OS is difficult in virtual systems, due to the overhead of the virtualization itself, but it might be a lot cheaper than keeping a dozen different hardware configurations around.

Can you imagine how this happened?

[Daimanta (1140543)]

Programming guy 1: So we're finally done with coding everything.
Programming guy 2: Yeah finally.
*Programming guy 2 tries to make a joke*
Pr. guy 2: Hey pr. guy 1, look at this
Pr. guy 1: lol, you appended a del boot.ini
Pr. guy 2: Well, I'm going to take a coffee break
Pr. guy 1: Yeah, me too
Pr. guy 2: Wait, lets put a sticky-note on the board that we're done
Pr. guy 1: Sure
*Pr. guy 2 puts sticky on the notice board*
*both walk off*
*manager walks in*
*manager looks at the board*
Clueless manager 1: Nice, the work is finally done.
Cl. manager 1: Ahhh, I'm on a tight schedule. Lets send this file to the head programmer so he can compile everything.
*Tries to click close*
Cl. manager 1: What, changes have been made? Whatever, save.
Cl. manager 1: Ok, open outlook. Send. Done. Wow, I know this will be a spectacular release.
*Cl. manager walks of*

Installer "technology"

[kabdib (81955)]

I'm not aware of a single installer package on Windows that isn't a useless, complicated, badly-documented, bug-ridden piece of crap. The times when I've had to use (say) InstallShield, I've seriously thought about finding a new job. This stuff sucks *and* blows. (Don't get me started on USB support and BlueTooth. Oh my God. Don't even think about reading about that stuff: Once you crack open the docs and see the wavy tentacles, the squamous mouths, the eyes, the eyes, the eyes that . . . well, you'll never be quite the same again. T-tr-trust me on th-that).

Remember the happy days of "just copy" installs, which worked great on MacOs in the 90s? Upgrade to a new system? Just copy your "apps" folder over.

The question, "What kind of installer should our OS have?" is like asking, "Should we drink the red poison or the green one?" Just asking the question seals your doom.

Re:

[_Shad0w_ (127912)]

CCP write it as EVE on the website.

Re:

[B3ryllium (571199)]

I would, but I had a power outage that terminated my 4-week uptime. :)

Worst part of it was, I was baking pecan butter tarts at the time. In an electric oven. They don't make UPSes for that.