Crashing an In-Flight Entertainment System 322
rabblerouzer writes "Hugh Thompson, who was interviewed by Slashdot on the dangers of e-voting, now has a cool blog entry on how he was able to bring down the gaming/movie console on an airplane. He calls it one of the most interesting examples of a software 'abuse case' he has ever seen." Fortunately the IFE system is totally disjoint from the avionics.
Looks like the airline got their own back. (Score:5, Funny)
Slashdotted Already (Score:2)
No replies and its already Slashdotted. And I thought nobody RTFAs.
Re: (Score:2, Insightful)
But seriously, those who generally read the article have less of a chance of getting a post higher up in the discussion that those who just jump in (Whether they jump in due to a high level of comfort in the subject, or sheer bravado.) This means that people who post early ar
Profit? (Score:5, Funny)
1. Connect to wireless WAN and Internet
2. Install web server and post link to slashdot
3. Short sell airline stock
4. ???
5. Profit!
Re:Profit? (Score:4, Insightful)
1. Steal underpants.
2. Short sell fruit of the loom stock.
3. Profit!
Long? (Score:4, Insightful)
Re: (Score:2)
I once crashed a bowling alley.... (Score:3, Insightful)
I made a trivial and totally unintentional mistake in the set-up (punching in
number of players, their names, etc) and it brought down the whole system.
Re:I once crashed a bowling alley.... (Score:4, Funny)
Re: (Score:3, Funny)
Go on strike?
Re: (Score:3, Funny)
>
+1 unintentionally funny
Probably redundant by now, but... (Score:4, Funny)
What, did they link
Go look up "fortune" or something (Score:3, Insightful)
Re: (Score:2)
It certainly was good fortune for the passengers that it was separate. It's not that way by happenstance, but it's still fortunate. I think the word you want to get all huffy about is fortuitous.
Re: (Score:2)
Face it, you failed as a grammar nazi (but now you get a second chance to redeem yourself by correcting my mistakes in grammar
The word wasn't "fortune" (Score:4, Informative)
Another example...if I give you "a murderous look" it does not mean (or even imply) that I killed you, attempted to kill you, or even contemplated a violent act toward you. "Murder" and "murderous" are not as close in definition as they are in derivation.
Re:Go look up "fortune" or something (Score:5, Interesting)
You are committing the logical fallacy of "Argumentum ad Stultum": argument from stupidity.
Arguments that commit this fallacy have the form:
It would be stupid to do X
No one would ever do anything stupid
------------
Therefore no one would ever do X
The second premise is so obviously false it hardly needs mention.
So, simply because there is no reasonable scenario that would put the IFE system in a position of affecting the avionics does not mean there is no probable scenario in which this could occur. It requires good engineering, good management and yes, good luck, to ensure independence. Every engineer knows that we must try to eliminate luck from the process and must never, ever rely on it, but also that it will always be a factor.
One obvious way in which the IFE could affect the avionics is via coupled grounds. Grounding in aircraft is never simple, and maintaining fully independent power supplies has been a challenge for IFE and avionics engineers. IIRC the 777 was delayed for a while due to the need to do some redesign on the power systems to ensure independence was retained. In any system so complex there will always be an element of luck, despite the engineer's best efforts.
Re: (Score:2)
Not a big surprise (Score:2)
Re:Not a big surprise (Score:5, Insightful)
Re: (Score:2)
Why bother spending time and money protecting a Tetris system? In TFA, it failed, they reboot, it works. Sorry if you lose your high-score.
TFA? (Score:5, Funny)
Carefull, this may encourage people to actually RTFA...
Re: (Score:3, Funny)
Hm. (Score:5, Insightful)
Re: (Score:2)
Not anymore
Re:Hm. (Score:4, Insightful)
A sad commentary on the state of freedom in this country.
Re: (Score:2, Flamebait)
It is absurd to treat the parent's post as anything other than the anonymous and hyperbolic ramblings that they are. The post is not a commentary on anything (well, a meaningful commentary at least).
Re: (Score:2)
intent... (Score:3, Insightful)
But this isn't just some kid accidentally hitting that remote, changing things to 5, then playing.. or then realizing they can hit up a few more times.. and then playing.
This guy actually knew, in his mind, what was going on.. Not only that... at the point where things would go wrong, he actually paused, sat back, made the change that might make things go wrong and enjoyed the h
As a side note... (Score:4, Informative)
Yeah tell me about it .... (Score:5, Interesting)
Re: (Score:3, Insightful)
I can't believe that people in the real world build systems based on such a piece of crap.
Re: (Score:2, Interesting)
http://i12.tinypic.com/2j17rc4.jpg [tinypic.com]
The IFE had to be rebooted 3 or 4 times during a 5 hour flight, some people's screens never worked at all. Luckly I caught a snapshot of the offensive software's startup screen.
Re: (Score:2)
Re: (Score:2)
Torture! (Score:5, Funny)
Re: (Score:3, Funny)
Bug in the article too! (Score:2)
Re: (Score:2)
That's suboptimal. (Score:2)
But, if you want to confuse them, use:
value = ((12 >> value) & 3)
as this gives you greycodes.
Okay (Score:4, Funny)
Some of these systems run Linux, and how I got bla (Score:5, Funny)
You can tell it's Linux when it crashes. (Score:4, Interesting)
I don't know enough about Linux to understand what it said on my screen when it was trying (and failing) to boot back up again:
http://washedashore.com/misc/inflight_error.jpg [washedashore.com]
(This was April 23, 2005, on a flight from Bucuresti Romania to NYC.)
-Ben
Re:You can tell it's Linux when it crashes. (Score:4, Informative)
Re: (Score:2, Informative)
The ldd call would make sense for debug output, but interestingly it doesn't print anything like what ldd would. In fact, it likely isn't the usual ldd(1) [tin.org], but another binary that happens to have the same name; especially since the debug output stops there, suggesting that it didn't return a
Abuse case (Score:4, Informative)
I think it's more of a case of bad quality control. If the testing environment of the developers had contained a single "lets throw an exception" or maybe a "lets try to lock up a process at 100%" test, they would have see that they needed to at a bit of exception handling (in the first case).
But writing good test cases can be hard.
Anyway. I've seen code like this tons of times. Some people apparently have issues with (how hard can it be), so they use equal instead, but one day, the step value is changed from 1 to 2 (make it go directly from 99 to 101), or some routine fails and returns a default value of -1. And suddenly the code is in the twilight zone.
Anyway^2, I actually did find this rather un-interesting.
Re: (Score:2)
Very hard, apparently.
Way too much effort (Score:3, Informative)
--Paul
Re: (Score:3, Funny)
Virgin Atlantic flight ... to London a couple of years ago
What, that low-end system with an Nintendo NES emulator and an analog TV tuner, run from an under-the-seat box that cuts into legroom?
cookie monster (Score:5, Funny)
Not connected to the avionics system.... (Score:2)
TSA (Score:3, Funny)
Similar Crash (Score:3, Interesting)
Playing around (there really was nothing better to do) I found that quickly wiggling the bat around with 4 balls on the screen would crash the system. After about the 4th or 5th crash the system came back up but this time with all the games enabled! After that I was careful not to crash the system but still about 30 minutes from landing it crashed again and came back up with only the free games.
I wondered at the time how such an easily triggered failure could have been overlooked. Unlike the article my crash only affected my screen...but at least there was some beneficial affect!
Re:Similar Crash (Score:4, Funny)
Now wait for the folks from IFEAA (IFE Association of America) to send their lawyers at you branding copies of the DMCA and calling you a pirate.
The Airline and Aircraft (Score:4, Informative)
Peanuts..... (Score:2)
Red Hat and Delta (Score:2)
I flew Delta today, and they rebooted their IFE to try and address an issue. I thought it was pretty interesting when I saw the familiar linux boot messages appear, and indeed it was Red Hat--complete with Tux icon. I hadn't heard of Delta's use of Linux, and if it wasn't for the reboot you would never have know--who knows where Linux is in use by the masses, and don't realize how important it is becoming to them?
Unfortunately, the reboot didn't fix the issue.
Level of Safety (Score:4, Informative)
There was/is no danger of this happening. I develop software for major airline Flight Management Systems (FMS) and the entertainment system is physically separated from the FMS as well as other "flight critical" systems. Also, Software on an aircraft needs to be developed according to the guidelines of RTCA's DO-178B, which classifies the fallout of software into "levels". The most critical, Level A, like autopilot and flight controls requires very stringent evidence of verification. The least critical, Level E, requires basically no verification or documentation whatsoever, and this is what entertainment systems are developed under.
There was a case in the early days when in-flight entertainment systems were first put on planes where a short in the video system crashed other critical computer components due to the entertainment system and flight system being on the same electrical bus. This obviously caused changes to the rules, so now everything is separated.
Re: (Score:2)
Re: (Score:3, Funny)
Re:Slashdotted already? (Score:5, Funny)
Mirrordot to the rescue (Score:5, Informative)
Re: (Score:2)
Re:Err (Score:5, Funny)
Well, gee. I hope that that little map of the Atlantic Ocean with my plane superimposed on it only has read privileges on /dev/autopilot :).
Re:Err (Score:4, Funny)
Re: (Score:2)
Re:Err (Score:5, Informative)
Re: (Score:2)
Ergo, true, ipso facto. Q.E.D.
Well done.
Re:Err (Score:5, Funny)
Re:Err (Score:5, Insightful)
Since 100 is an average IQ then the opposite of a number 80 points above average would be a number 80 points below average.
Re: (Score:2, Insightful)
Then the opposite of 180 would be appx 55-60...?
Re: (Score:3, Interesting)
There are quite a few IQ tests and they are usually structured so that the majority of people fall right around 100 with a max possible score of 200.
IIRC, over 80% of all people fall in the 10 point range around 100 (or maybe it was with 10 points of 100).
A 150+ on most tests is in the upper 2% of the population.
Mycroft
Re:Err (Score:5, Informative)
IQ scores are a standard distribution with a standard deviation of 10 and a mean of 100. Therefore,
IQs +/- 1 standard deviation from the mean, that is, 90-110, account for approximately 68% of all scores.
The 80-120 range will account for roughly 95% of the scores.
And 70-130 will include over 99%.
Obviously, an IQ of 180 is astoundingly high. An IQ of 55-60 is, I believe, in the mentally retarded range. Since there's not really a good way to quantify "half as smart" and "twice as smart," you could consider that accurate if you wanted, I suppose. Personally, when I think of somebody who is "half as smart as average," I don't think it's that bad.
From Wikipedia: [wikipedia.org]
* mild mental disability: IQ 50-55 to 70; children require mild support; formally called "Educable Mentally Retarded".
* moderate disability: IQ 35-40 to 50-55; children require moderate supervision and assistance; formally called "Trainable Mentally Retarded".
* severe mental disability: IQ 20-25 to 35-40; can be taught basic life skills and simple tasks with supervision.
* profound mental disability: IQ below 20-25; usually caused by a neurological condition; require constant care.
There are also a bunch of debates as to bias and whether IQs really measure anything worthwhile which I'm sure you can find on the same Wikipedia page if you're interested.
Re:Err (Score:5, Funny)
IQ 70-85 - idiot that will buy what advertisers tell them to buy. #1 buyer of 4WD SUV's because they believe they will be safer. Believe that they really are the center of the universe. Prime candidates for Middle managenent, Sales and Marketing departments.
Re: (Score:3, Informative)
per the WAIS-III manual sitting in front of me, the std. dev is 15, not 10. Therefore, 85-115 is +/- 1 s.d. from the mean of 100. But your point is still accurate that a an IQ of 185 is astoundingly high. Mental retardation is -2 s.d.'s below average, which puts that at an I.Q. of =70. You also need significant adaptive impairment in at least two domains (e.g. communication, self care, interpersonal skills, etc...)
just my
jeff
Re: (Score:2)
Re:Err (Score:5, Informative)
Re:Err (Score:4, Interesting)
Interesting. I went to swissair111.org [swissair111.org] and read up on the incident. They are now reporting that "MICHAIL ITKIS, CEO OF INTERACTIVE FLIGHT TECHNOLOGY CHANGES NAME TO MIKE SNOW". So apparently we need an extra step in the old cliche:
Re: (Score:2)
It's no more because it was manages by a bunch of greedy, prentencious bastards that are currently in court to answer for their doings. At least one guy is going to jail for a couple of years.
Re:Err (Score:5, Informative)
Re:Err (Score:5, Interesting)
Yes, the wiring insulation burned and brought down the plane. A friend's wife was on that plane, so I have an interest beyond the technical.
Another interesting event was the crash of an Airbus flight control system, resulting in an inflight rebooting message; the pilots flew on in manual.
Greed. SwissAir is no more.
Yes, but it was due to them overpaying their employees and not controlling other expenses as well - a problem many European state run airlines have. Look at Alitalia for example - they could lease planes with crews for less than it costs to fly their own. Europe's carriers are heading towards teh same consolidation and liquidation taht US ones have expereineced and only a handful will survive. I think BA Lufthansa and Air France will probably be the last standing.
Avionics programmers (Score:5, Informative)
FAA regulations categorize software in 5 different levels of criticality, depending on how a failure of the software would affect the safety of the plane. Level "A" software is reserved for things like the "low fuel" alarm, which could potentially knock the plane out of the air on failure, to level "C" for things like the cabin pressurization system where the pilots can take emergency actions to compensate, to level "E" for things like the microwave in the kitchen.
(Beware: I gloss over a few details for clarity.)
The higher levels of software criticality have progressively higher levels of standards for testing. In the case of level-A software, each individual line of code must be examined for correctness in the context of the rest of the code. Each line of code must be executed as part of testing and actively shown to be correct, and each line of code must be individually code reviewed by another engineer.
At the higher levels of software, limit testing is required for all function arguments and if-statements. Multiple-clause if statements such as "if A and B but not C" must be tested for all combinations of the subject clauses, and so on.
In addition to this, all avionics software I've worked on makes a distinction between showing erroneous information and showing *no* information (or, working incorrectly versus not working at all). If the digital altimeter goes blank, the pilots will notice and can take corrective action. If the altimeter is reading the wrong information, then that's a critical failure which could cause an accident.
Thus, avionics software innards are heavily checked throughout execution to ensure proper operation, and any failure causes the system to immediately go offline. All function arguments are ASSERT'ed for correct range, all calculations are checked for range and accuracy, &c.
The entertainment system, and in particular a game within the entertainment system, is almost certainly a level-E software component, and so is not required to go through such rigorous testing. The hardware has to be shown to not interfere with the avionics and that's about it.
Re:Avionics programmers (Score:5, Interesting)
Anyway, the level of testing required was very, very high. I say that even though we were grads working on it, because it was not our choice what to test and what to leave, and they were done multiple times with different people, the the branches, lines run etc compared. It was the lowest level of the software tests and everything was in modules about 30 lines long that needed 100% coverage, every logical combination tested out etc. Plus the languages they used (ADA95, fortran(77?) and assembly) were cut down to remove anything too untestable. I think 'while' loops were out because, as opposed to 'for's, there is a chance of a infinite loop. That sort of stuff. Would be a nightmare to code in.
In not sure how other FCC's usually are, but interestingly this one had 4 CPU's with a fifth 'controlling' one or something and basically each calculation would be performed on all 4 then the results correlated and the majority answer taken. I guess to protect it from a freakish glitch or maybe some deliberate interferance? Not sure, but surely accurate.
Fly-By-Wire - It's not just the software that crashes.
Re:Avionics programmers (Score:5, Interesting)
Re:Err (Score:4, Informative)
One of the most interesting examples of a software "abuse case" came to me rather abruptly on an airplane flight from Las Vegas to Orlando in mid 2005.
Each seat in the airplane had a small touch screen monitor built into the head rest of the chair in front, and on this particular airline, passengers could watch a variety of television channels and play a few simple games. One such game looked remarkably similar to the classic strategy game Tetris, where players use their skills to manipulate falling blocks on a screen to try and form horizontal lines. I'm a big fan of Tetris; for a few months in 1998 I was borderline obsessed with it. I would start looking at everyday objects and start mentally fitting them together with other tings in the room to form weird line configurations. One of the options on this particular airborne version of Tetris was to alter the number of blocks one could see in advance on the screen before they started falling.
To give myself the biggest advantage in the game, I pressed the + control as many times as it would allow and got to the maximum value of 4. I then put on my "bad guy" hat on and asked: How *else* can I change the value in this field? Near my armrest was a small phone console; you know, the one where you can make very important calls for a mere $22 per minute. I noticed that the phone had a numeric keypad and that it also controlled this television monitor embedded in the seat in front of me.
I then touched the screen in front of me to highlight the number "4" in the options configuration shown in Figure 1. I tried to enter the number 10 into that field through the phone keypad with no luck: it first changed to the number "1" followed by the number "0". Frustrated, I then made the assumption that it would only accept single digit values. My next test case was the number "8"; no luck there either, the number didn't change at all. I then tried the number 5: success! '5' is an interesting test case, it's a "boundary value" just beyond the maximum allowed value of the field which was '4'. A classic programming mistake is to be off by 1 when coding constraints. For example, the programmer may have intended to code the statements:
0 value 5
When what actually got coded was
0 value = 5
I now had the software exactly where I wanted it, in an unintended state; the illegal value 5 was now in my target field. I then turn my attention back to the screen and hit the + button which, to my complete surprise, incremented the value to 6! Again, an implementation problem, the increment constrain probably said something like "if value = 4 do not increment." In this case, the value wasn't 4 but 5 so it happily incremented it to 6! I then continue to increment the value by pressing the + button until I get to 127 and then I pause for a moment of reflection. 127 is a very special number; it is the upper bound of a 1 byte signed integer. Strange things can happen when we add 1 to this value, namely that 127 + 1 = -128! I considered this for a moment as I kicked back a small bag of peanuts and in the interest of science I boldly pressed the + button once more. Suddenly, the display now flashes -128 just for an instant and then poof...screen goes black.
Poof...screen of the person next to me goes black.
Screens in front of me and behind me go black.
The entire plane entertainment system goes down (and thankfully the cascading system failure didn't spill over to the plane navigation system)!
After a few minutes of mumbling from some of the passengers, a fairly emotionless flight attendant reset the system and all was well. I landed with a new-found respect for the game of Tetris and consider this to be the most entertaining version of it I have ever played.
.
Re:Err (Score:5, Interesting)
Can anyone intuit the airline? Because without an airline name, I call bullshit on this story. I would guess it had to be business class, and probably a foriegn carrier, if the story is to be believed.
Re: (Score:2, Interesting)
Re:Err (Score:4, Funny)
Re:Err (Score:4, Funny)
Re:Err (Score:5, Funny)
uh.... yeah.... (Score:3, Funny)
You just summarized TFA's explanation of "what went wrong," but less succinctly and less coherently than TFA.
Way to go.
Re: (Score:2, Funny)
There is a NAME for the bug... (Score:5, Informative)
Dam lazy programmers not using Assert() these days...
(And yes, I am one, programmer that is, not lazy
Re:There is a NAME for the bug... (Score:5, Interesting)
Re: (Score:3, Interesting)
Re:There is a NAME for the bug... (Score:5, Insightful)
Because we all know that the constraint that you got wrong in the actual code will be correct in the assert. Right? Right?!
Re: (Score:2)
``Dam lazy programmers not using Assert() these days... :)''
Because we all know that the constraint that you got wrong in the actual code will be correct in the assert. Right? Right?!
Moreover, an assert would not have fixed this particular problem. The purpose of an assert is to highlight errors by making the code crash right away as soon as an unexpected condition arises, rather than dragging the error condition along, and only crash much later. In our case, this would have made the screens blank as soon as he pressed 5...
Re: (Score:3)
#ifdef DEBUG
#define ASSERT(x)
#else
#define ASSERT(x)
#endif
ASSERTS are all well and good but not as useful as they seem for 2 reasons. Firstly the assert would never have been fired in the IFE software as, clearly, nobody tested this code path anyway and secondly if an assert _does_
Re: (Score:2)
Re: (Score:3, Informative)
http://www.shelleytherepublican.com/2006/12/01/li