Software Bug Halts F-22 Flight

mgh02114 writes "The new US stealth fighter, the F-22 Raptor, was deployed for the first time to Asia earlier this month. On Feb. 11, twelve Raptors flying from Hawaii to Japan were forced to turn back when a software glitch crashed all of the F-22s' on-board computers as they crossed the international date line. The delay in arrival in Japan was previously reported, with rumors of problems with the software. CNN television, however, this morning reported that every fighter completely lost all navigation and communications when they crossed the international date line. They reportedly had to turn around and follow their tankers by visual contact back to Hawaii. According to the CNN story, if they had not been with their tankers, or the weather had been bad, this would have been serious. CNN has not put up anything on their website yet." The Peoples Daily of China reported on Feb. 17 that two Raptors had landed on Okinawa.

Real redundancy

[chriss]

As far as I remember the Space Shuttle not only has redundant computer systems, but also redundant software, i.e. the software has been developed twice to ensure that software bugs don't cause a catastrophe. I'd prefer to know that systems capable of carrying weapons which can kill hundreds of thousands of people were designed with the same safety in mind.

Re:

[omeomi]

I'd prefer to know that systems capable of carrying weapons which can kill hundreds of thousands of people were designed with the same safety in mind.

nope!

No guarantee

[Goonie]

What you're talking about is called N-version programming. It's no guarantee of reliability, unfortunately. Often, the "bug" is related to mistakes in the program specification, not the implementation of that specification. Therefore, the same bug gets faithfully and correctly implemented twice.

Re:

[Anonymous Coward]

Airbus has taken that to the extreme - their first fly-by-wire aircraft, the A320 has eight independently developed software and hardware systems which must agree. If any one of them computes a different result than the others it is restarted once and disabled if it happens again (and obviously the incident is recorded for maintenance to report). They increased the number to 32 in the A380.

And just to preemptively debunk bullshit that is always brought up when someone mentions Airbus and computers on slas

a bit of nit-picking...

[Anonymous Coward]

What you state about Airbus is absolutely correct but FADEC stands for Full Authority Digital Electronics Control but many seem to remember it as E = Engine, like you do. As far as the fly-by-wire system is concerned, I might add that it has already saved at least 300+ lives - an Emirates A340 attempted to rotate with insufficient airspeed at takeoff (but past V1 so they couldn't stop either) and the FBW system stepped in and throttled up (fortunately autothrottle was on so it was permitted to do so) and ro

Re:Real redundancy

[spagetti_code]

Actually, the space shuttle is not a good example.

NASA do not fly the space shuttle during 31 Dec -> 1 Jan [newscientist.com] as
they are not confident of what would happen. Better just
to avoid the problem.

That was one of the pressures to getting the Dec 2k6 flight off the ground.

Re:Real redundancy

[0123456]

"NASA do not fly the space shuttle during 31 Dec -> 1 Jan"

But they fly over the international date-line every 90 minutes or so with no problems :).

they are very confident

[Tharkban]

You missed a point in that story.
NASA is extremely careful with its software.
They don't fly from Dec 31 to Jan 1 because they know exactly what would happen.

Re:Real redundancy

[Richard_at_work]

The F-22 can carry the standard USAF air delivered nuclear weapon as maintained within the US military arsenal today, either one internal or two external. The radiation from the weapons has no effect on the stealth, either before or after detonation (the stealth capability involved is an advanced form of that used on the B-2 and B-1B bombers, both of which were at their inception designed to be purely nuclear armed bombers).

Re:Real redundancy

[SnowZero]

The F22 does not normally have external mounts, but there are hardpoints where they can be added. Of course that would defeat the stealth, but if you're about to drop two nukes, at that point you're probably beyond being sneaky.

Re:

[TimSSG]

Both of you can be correct. Each group of models of the F-16 had more digital components and less analog ones. Tim S (retired F16 Radar Repair Technician, F-16 C/D models)

Re:

[rtb61]

Defeat stealth technology, easy. Use radar to accurately measure the density of water vapour within the atmosphere, as long as the planes stay still and don't use their engines they are stealthy.

Once they move, don't aim at the aircraft aim at the atmospheric affect with a big enough war head, and problem solved, and fortunately, modern aircraft are far more succeptable to damage than older aircraft so that war head doesn't need to be all that big.

As for ARM, use multiple digitally encoded emitters (loc

Re:Real redundancy

[arivanov]

There is a grain of truth in this one.

During the Serbian wars NATO was scared shitless off all weather radars and shot at them without any second thoughts even if they were in neighbouring non-combatant countries. Both incidents when missiles hit buildings near Sofia (70km+ outside the Yugoslavian border) were actually firings at the Sofia Airport Gematronic radar system (the same kind some NATO country use).

In addition to that Stealth works effectively only if your receiver is colocated with the transmitter. It is easily defeated by decoupling them. There is a host of technical problems in doing this, but nothing that cannot be solved with enough software analysis of the reflected signal. It is only a matter of time until all "rogue" countries possess the relevant signal processing tech to do that.

So as far as AAA is concerned Stealth is a technology which is dead on arrival.

Re:

[gardyloo]

If the radiation is not high enough to kill the pilot at 3 m or so then it is not going to be detectable at 10 km.

Er... You're saying that because I can see the moon, people who've walked on the moon should have been killed? Or because I can see lights from the next town over, I shouldn't go to the next town over?

Lethality and detectability are drastically different things. Admittedly, my eyes are tremendously sensitive, whereas the lethality of visible light is *

Re:

[GeffDE]

Nuclear bombs generally use plutonium-239, which emits either alpha or beta radiation. The number of particles emitted by such material is several orders of magnitude less than the number of photons given off by an incandescent lightbulb. At 10 km, the number of alpha or beta particles that would hit a detector (unless the detector were very large) would hardly be above background. Additionally, because Pu-239 is an alpha emitter, the metal encasing it is enough to block (most) of the radiation.

Re:Real redundancy

[cpt kangarooski]

You're saying that because I can see the moon, people who've walked on the moon should have been killed?

Yes. I hate those guys.

Re:Real redundancy

[Indigo]

The Shuttle does indeed have two sets of flight software, Primary Avionics Systems Software (PASS) [nasa.gov], and Backup Flight System (BFS) [nasa.gov]. During critical phases of flight, PASS is loaded on four of the GPCs and BFS is loaded on the fifth. BFS doesn't have all the capabilities of PASS - it is intended to take over in case of an emergency.

crash narrowly averted

[User 956]

CNN television, however, this morning reported that every fighter completely lost all navigation and communications when they crossed the international date line.

I've heard of a software glitch causing a crash before, but this is ridiculous.

Re:

[ColaMan]

I've heard of a software glitch causing a crash before, but this is ridiculous.

Not really - read the Risks-Forum Digest [ncl.ac.uk], especially the earlier years, and you'll find that software quite often causes physical harm.

I doubt they lost communication...

[sonofagunn]

I find it hard to believe that they would have lost all communication from a software glitch like this. Things like radios, compasses, radars, etc surely still worked. Hopefully this just crashed a navigation system and left the pilots able to fly the plane using conventional navigation techniques. If it brought down everything else, that's a serious design flaw, not just a bug.

Re:

[daeg]

I really doubt such an advanced (and stealth) aircraft would have any "traditional" radio capabilities that could easily be intercepted. If the encryption is written such that the position of an aircraft matters, they may have no communication channel at all.

That said, I'm not sure how this bug would have escaped QA. I mean, it's an airplane. Hundreds of commercial jets fly over that line day in and day out, as do other American military planes. I wonder if the bug also exists at the Prime Meridian?

I hate t

Re:

[abradsn]

Dude, those planes carry full on CRAY (or other brand) super computers in them. They need them for communication, weapons, enemy identification, and geographic location. That's their purpose. Communications aren't done through simple radio communication. It's encrypted and probably bounces off of satellites. Also, I bet those buggers are slighly harder to fly with no computers working onboard.

Not to worry though, they likely have the best pilots in the world flying billion dollar planes. Pilots like t

Re:

[theEteam]

In most modern aircraft, control for all avionics equipment is done through a central mission computer. If that computer crashes(usually there are two but they have identical software), all avionics will be unavailable. This includes radar, navigation, most radios, etc. Usually there is a backup RCU(remote control unit) for one of the radios and of course you can still steer, but that is about it.

Don't worry

[Realistic_Dragon]

We will happily sell y'all Eurofighters. Half the price, twice the bombs... and who the hell do you need stealth to fight anyway? Expecting the France to try and invasion any day now or something?

Re:

[MSFanBoi2]

Actually not really. The Eurofighters have very limited air to ground functionality at least until Block 5 hits sometime in late 2007. It was expected in 2005 but per usual, was very behind. The F-22 can carry two 1000 lb JDAM internally (or 8 GBU-39s) for a total of 2000 lbs of internal weapons and up to 5000 lbs of external weapons on four (two per wing) removable hard points (two of which are plumbed for fuel).

We won't even go into the fact that the F-22 is faster with a full weapons load and much faster

Re:

[stevew]

You forgot one major issue with the F22 versus the Eurofighter. The Eurofighter & every other modern fighter in the world for that matter can't see the F22 on their radars. F22 is full-up stealth (assuming no external stores).

In wargames held in the US with 1 F-22 versus 5 F-15's. 5-0. The F-15 pilots never saw the F-22. Not a fair fight - but then that's the idea.

Re:Don't worry

[MSFanBoi2]

Actually the whole story is a lot more exciting that 5 on 1...

The 27th Fighter Squadron (8 F-22s) at Langley AFB, Virginia fought against 33 F-15Cs and didn't suffer a single loss. The F-15's again didn't even detect the F-22's until they were all locked and targeted.

Then some months later during Exercise Northern Edge F-22's reached a 144-to-zero kill-to-loss ratio against F-15s, F-16s and F/A-18s. Only 12 of the F-22's accounted for nearly 50% of all kills for the Exercise.

Winning Tactic

[R_Ramjet]

The F-15's should have headed towards the international date line....

Knew someone on the development team

[Beryllium Sphere(tm)]

That machine handles and fights so differently that when the engineers flew simulated combat against Air Force pilots, the engineers won. The first few times. As soon as the pilots got the hang of it the engineers were toast.

Re:

[Realistic_Dragon]

I'd be interested to see your cite that the US have lost more than a very small number of aircraft to radar guided SAMs since Vietnam - and MANPADS aren't a big problem for high flying interceptors.

Re:Don't worry

[Dunbal]

Not to mention that the only decent fighters Europe have ever created were all WWII era...

      I dunno, the Americans seemed to quite like the idea of the AV-8A Harrier [wikipedia.org], a British creation.

Re:

[Dunbal]

The US outclassed Iraq pretty badly too. That one is working out well.

      The big problem is a lack of anti-insurgent missiles.

Re:Don't worry

[ralphdaugherty]

As shown in Vietnam and the current Iraq situation, America has great difficulty in fighting a loosely-organized resistance.

      It's because we care about killing innocent civilians, and they are indistinguishable from innocent civilians.

      If we can't identify the enemy, it's a good sign we shouldn't be there.

  rd

Source code...

[alexhs]

That's the real reason why they don't want to give source code to foreign armies... They don't want to be covered in shame :)

UTC

[Excelcia]

The answer to all these problems is very simple. For any mission critical application, use UTC and only UTC. No time zones, no date line, no converting. If the software isn't even aware of the concept of date/time localization, then it's not going to run into problems.

Oh, and while they're at it, standardize on metric too. Maybe we can save our interstellar probes at the same time we are saving our warplanes.

Re:UTC

[Chmcginn]

They probably already do... When I was spending time in uniform, all our (non-workstation) computers did all their work in GMT, anyway. And considering it was the navigation systems that crashed, I think the "international date line" thing is spurious - the problem was more likely going from W to E, not today to yesterday.

Re:

[Dun Malg]

As others have noted, this is less likely to have been an issue of the time/date than it is of transitioning from 179.9degW longitude to 180E longitude. You're just assuming it's a date/time issue because we call it the International Date Line. Note how none of the sources have details.

Reminds me of the Bismarck

[blind biker]

The Bismarck battleship had a bug also: when the main turrets would fire, the aiming radars would be disabled. That's no joke when you're in the midst of a battle and everyone of those large caliber shells counts. As I understand, the radars would be disabled by the vibrations of the turret cannons firing. Not a software bug, but bug nonetheless, and you do wonder how did this battleship pass testing.

Gotta know your limitations...

[TigerNut]

When I worked at a high end civilian GPS equipment manufacturer, we had a test department where, among other things, a complete list of "special" dates and locations were kept on file. Any new position solution software release was regression tested against all previously known and guessed potential date/time rollovers, as well as making sure that motion across geographic coordinate boundaries didn't cause erratic behavior. Obviously whoever supplied the inertial navigation solution for the F22 hasn't quite gotten there yet... Testing in the lab is cheap. Burning a couple of tons of Jet-A and putting a bunch of people at risk is not.

Re:

[TigerNut]

No way for me to know exactly what happened in that case. However, the inertial navigation system on an aircraft has to know its orientation with respect to the earth (in terms of roll/pitch/yaw) and also its position (lat/long/altitude). THere are a bunch of different ways that you can do that. One of them, for example, is the "earth centered, earth fixes" coordinate system. In this system you compute your position in a 3D orthogonal system and then do a mathematical transform to get your lat/long/altitude

Some exaggeration in the story, I suspect

[CardinalPilot]

The F-22 has a fly-by-wire control system. If there really were a crash of ALL on-board computer systems, communication and navigation would not have been the most immediate concerns!

I think not

[brunes69]

Modern fighter jets are aerodynamically unstable by design. A human can not fly them alone, the computer has to correct the flight path hundreds of times a second.

The flight control software thus most certainly *does* have to keep the plane "stable".

Design? Lack of foresight?

[the_skywise]

Assuming it WAS a time issue upon crossing the International Dateline...

Design problem? Why should navigation software require "local time"? They knew they were crossing the international dateline, so they must be linked to GPS timing systems... why not just use GPS' universal time? (Sure, you want local time eventually for your displays but that's a "view" calculation, not one intrinsic to the navigation software)

Bug tracking problem? Did the testers not think of testing about a time zone change? Did they assume the above that everything would be on a universal time and therefore didn't see the need for crossing time zones?

Why wasn't this a stock reusable code module in Lockheed Martin's labs?!?

(And for a media look at this issue, check out the anime Geneshaft or the movie The Pentagon Wars)

How do I mod this foresight post?

[deft]

I'd like to mod it +1 hindsightful.

Were they running Windows?

[pestilence669]

I just want to know if this is in any way connected to the nuclear subs that lost navigation after they switched to Microsoft Windows based software. Generally, when this kind of thing happens, some external vendor is to blame.

Ironically

[mbrod]

A few days ago reading up on good C++ coding techniques I came across Stroustrup's (creator of C++) page citing the coding rules used [att.com] when working on the Joint Strike Fighter [wikipedia.org]. Reading through the various rules used, this one caught my attention:

AV Rule 25 (MISRA Rule 127)
The time handling functions of library <time.h> shall not be used.

I got to thinking if we had any decent alternatives (at least in C++). And yes there are alternatives and all of them looked equally bad to me. Looks like the F22 guys might have had the same problem finding and using a robust fault tolerant time library.

Re:

[Alizarin Erythrosin]

With the inertial navigation systems I work with, time stamping of data is very important. Clocks that are accurate down to nanoseconds aren't uncommon, synching with GPS 1-PPS signals (1 pulse per second) to determine and correct clock drift per inertial sensor read cycle, etc. Timing systems are usually custom built for the product in question as part of the design.

Re:Ironically

[julesh]

A few days ago reading up on good C++ coding techniques I came across Stroustrup's (creator of C++) page citing the coding rules used when working on the Joint Strike Fighter. Reading through the various rules used, this one caught my attention:

        AV Rule 25 (MISRA Rule 127)
        The time handling functions of library shall not be used.

I got to thinking if we had any decent alternatives (at least in C++). And yes there are alternatives and all of them looked equally bad to me. Looks like the F22 guys might have had the same problem finding and using a robust fault tolerant time library.

Why would you need to use a library? The only format you're likely to need in such software is milliseconds offset from some suitable epoch. As long as your hardware can produce such a time value, you're fine.

I call bull...

[flyingfsck]

but it is a nice story anyway.

Er what?

[chanrobi]

Are you telling me that the F-22 has no analog backup flight system? For gosh sakes even the F-16 has a similar system. A cursory google search that the F-22 is equipped with an "LN-100G Inertial Navigation System with Embedded GPS". It sounds incredible that the summary implies that the only way they would've made it home was via formation flying with a tanker? Can anyone with more detailed information on the F-22 clarify?

Re:Er what?

[Jerf]

Before you go ballistic, bear in mind that unless you've got data sources beyond those cited in the Slashdot blurb, the most technical details come from CNN, which is about one step from priding itself on its ignorance of military matters, and has a less-than-distinguished history on the technical details front as well. Put the two together and the odds are low that you've got anything like an accurate view, let alone a complete one.

You can trust the what and the when; I wouldn't trust their how or why any further than I could spit.

(This isn't anti-CNN; this is anti-almost-everything news media. Journalists aren't required to learn squat about science or technology for their degree and it tends to show up in every last article they write with even a passing connection to science or technology. Any even cursory overview of stories on any technical subject you know about will reveal this. Remember that "multi-gear rocket" atrocity from a day or two ago?)

Most modern fighters are intrinsically unstable

[igb]

You can't fly planes like that manually, becuase they are inherently unstable. Even non-stealth aircraft have this property, in order to make them more sensitive in roll. A civilian plane will self-centre from small roll inputs, and you have to overcome that effect to actually roll. The stealth aircraft are such weird shapes, for which aerodynamics come second to radar cross-section, that the designers don't even have the choice.

ian

So...

[WaZiX]

So, when is Service Pack 1 coming out?

Actual dialog message...

[Anonymous Coward]

You are flying to Japan, Cancel or Allow?

F16 Software had similar problems

[Anonymous Coward]

When F16s crossed the equator, the computer would roll the aircraft 180 degrees and fly inverted:

http://catless.ncl.ac.uk/Risks/3.44.html [ncl.ac.uk]

Re:

[patio11]

Its important to note that that bug was present *in simulation only*. My unit tests catch all sorts of nasty edge cases, including some which cause the system to drop huge chunks of the database -- that is what testing is for!

Microsoft?

[Yvan256]

every fighter completely lost all navigation and communications when they crossed the international date line.

Where do you want to go today?

READ: Get Ready For More

[N8F8]

I tried posting this on several sites but on March 11th [wikipedia.org], when the new daylight savings [wikipedia.org]regime kicks in for the first time there will probably be a lot of Java applications that will start having data issues because the latest Java version IS NOT BACKWARDS COMPATIBLE for several three character time codes that have bee removed. Several codes have been deprecated in a way that is not backwards compatible. I could be wrong about the severity, but for he last two weeks my software team has been dealing with this issue and the interaction between Oracle and Java.

I told them...

[guruevi]

...not to run Windows on those machines. They HAD to upgrade to Vista because of all the cool 'features' the pilots would like to see. First we had to put more ram in and an extra video card, now this... I'm telling ya, next time Microsoft gives them a better deal because they're switching to Linux, they shouldn't accept.

not the only problem I read...

[Bobzibub]

All complex systems have bugs that need to be ironed out....

http://www.atimes.com/atimes/Middle_East/IB10Ak05. html [atimes.com]
"Keys notes, however, that the electronic spectrum around Baghdad is polluted by the myriad jamming devices that coalition forces primarily employed to thwart remote detonations of the improvised explosive devices that have inflicted 70% of all US fatalities in that war." ...
"The potential problem was discovered when the first F-22s were operating near US Navy ships off the Atlantic coast. Navy radars overwhelmed the F-22's automated sensors. Even now, larger, multi-station, purpose-built electronic-intelligence-gathering airplanes encounter difficulties around the Iraqi capital because of the extreme density of jamming devices."

Carry backup.

[jo7hs2]

My advice to F-22 pilots: 1) Superglue a handheld GPS into your cockpit. 2) Carry a backup radio. Superglue this to your cockpit. 3) Remove your cockpit, and superglue it onto an A-10. 4) Fly safe. Carry superglue.

Re:

[Dunbal]

4) Fly safe. Carry superglue.

      You know, I shudder to think about what could happen in a cockpit with an open tube of superglue at 9 G's...

Now we know why they won't sell these to Oz

[ross.w]

They wouldn't have made it here. I knew there was a good reason.

Aero Glass

[lord_sarpedon]

What's more, one pilot tried to turn off Aero Glass -- suddenly, he lost cabin pressure.

Re:Overflow

[Anonymous Coward]

The problem probably isn't with the time change. Airplanes use GMT so the local time doesn't matter. The problem is probably related to the longitude going from W179.99 degrees to E180 degrees.

Position problems more likely

[EmbeddedJanitor]

It is amazing how much navigation software does not handle the East/West rollover properly. Having international development/testing teams scattered over the whole globe sure helps.

If you're going to write software like this, then test it or simulate it at all the wierd places in the world: date line [East/West rollover], equator [north/south chnange], GMT+13 hours [NZ daylight saving time].

Re:

[im_thatoneguy]

That could be the dumbest idea I've heard all day.

So... a program that's in danger of being cut back intentionally causes a significant failure! Why not just submit a proposal to cancel the program? These are not the headlines LM wants right now. When lots of money has been spent, people irrationally expect perfection. Flying to Japan participating in exercises and kicking ass would have gone much further to proving the program viability than creating false doubts of reliability!

Re:

[OECD]

That sound you just heard, the one right over your head? It was not an F-22 returning to base.

Not the first time this bug has shown up.

[sbaker]

You'd think they'd have learned from this one:

        http://www.f20a.com/f20ins.htm [f20a.com]

F-16 had a similar bug

[Stephen Samuel]

Luckily they found it during simulations of the F-16. A bug in the fly-by-wire software caused the plane to think that it was upside-down whenever it crossed the equator [google.ca]. It would try to correct the problem immediately -- A maneuver that the plane could probably survive, but that would probably kill the pilot had it occured in real life.

Fixed

[daveschroeder]

Well, whatever the issue - which is probably something similar to what you suspect - it's now fixed. Here's the transcript from CNN this morning. Since the F-22 is fly-by-wire, it's also worth pointing out that all systems didn't crash, else these F-22s would be sitting in the Pacific. I've no doubt it affected navigation, communications, and similar subsystems, and was probably related to physical location in terms of time, position over the Earth, or both, given the nature of the issue.

>> 25 Years from development to deployment, the F-22 Raptor is the most advanced fighting machine in the air. It was no match for a computer glitch that left six of them high above the pacific ocean, deaf, dumb, and blind as they headed to their first deployment. So what happened? We turn to a man that's at home in the cockpit. Retired Air Force General Don Shepperd. Let me set the scene, Don. These F-22s, headed from the Air Force base in Hawaii to an Air Force base in Japan. They were approaching the international date line, pick it up from there.

>> You got it right. You want everything to go right with the frontline fighter. $125, 135 Million a copy. The F-22 raptor is our frontline fighter, air defense, air superiority, and it can drop bombs. It is stealthy and fast. You want it to go right. On the international deployment to the pacific, it didn't. At the international date line, whoops. All systems dumped. When i say all systems I mean all systems, navigation, part of the communications, fuel systems, and they were -- they could have been in real trouble. They were with their tankers. The tankers -- tried to reset their systems. Couldn't get them reset. Tankers brought them back to Hawaii. This could have been real serious. Certainly could have been real serious if the weather had been bad. Turned out okay. Fixed in 48 hours. It was a computer glitch in the millions of lines of code; somebody made an error in a couple lines of the code and everything goes.

>> This is almost like the feared Y2K problem that happened to these aircraft. We should point out, the computer problems in 2000. The computers absolutely went absolutely haywire and became useless?

>> Absolutely. When you think of airplanes from the old days, with cables and that type of thing and connects between the sticks and the yokes and the controls -- not that way anymore. Everything is by computer. When your computers go the airplanes go. You have multiple systems. When they all dump at the same time, you can be in real trouble. Luckily this turned out okay.

>> What would have happened if these brand-new $120 million F-22s had been going into battle?

>> You would have been in real trouble in the middle of combat. The good thing is we found this out. Any time -- before, you know, before we get into combat with an airplane like this. Any time you introduce a new airplane, you are going to find glitches, and you are going to find things that go wrong. It happens in our civilian airliners. You don't hear much about it. These things absolutely happen. And luckily had time we found out about it before combat. We got it fixed with tiger teams in about 48 hours and the airplanes were flying again, and completed the deployment. This could have been real serious in combat.

>> You had these advanced air -- not just superiority but air supremacy fighters in there, up there in the air, above the Pacific Ocean, not much more sophisticated than a Cessna 152 with a jet engine?

>> You got it. They are on a 15-hour flight from Hawaii to Okinawa. When all their systems dumped, they needed help. Had they gotten separated from their tankers or weather gotten bad they had no reference and no communications or navigation. They would have turned around and could have found the Hawaiian Islands. If the weather had been bad on approach there could have been real trouble. You get refueling from your tankers and you don't run -- you don't get yourself where you run out of fuel. You

Re:Fixed

[rbanffy]

It is the fact of life. In the modern computer age.

It sure doesn't need to be like that.

Our desktop computers crash because we can tolerate crashes. There is some redundancy - if my notebook crashes, I reboot it and, in a couple minutes, I am back to work. If it breaks, I grab another computer and continue.

A plane, on the other hand, should work at all times. When lives depend on some equipment, one should enforce much higher standards than we do on desktop or even mission-critical busines software. Nobody dies if your sales people have a 5 minute outage. Nobody dies if you can't create a patient record. People die when the computers a plane relies upon fail.

It's completely unacceptable - and quite alarming - to see a plane malfunction like that on its first deployment.

Things like that should have been exercised years ago. By now, the code should be rock-stable. Whant kind of quality assurance they did?

Re:Fixed

[0100010001010011]

And even more people die when all systems work perfectly.

The post-incident report

[DingerX]

His comments are based on a post-incident report that's been making the rounds on teh intardnet. I'll just paste it in here, if anybody's still reading. I don't vouch for its authority, other than A) I got it off the net, and B) it came with a note saying it was unclassified. Oh yeah, and it matches what the talking head says -- the navigation system brought down all their avionics. it also states what the QA process was that led to the problem:

Date: 12 Feb 07

To: CC

Info: CV, DS

Narrative:


1. A 1st Fighter Wing AEF 6-ship (Petro 91) departed Hickam AFB enroute to AEF location on 10 Feb. Approximately 4 hours into the mission and coincidental with crossing over the International Date Line, all six aircraft experienced a significant avionics failure including:

Both GINS 1 and 2 Fail

FLCS Degrade

Radar Fail

Fuel Degrade

Loss of all attitude references

Loss of Flight Path marker

Loss of all navigation aides (TACAN, ILS, Computed, etc.)

Loss of all heading indications

2. Aircraft communications were available via backup radio only. Only navigation available was via cockpit airspeed and altitude indications (both deemed accurate). All other aircraft systems, to include engines, electrical system and air refueling, were nominal.

3. Flight Lead, Lt Col Tolliver, initiated via the tanker a CONFERENCE HOTEL (CH) call with LM Aero. All CH team recommended workarounds (avionics restarts, date and time resets, etc.) did not resolve the problem.

4. Lt Col Tolliver assessed pressing to the AEF location but decided to turn back and return to Hickam. He also directed the second deployment cell, a 2-ship approximately one hour behind him, to return to Hickam. NOTE: This 2-ship never crossed the International Date Line.

5. Enroute back to Hickam, after crossing back over the International Date Line, avionics restarts were unsuccessfully attempted.

6. All aircraft successfully recovered at Hickam, shut down (cold iron), restarted engines and all avionics malfunctions cleared.

7. An F-22 Crisis Management Team (CMT) has convened. Two telecoms (1300 and 1700 EST) were conducted on 11 Feb. Participants included F-22 Program Office, LM, Boeing, NG and A8F personnel.

8. The F-22 Program is working 24/7 to resolve this issue. Both F-22 avionics integration labs (RAIL and AIL) have successfully duplicated the problem. The problem resides within the GINS software when the aircraft transitions between East/West Longitude. NOTE: Most RAIL and AIL testing simulate GINS inputs and past testing discovered no issues with over flying the Dateline or Poles. It took testing this weekend using actual GINS hardware and software to duplicate this problem.

9. A fix for this software problem has been developed at NG and currently is being evaluated in the RAIL. We should find out at our 1300 CMT telecom today if this fix works.

10. This fix will require an OFP update to be loaded on the aircraft. Currently no IMIS OFP loading support is on-site at Hickam. 1 FW IMIS was previously deployed to AEF location.

11. F-22 Program currently expects software fix, OFP loading hardware and LM support team in place at Hickam by mid-week. Aircraft possibly will be able to depart Hickam for their AEF location by the end of the week.

12. Updates to this issue will be provided as additional information becomes available.
Translation: The navigational system (Global Positioning Inertial Navigation Systems (GINS)) had never been physically tested crossing the date line, but only on simulated real-world inputs. When it crossed the date line for the first time, it crashed, as did the backup, bringing down with it all navigational systems and much of the aircraft's instrumentation, leaving them with backup systems reminiscent of a Cessna 172 (without the navigational stack).

Re:

[nicknack]

in other news: Lockheed Martin has decided to open it's sources and announce a bug-athon
check out https://repo.airforge.gov/raptor/ [airforge.gov]

but don't everyone look at once - you'll slashdot the server.

Moderation?

[Anonymous Coward]

Why do you guys give +5 to someone who doesn't know for sure how the date line works, and who merely looked up which SI prefix was small enough to cause a 64-bit overflow? Most likely the bug has to do with overflow in position, not time. Even assuming this has to do with time overflow, modern GPS electronics can only measure signals to within 10 nanosecond. Using femtoseconds (10,000,000x smaller) is complete BS to make his argument work.

Re:Moderation?

[tigga]

Why do you guys give +5 to someone who doesn't know for sure how the date line works .. ?

Because it's a Slashdot!

Dumb vote for stupid ;)

Re:Overflow

[Sinical]

Please stop. No one is using femtoseconds for uptime.

Something more reasonable is that the nav system (presumably GPS) didn't like having the date change after aquisition. You'd think that'd be a fairly normal thing to have happen, but after the horrible crap I've seen happen with Rockwell Collins' receivers (they SUCK), it wouldn't be too surprising.

To expand on the Rockwell Collins (they SUCK) theme, we eventually got them to admit to us how to retrieve their diagnostic info, including a register that counting up floating point exceptions (yay, divide by zero!). It had well and truly saturated. On a test flight of an, in part, GPS-guided missile, it once croaked right at launch. Since we never understood that we were moving, we never turned on the autopilot. However, rocket motors don't have much in the way of an off switch, so away we went without autopilot. Boink!

So there are plenty of ways for nav systems to suck (especially if they are made by Rockwell Collins (they SUCK)) without needing something completely stupid like measuring data in femtoseconds.

Hold up, I got a few more of these:

Rockwell Collins (they SUCK)
Rockwell Collins (they SUCK)
Rockwell Collins (they SUCK)

That is all.

Re:Overflow

[El Torico]

How do you really feel about Rockwell Collins?

I have worked on Commecial and DoD avionics

[EMB Numbers]

I have worked on Commercial and DoD avionics, and this type of thing is inexcusable.

Commercial avionics software of the sort described is governed by a standard called DO-178B level A or level B. The process is so rigorous that the slogan is "no-one has ever died from software failure in a commercial airliner, yet." DO-178B level A is expensive. It is virtually impossible that a software error of the nature described could get into a certified aircraft.

Having said that, the military is not obliged to follow commercial standards, but there is a trend toward using DO 178-B in military systems in part because the Europeans are starting to require commercial JAA/FAA certification for all aircraft that enter their air space. But even in the more lax military world, every line of code is typically formally reviewed and there are independent testers. The type of error described should have shown up in simulators before the first flight of the aircraft. Test flights should have stimulated the error long before a squadron ever attempted a transpacific flight.

Even worse still, avionics systems are supposed to be isolated from each other. Navigation radios typically share nothing but power with GPS or with engine instruments etc. Great effort prevents one system from disturbing the power of another too. Aircraft typically have two or more separate primary navigation systems plus inertial guidance and old fashion compass + baring/vector navigation. Military aircraft need to survive both normal equipment failures and battle damage. Military radios (including navigation) need to be isolated from other systems for security reasons too. Those NSA guarded encryption systems can not be contaminated by software that has lower security classification (like navigation)without somebody going to federal prison for a long time.

The bottom line is that something very very wrong, negligent, and illegal needed to happen for the described error mode to manifest. That makes me doubt the story.

Re:I have worked on Commecial and DoD avionics

[nonsequitor]

I too have worked as a contractor doing avionics work. What really surprises me in all of this is that there was no hardware watchdog, or way to reset the box on the way back. I used to work on multi-function displays, ADIs, HSIs, TCAS, etc... The adage goes that no information is better than old information so after going blank, it should have come back up in less than a minute. The fact that the failure state entered by crossing the dateline was persistent after a reboot is criminal negligence, these are people's lives here. Pilots have breakers for everything, they would have cut power and restored it after exhausting all other options, the fact it still was not operational says a lot, none of it good.

Re:Cost Efficiency: EuroFighter vs. F-22

[Rakishi]

The F-22 is a full stealth fighter, the EuroFighter is not in any way. If you cannot understand why you cannot base a stealth plane design in any major way on something that is not a stealth plane (hell, no stealth fighter has existed before the F-22) then why are you eve talking about this as you clearly have no idea about anything involved.

And yes the F-22 is likely worth 84% more than the Eurofighter in terms of performance due to stealth alone.

Incidentally since the F-22 is what the F-35 is based on that $70billion has technically led to the creation of two planes, the later of which is being sold quite widely.

Re:Cost Efficiency: EuroFighter vs. F-22

[Anonymous Coward]

And yes the F-22 is likely worth 84% more than the Eurofighter in terms of performance due to stealth alone.

Only if stealth is a requirement. In a real dogfight, the Eurofighter likely wins because maneuverability was foremost in its design, whereas the F-22 has stealth as the foremost design priority. The thought is that engagements are likely to be fought a distance with missles, and the low observability tech will allow the American aircraft to engage long before the enemy can return fire. This does not jive entirely with engagements of the past, which often involve close range encounters to verify enemy, or orders to wait until fired upon to return fire.

Compare this to the ability to put twice as many aircraft in the sky, carrying more munitions (while the F-22 has some stealty weapons bays, maxed out with a full bomb load involves external mounts with has a huge impact on radar visibility). Point is, whether stealth is worth 84% more has more to do with your mission profile and expected enemy/target,

Re:

[richdun]

The F-22 was built to shoot down anything in the sky for the next 20 years - period. It's the most maneuverable machine known to man and can do maneuvers that were physically impossible before it was built and tested. Its vectored thrust is ridiculous - the thing can fly at something like 25 degrees from vertical without stalling. Its stealth beats the B-2 and F-117A by a generational advance. Its avionics and radar can pinpoint targets outside the range of most missiles. You don't need to put more in the s

Re:Cost Efficiency: EuroFighter vs. F-22

[voidptr]

If you'd actually read the article you linked to.. The F- designation on the F-117 is a curious bit of aviation history and Air Force infighting, but the F-117 is a ground attack aircraft, not a fighter, and should really have an A- or B- designation, while the F-22 is an air to air combat plane with limited ground attack capabilities. The 117's internal payload capacity is huge compared to the F-22's ground attack loads (some of which have to be carried outside, destroying the stealth capability) and it's therefore unlikely the F-22 is going to completely replace the F-117 completely anytime soon.

Re:Cost Efficiency: EuroFighter vs. F-22

[LibertineR]

You might consider that the F-22 makes the Eurofighter look silly in 1v1 dogfights, is faster though both aircraft have supercruise, flies higher, and has that little thing called stealth that allows it to attack heavily defended targets, and close to attack range and shoot other aircraft before being seen. It has greater range, higher operating altitude, better situational awareness for the pilot, monster engines, and did I mention STEALTH?

The return on investment is HEAVILY in favor of the F-22. There is no aircraft anywhere even close. The Eurofighter is the second best fighter aircraft ever built, but it is miles from being in the same class as the F-22 Raptor.

2 EuroFighters > 1 F-22

[reporter]

The price of a single F-22 [afa.org] is about $100 million. The price of a single EuroFighter [nationalde...gazine.org] is about $50 million. So, you could buy 2 EuroFighters for the price of a single F-22.

Here is an interesting question.

In a fight between 1 F-22 and 2 EuroFighters, who would prevail? If the F-22 prevails, then the F-22 is an excellent investment.

However, the United States Air Force has never claimed that 1 F-22 can beat 2 EuroFighters. I suspect that the 2 EuroFighters would reduce the 1 F-22 into a pile of smoking

Re:2 EuroFighters 1 F-22

[DikSeaCup]

Can you be sure of that? I saw something on Discovery (or was it History?) where they were interviewing F22 pilots, and they were mentioning flying against F-15s. They said that it really didn't get all that challenging until it was at least 4 on 1. And even then, sometimes it wasn't a big deal.

Mind you, the EuroFighter may greatly outclass an F-15.

Re:2 EuroFighters 1 F-22

[im_thatoneguy]

Actually they're significantly better than the Eurofighters.

Let's look at a few simple theoretical examples.

You're flying into heavily armed enemy space at night:

      - You fly in 100 Eurofighters. Your enemy has 1000 missiles. You lose 100 Eurofighters
          and hit no targets.
      - You fly in 1 F-22. Your enemy has 1000 missiles, they never detect you. You hit your
          target and leave enemy airspace.

        In this case the F-22 was better than 100 Eurofighters.

        -You're flying alone into enemy territory. You spot a flight of 3 Eurofighters flying in
        formation. You fall into a following position on their tail. You fire 3 missiles
        simultaneously and before the enemy pilots can react. They're dead.

In the Alaskan trials the F-22s ammased 144 kills to 0 losses. That's a pretty good investment. And while they weren't flying against Eurofighters, I'm not sure it would have helped. It doesn't come down to who can turn twice as fast. It's who can fight twice as smart. During this same combat exercise Raptors engaged enemy forces out numbered 4-1 and stil came out victorious.

In previous exercises a single pilot was able to engage 9 enemy fighters, and then ran out of targets, but still had some ammunition remaining. What's most impressive is the ability for the F-22 to multiply the effectiveness of the existing airforce. In the same engagement that F-22 enabled a supporting flight of older aircraft to achieve a kill/loss ratio of 83-1.

Re:2 EuroFighters 1 F-22

[PPGMD]

You fire 3 missiles simultaneously and before the enemy pilots can react. They're dead.

Not quite the Eurofighters likely have RWRs so if you are using radar guided missiles they will likely detect your search, and targeting radars. So even with the newer harder to detect radars installed on the F-22 there is still a chance that they detect you from your radar emissions.

The F-22 is a fantastic aircraft, and is the best aircraft flying, but it isn't a perfect aircraft, and it doesn't have the capabilities that some people exaggerate it having. The Alaskan trails were set up by the fighter mafia at the Pentagon trying to justify their decisions in trying to keep the F-22 orders as high as possible.

It's not the first time that they have done this, during the training maneuvers against against the Indian Air Force they sent outdated aircraft and crippled the ROE and engagement envelopes of the AIM-120s. While the IAF didn't have such restrictions, at least none that we know of.

Re:

[ReTay]

Nobody says here that there exist anti-stealth technologies on going which could eventually make all the stealthy worth nothing.

Right as of yet no one has figured out a way to beat stealth.
When they do the bar will be raised.
Until then the F22 is invisible to radar.
And as far as a missile failing fine the pilot of the F-22 just slides in below and behind and hits with a simple AIM-9 from it optimum position.
Kill ratio 90% or better when fired like this.
And even if by some chance it misses or fails you have

BBC Report on EuroFighter & EuroFighter Beats

[reporter]

The BBC reported on the status of the EuroFighter in a report in 2006 August [bbc.co.uk]. "This [F-22] is very stealthy but costs twice the price of the Eurofighter, and reports suggest that RAF's Eurofighters have flown highly successful missions against the F-22 during recent exercises in the US."

A Scottish report [scotsman.com] describes a dogfight of 1 EuroFighter against 2 F-15s. The EuroFighter reduced both F-15s to smoking rubble.

Based on these reports, we can surmise that the EuroFighter substantially outclasses an F-15

Re:

[Goonie]

The BBC report is probably wrong [f-16.net] There's no evidence to suggest that the Raptor has gone up against the F-22 in exercises yet.

If the Eurofighter's radar can't detect the F-22, multiple Eurofighters won't be any great advantage compared to one.

Re:

[hachete]

I smell a lot of "likely" coming off this thread - mostly from Amurricans justifying the F22.

Has anyone seen the results of exercises btn the F22 and Eurofighter? I thought not. Most of the combat exercises people have mentioned have been btn F15s and F22s, and even then under test conditions. Give it a more agile opponent, the F16 or a more modern opponent, and a mixed-mode operation.

Remember, expensive is not always better, specs don't always relate to combat. Interesting that the Eurofighter's turning ci

Re:Overflow

[Dun Malg]

just as well it wasn't in the fire a [missile] flag byte ... or worse still, the ejector seat flag byte.

Yeah... uh... you know, in the same way they simply do not and never would have the navigation system connected to the In-Flight Entertainment system in an airliner [slashdot.org], likewise they would never slave the ejection system to anything other than the mechanical operation of that yellow handle between the pilot's knees.

As for missiles? First, they fly unarmed on ferry missions because ammo is dead weight that reduces range; and second, even if they were armed, what do you really think would happen if an AMRAAM missile was free launched without being turned on, much less having had targeting info downloaded? Drop like a stone, it would, right into the pacific. Bloop. All gone.

Say it's also a good thing water isn't flammable, otherwise fire trucks would show up to fires and only make the situation worse, right?

Re:Overflow

[Gospodin]

As for missiles? First, they fly unarmed on ferry missions because ammo is dead weight that reduces range; and second, even if they were armed, what do you really think would happen if an AMRAAM missile was free launched without being turned on, much less having had targeting info downloaded? Drop like a stone, it would, right into the pacific. Bloop. All gone.

So you say. But if you think sharks with frickin' lasers on their heads are scary, imagine sharks with fricking' AMRAAMs.

Re:first post to say..

[alx5000]

Sure, but it seems they turned Aero off..

Re:

[kabloom]

Yes. When Jews cross the dateline, we don't [star-k.org] know [koltorah.org] what day it is anymore, much to the consternation of those of us who may need to travel to Japan or Hawaii.

Re:I would not want to be him.

[Simon Garlick]

Someone is going to get fired for this.

Welcome to defense contracting, you must be new here.

Re:

[uncleFester]

from tfa...

Lockheed Martin is rushing a software fix to Hawaii after 12 US Air Force F-22A Raptors en route to Japan for the stealth fighter's first overseas deployment had to turn back because an unspecified problem with their navigation systems.

well, THAT patch hasn't had much time for a burn-in/test period. how comfy would you feel flying with that in place?

-r (*shudder*)