Cisco Confirms Regex Flaw in IOS

gattaca writes "Cisco has announced a confirmation of an unpatched denial of service vulnerability in Cisco IOS. From the NetPro Forum post: 'I have just discovered a regular expression that crashes the router. I suspect the error is because of division by zero. Since I work for the Enterprise, I do not have direct access to TAC. Please somebody report this to Cisco. I have tested it on ranges of routers (2611, 2821, 2851, 7206) and IOSes (12.0-12.4). All routers crashed with some type of BUS ERROR. Command can be issued in user mode, therefore I think it can be considered as vulnerability to potentially cause DOS.'" Of course, the command has to be entered in user mode, so while potentially a vulnerability, chances are your local IOS-based router won't be DoSed via the bug any time soon.

does it could as denial of service

[Ferzerp]

if your own people have to do it?

Re:

[Ferzerp]

That's supposed to be does it COUNT, not could.

I can't type early on Saturday mornings.

Re:does it could as denial of service

[blantonl]

It only if works you authenticated are router to the.

Re:

[packetmon]

so means that you're not if authenticated router to the can't it do to crash a cause?

Re:

[pacman on prozac]

I think it does, the reason being that your own people could do it accidently.

All it needs is someone to use a back reference or a repetitive match (*) in a regexp and the router could reload.

Re:

[3vi1]

>> All it needs is someone to use a back reference or a repetitive match (*) in a regexp and the router could reload. I've used complex regexp's on hundreds of devices and never seen the problem. Can you please give me an example? And, could it not be so contrived as to force the issue?

Re:

[pacman on prozac]

There is an example in the second link in the article.

Re:

[Xerxes_au]

Fair enough that you mention that it requires a valid login to trigger this bug. Once logged in there's plenty of nasty things you can do to a Cisco router (or any other) without needing to trigger random bugs.

I've worked in the area for a number of years now supporting both Cisco and Linux based network infrastructure, and I can say with some confidence that on a large scale, bugs in various programs which can lead to total loss of service are hardly rare. DoS bugs exist, and while many will just randomly

Re:

[3vi1]

Exactly. This is a non-story. The person first needs access to the device, and even then it's not something that would happen except on purpose.

This bug affects IOS versions all the way back to 12.0. I've used regular expressions on these IOS's every day for years and years and have NEVER had the device reboot on me. You must have to get really complicated with the expression or be back-referencing very large strings in order to crash a box this way, because I never have, nor has anyone at my company (w

Re:

[ddieder]

It's certainly less critical than a remote denial of service bug, but in many cases, it still needs to be looked at pretty carefully.

It's possible that a larger set of staff besides just network engineers have access to different levels of automation. Some of that automation might be able to run commands like this if abused correctly.

Re:

[Cut'n Paste]

Does it count as a denial of service if a *customer* does it *accidentally*?

OK more than a little off topic, but maybe interesting in an historical artifact sort of way...

It was 1976 and I was working on an IBM mainframe for the first time, using TSO (Time Sharing Option) to a local service bureau with hundreds of clients. Typing in my commands when dang, TSO went down and the whole system with it. After about 10 or 20 minutes it came back up, and I re-entered my stuff until dang, down it went again, an

Get off the bus

[The Clockwork Troll]

Nitpick: if it were a division by zero fault, would it really trigger a bus error, or more likely a ... division by zero error?

Then don't do that

[Anonymous Coward]

FTA: "I have just discovered a regular expression that crashes the router. I suspect the error is because of division by zero."

Reminds me of:

Patient: "My arm hurts when I do this." <wiggles arm>
Doctor: "Then don't do that."

The solution is obvious: don't use that regex/divide by zero. Duhhhh. Problem solved. Thank you, come again.

Re:

[Tribbin]

a: What 's the problem?

b: When I press here, here, here or here it hurts.

a: Ah, I see. You finger is broken.

A bigger IOS flaw discovered

[packetmon]

A bigger vulnerability has been discovered just now as well...

r8#sh ver | in IOS
IOS (tm) C2600 Software (C2600-IO3-M), Version 12.2(15)T2, RELEASE SOFTWARE (fc2)
r8#reload
Proceed with reload? [confirm]

Seems like anyone with admin access can reload your router.

IOS (tm) 4500 Software (C4500-A3JK9S-M), Version 12.2(40a), RELEASE SOFTWARE (fc1)
frSwitch#reload
Proceed with reload? [confirm]

Confirmed on multiple routers as well! OMFG. On another note, anyone with local access to the router can power down the router causing a massive denial of service. Our admins here at GoodyTwoShoesNetworking.com are placing epoxy across all power buttons and cables to prevent this

The Enterprise

[AntEater]

"Since I work for the Enterprise, I do not have direct access to TAC. "

Yes, Capt. Kirk can be very protective of the TAC.

Re:

[Anonymous Coward]

I always wonder why a company like Cisco, of which you would expect that it puts quality at a top priority, does not accept bug reports from owners of their devices who have not paid extra for a support contract.
Even when they don't want to guarantee response times or resolution times, at the very least they could register the problems their customers have discovered.

Re:

[k8to]

When I worked at Wind River, which in some ways was an arrogant culture, we routed support issues through from any customer with a defect issue, whether or not they were paying the four thousand a year to be allowed to talk to us. Sometimes these issues involved more work than the paid customers issues, with both sides spending many hours delving into and isolating the problem.

A defect in the product is the responsibility of the manufacturer/developer! Improving the product by removing/resolving defects is

Re:

[Phroggy]

Yeah, I don't buy that.

Call Cisco's TAC, give them your serial number, and tell them you've found a security flaw. Record the conversation, editing out any confidential bits. If they won't listen to you, post the MP3 of the call online. See if that gets any attention.

Re:

[bwindle2]

From the article:

"In addition to that, the Cisco PSIRT Security Vulnerability Policy is available at http://www.cisco.com/en/US/products/products_security_vulnerability_policy.html [cisco.com] - for any customer, with our without a service contract, which might be interested in contacting us.
Thanks,
Dario "

Re:

[Cramer]

Depends on the bug. Sometimes they'll listen; sometimes they don't. The real problem comes after they fix the bug. If you're not a customer, you're not likely to get the fix.

(And they really hate people logging bugs that have already been fixed.)

RegEx's are incredibly dangerous

[mosel-saar-ruwer]

Writing code that can parse for any given syntax is, well, pretty much as difficult as writing a parsing front-end to a compiler.

I.e. it is not trivial and it is fraught with danger.

Any time you allow the user to submit arbitrary, un-screened, un-filtered data, you're just asking for trouble.

Of course, I guess you could argue that the job of a RegEx parser is precisely to do the screening & the filtering for you, but it is not a trivial business, and anyone who approaches the problem as though it were a mere triviality is a fool.

I.e. from the security point of view, the RegEx parser is a firewall [and, in all likelihood, is the only firewall], hence anyone writing a RegEx parser has to assume that the user submitting the input is a blackhat, not a whitehat.

PS: And the problem undergoes manifold [if not infinite] complexification when you're dealing with languages [or "environments"] like HTML, Javascript, and XML, which can re-write themselves on the fly.

Re:

[Anonymous Coward]

Is it as dangerous as say... This:

lynx -dump http://linux.slashdot.org/article.pl?sid=04/11/03/0358246|awk '/Shootout/{gsub(/'\''/,"");print $2,$3,$4}'|uniq|sh

I wouldn't run that as root if I were you

Looking Glass

[Anonymous Coward]

There are many routers out there running IOS that are used for Looking Glass purposes, so, yes, this is a problem I guess..

LGs aren't a problem but public route servers are

[macdaddy]

I run a public route server with a web frontend (ie, a looking glass). I have yet to see a LG that allows the use of regular expressions. Some CLI-based route servers allow it but this is easily controlled when you explicitly state what commands a non-privileged user can execute with the 'privilege exec' global config mode command.

That said, I'm on AT&T's route server right now and I can clearly see that it's been abused by the regex bug:

route-server> sh ver
Cisco Internetwork Operating System S

Re:

[Tanman]

/0 does not mess up windows calc. They prepared for the error and it returns "Cannot divide by zero," then lets you continue on your merry way to further calculations.

Re:

[Algorithmnast]

Dividing by zero screws everything up. Even Windows Calc, one of the most advanced pieces of software on the planet, can't do it.

As it happens, I can divide by zero, but only when I try to figure out the inverse of the percentage of well-spent money from my tax dollars.

Or perhaps, the ratio of posts to informational-posts.

After all, Godwin needs revision - to paraphrase "A Beautiful Mind".

Old news (to everyone but Cisco)

[OriginalArlen]

This was widely publicized (amongst the loose communities of Cisco users, anyway) back around the time [secunia.com] the original post [nether.net] was made. Hey, that would have been... 18th August! :)

To be fair, there IS a story here, which is that Cisco only just acknowledged this officially.

Service Provider types (the operators of routers whose successful attack would actually affect anyone in the real world) have been well aware of this. But as others have pointed out, if you don't trust your admins, and you're not running proper logging and a proper audit trail of admin sessions already, you've got bigger problems than this.

A question

[Poromenos1]

Can someone explain to me the difference between a $50 OpenWRT router and a $2k Cisco one? I have both, and the OpenWRT router is by leaps and bounds more featureful than the Cisco one (I guess that doesn't really make sense, because for $20k the Cisco can have the same features). Obviously the difference is reliability/performance, but what are the exact limits? How many people do I have to have in my network before getting a Cisco? How will I know that?

Re:

[bagboy]

Buying Cisco equipment (typically - and through proper purchase channels) gives you access to TAC (Technical Assistance Center), worldwide support 24x7 for network emergencies and problems, as well as extensive troubleshooting support. When you buy Cisco these days, that is what you are mostly paying for. Try getting that level of assistance from Netgear, Linksys (ironically owned by Cisco), DLink, etc... You really can't - as they do not put that level of resource behind their products.

Re:

[Poromenos1]

Really? So they can tell me how to get both ADSL modules to work? I've been trying to do that for a year, do you know whom I can contact? I tried their site's support section but they never replied (which is less than the support I've gotten from DLink or the OpenWRT people).

Re:

[un1xl0ser]

I've only interacted with Cisco support once, for replacing a PIX. Called them up, RMA was delivered shortly thereafter. Try calling them, if you do pay for support.

Re:

[Poromenos1]

I'm in Greece, sadly a phonecall would cost more than the router. Maybe they have headquarters here, I'll look into that, thanks.

Re:

[Anonymous Coward]

Quite a bit. If you look at a standard linksys router, it is a simple Broadcom (or Marvell) CPU+Network processor. Most companies use one of these chips in their consumer routers . They are cheap, and give the features most home users want. (Routing packets, simple firewall, wireless etc.) However, they are not as fast, nor are they flexible as this would add to the cost. (Actually, many other "enterprise" routers/switches use the same $5.00 chips) . Once you need a new feature.. you buy a new router..

Re:

[Spazmania]

Can someone explain to me the difference between a $50 OpenWRT router and a $2k Cisco one?

Answer #1: The latter one can be installed and operated by mere mortals, or at least folks reasonably close to such.

Answer #2: The latter operates effectively within the scope of your existing monitoring and management processes while the former does not.

Answer #3: The latter is targeted at and marketed to companies (not individuals) where Answers 1 and 2 are much more important that the initial acquisition cost.

Re:

[Repossessed]

The biggest thing I see between the expensive Cisco stuff and the cheap WRT stuff (once you flash the firmware, and thank god for the GPL, cause the original stuff sucked), is that the Cisco kit will support large gigabit networks, (And you can get older Cisco branded stuff at Linksys prices that doesn't if you shop the right channels).

Assuming you only need 100Mbit though (which is fine for lower tiered subnets), Well... there are three systems hooked up to my WRT54GL, and it's running at about 10% of ca

Re:A question

[OriginalArlen]

At the low end, there's not a great deal of difference beyond the value of the brand (which is non-zero: how many replies do job ads for "network engineer, min 4 years experience with Linux based routers" get vs. "cisco-based routers"? )

At pretty much anything above the branch office level, however, there's a huge difference. The two biggies are the backplane, and the ability to support proper linecards with offload routing processors. When you have a fat high-end device in your network core with 8 16-way OC3 linecards, there's just no way the standard PC architecture can keep up. The PC architecture jus isn't designed to shift massive amounts of IO, twiddle bits on a zillion and one packets per second, then route them out a different interface.

If your cable runs look like this [tmk.com] then you are not going to be using PC hardware, believe me.

Juniper are a good alternative to Cisco, though. There is now finally some competition.

Re:

[Bigjeff5]

http://seclists.org/politech/2002/Dec/0004.html [seclists.org]

Check that out, that's what your money buys you from Cisco.

Boston's Beth Israel Hospital went down due to a spanning tree protocol loop (caused by a network infrastructure that was improperly patched together). Cisco had a team on-site in under four hours working on the problem. They did a massive re-structuring of the major parts of the network in less than three days. They flew in two of their massive multi-switches to get the network working.

It's a cisco p

Re:

[pyite]

Boston's Beth Israel Hospital went down due to a spanning tree protocol loop (caused by a network infrastructure that was improperly patched together).

And let the lesson for this be never to let spanning tree have to be used in the first place ;-)

Design layer 2 networks such that ports are always in a forwarding state, i.e. a loop free physical topology.

Re:

[Lennie]

I agree layer3 is an easier setup for any failover or whatever else reason you might want more links.

But spanning tree can still be usefull for when someone creates a loop by accident.

But then again, it may stay undetected.

So it depends on what your prefereences are.

not /0 error

[v1]

In case anyone cares, the reboot (or "reload" as cisco likes to call it) is caused by a stack overflow resulting from an uncaught recursive processing of specific combinations of regex options. The overflow must be input from the command line interface, after providing a valid username and password to login to the device. If you are being DOS'd by someone that has a valid login and password on your hardware, you have bigger issues that need dealing with before investigating firmware bugs in your router.

Re:

[Raideen]

I think that the bigger issue is that an admin may create a similar regexp and inadvertently lockup a router. As a security issue, it's pretty minor, but accidental lockups are not a good thing.

Crash a router using CLI access?

[Anonymous Coward]

If a rogue has CLI access to your router, you have bigger issues. Proper filtering, TACACS and Logging, Out of Band Management makes this a non-issue.

The risk is almost the same as "reload" or the even more fun undocumented "test crash" commands.

Granted I do not think this vulnerability requires "enable" access, which does increase the risk. However, nobody should have any CLI to a router that you do not trust.
   

I guess I'm a security researcher then

[twigles]

Since I did a "show buffers all" on a 4948 and it reloaded the box. General rule I follow is that if you have to have root access to do something, it's not a vulnerability. This is just a TAC case/bug fix.

User mode

[bluefoxlucid]

If you can telnet to the router's IP address and it doesn't block you (i.e. if there's any kind of remote administration), you get user exec mode. Good job.

Re:

[Tony Hoyle]

No, you get the login prompt. Good luck finding the password.

Your problem is not in the router.

[Medievalist]

If you can telnet to the router's IP address and it doesn't block you (i.e. if there's any kind of remote administration), you get user exec mode. Good job.

Um, seriously, if that works on your network, you need to fire your Cisco admin immediately.