7219548
story
Comments: 291 +- IT: English Shell Code Could Make Security Harder on Monday November 23, @08:33PM
Posted
by
ScuttleMonkey
on Monday November 23, @08:33PM
from the little-bobby-tables-takes-up-writing dept.
from the little-bobby-tables-takes-up-writing dept.
background: url(//a.fsdn.com/sd/topics/topicsecurity.gif); width:59px; height:78px;
security
background: url(//a.fsdn.com/sd/topics/topicprogramming.gif); width:80px; height:48px;
programming
An anonymous reader writes to tell us that finding malicious code might have just become a little harder. Last week at the ACM Conference on Computer and Communications Security, security researchers Joshua Mason, Sam Small, Fabian Monrose, and Greg MacManus presented a method they developed to generate English shell code [PDF]. Using content from Wikipedia and other public works to train their engine, they convert arbitrary x86 shell code into sentences that read like spam, but are natively executable. "In this paper we revisit the assumption that shell code need be fundamentally different in structure than non-executable data. Specifically, we elucidate how one can use natural language generation techniques to produce shell code that is superficially similar to English prose. We argue that this new development poses significant challenges for in-line payload-based inspection (and emulation) as a defensive measure, and also highlights the need for designing more efficient techniques for preventing shell code injection attacks altogether."
Related Stories
Technology: Arbitrary Code Execution With "ldd" 184 comments
Submission: English Shellcode by Anonymous Coward
A great many people think they are thinking when they are merely
rearranging their prejudices.
-- William James
All trademarks and copyrights on this page are owned by their respective owners. Comments are owned by the Poster. The Rest © 1997-2010 Geeknet, Inc.
This is (Score:4, Funny)
quite terrifying :(
If hackers convert arbitrary x86 shell code into sentences that read like spam, but are natively executable .. we're all screwed :(
We'll either need to tighten up how architectures execute instructions to make it harder to execute shell code in the first place.. or come up with sophisticated AI to help filter out the shell code. Of course, as soon as we do that, hackers will develop AIs which can write convincing (and even compelling) shell code.. and THEN what the hell do we do.
Now where I live you can get a pretty decent hair cut for $17 (they even trim up the beard). You can't get anything fancy.. but a decent, professional-ish type haircut is definitely no problem.
My employer is giving us a pretty generous Christmas vacation.. really looking forward to that!!
Also this time of year is great cause CHRISTMAS is everywhere :D
Re:This is (Score:5, Informative)
Guess you missed their "compromised" machine assumption. "..After successful exploitation of a software vulnerability, we assume that a pointer to the shellcode..." . The sky is not really falling any faster today than it was yesterday.
Parent
Re:This is (Score:5, Informative)
Pinning down terminology use by security researchers is tricky.
In this case, what they mean is that the system has a vulnerability that enables code from a remote source to be executed, and that the input from the remote source is being run through a filter that attempt to identify executable code (in order to block it) versus English text.
On an already-secure system, this makes no difference at all. Those don't exist, much. If you were relying on a "looks like executable code" filter to protect you, this is a tip that it's not that secure. The paranoid should already assume so (based on things that already are available in Metasploit, if nothing else).
Parent
Re:This is (Score:5, Insightful)
As is being argued all the time: security is about layers. Layer upon layer. One layer to prevent executable code to reach your system in the first place by looking at the content of a message. Another layer to prevent code that does reach your system to be executed at all. Another layer to prevent untrusted code that does manage to be executed to do any damage (sandbox, permissions). Relying on a single layer of defense is not secure, no matter what that layer is or how strong that layer is. Breach that one layer and you're in.
This research gives at the very least a proof-of-concept on how to breach that first layer of security. And that of course is significant.
Of course there are no 100% secure systems - but the more layers of defense, the more secure it becomes. This takes away one layer of defense, thus making a system less secure. So yes it does make a difference even on "already-secure" systems.
Parent
Re: (Score:3, Insightful)
Binaries that opt out of NX (Score:3, Informative)
Isn't this what NX is supposed to stop, execution of arbitrary data as code?
Then you compromise a binary that has opted out of strict NX, such as a Java virtual machine that needs to dynamically recompile JVM bytecode to x86 bytecode.
Re: (Score:3, Interesting)
Unfortunately, this does not fully solve the problem. Say, for instance, that you've managed to get a buffer overflow on a system, and you now have control over the stack (which is marked RW, but not X). Then, you overwrite the return address of the current function to mprotect() and stick some arguments on it which change the stack protection to RX (there are good reasons for doing this in actual practice, e.g. executable compressors like UPX, or executable thunks on the stack); this type of attack is know
Re:This is (Score:4, Insightful)
Even better: inputs that can overwrite the stack can perform arbitrary code execution even if the stack is never executable, via "return-to-libc" programming.
Parent
Re:This is (Score:4, Funny)
I beleive you missed the virus he just sent you. :)
Parent
Re: (Score:3, Insightful)
Well then that won't be the x86 instruction set, will it?
Re: (Score:3, Insightful)
If you've got the ability to market a processor that won't run peoples old software, and using it makes software slower, take up more memory (think for single byte instructions, a single byte of padding is doubling the space it takes up, which is in effect halving the size of your L1/L2 caches), to a level sufficient enough to get people to actually buy it, then you may as well not even bother with the CPU, just convince them to give you money for nothing, as obviously your marketing team are that good that
Thanks (Score:4, Informative)
What is "shell code" supposed to be? Bourne shell scripts?
Someone had to ask it!
From the wikipedia [wikipedia.org]: In computer security, a shellcode is a small piece of code used as the payload in the exploitation of a software vulnerability. It is called "shellcode" because it typically starts a command shell from which the attacker can control the compromised machine. Shellcode is commonly written in machine code, but any piece of code that performs a similar task can be called shellcode. Because the function of a payload is not limited to merely spawning a shell, some have suggested that the name shellcode is insufficient.[1] However, attempts at replacing the term have not gained wide acceptance.
So it's a poor piece of new terminology that has stuck, unfortunately.
Parent
Oh great - that love letter from the IRS (Score:4, Funny)
This very comment (Score:5, Funny)
Re:This very comment (Score:5, Funny)
Parent
OMG! (Score:5, Funny)
Re:OMG! (Score:5, Funny)
Leave the bible out of this!
Parent
Re:OMG! (Score:4, Interesting)
You joke but what is a meme (religions are "memes") really other than a self replicating piece of language? The *extreme* bits act in many ways like a virus does: self replication, performing specific tasks, adapting to their environment (like some of the more insidious malware) and neither viruses nor memes can replicate on their own; they need a "host."
Parent
Re:OMG! (Score:5, Funny)
So now that you've explained my joke, do you get it?
Parent
Re:OMG! (Score:4, Funny)
Yes, its' a simple head code. Any English schoolboy could catch it.
Parent
Antelope museum (Score:5, Funny)
Consume more trains, Elvis! He, and snorkels, drink elephant's sock puppet master. Steamed cabbage can reverse big piles of ducks. Additionally, cheese log cabin nightmare.
You're screwed now, x86 suckas!
Re:Antelope museum (Score:5, Informative)
The bold characters are code. The rest have no net effect.
Their strategy is to break the exploit into two pieces, a small executable decoder, and the payload. As you might imagine, the decoder decodes the payload. The payload is encoded in a benign-looking format which is simple enough. Their goal was make the decoder also look like benign data. To achieve that, their tool takes an existing decoder and automatically converts it to English-looking prose like the paragraph above. The tool is able to convert a decoder is less than an hour on commodity hardware.
Parent
I CAN BE PLAYED ON RECORD PLAYER X (Score:3, Insightful)
Let the T-C wars continue!
So what? (Score:3, Interesting)
Linux version (Score:5, Funny)
"Please type the following on your command-line:
rm -rf *
Thank you."
Excellent Presentation (Score:5, Informative)
This talk was probably my favorite at CCS this year. Unlike MANY researchers, the lead author of this paper was quite entertaining. Regarding the work itself, there are a few details that the current discussion has missed.
First, I would not say that they can convert arbitrary shell code to English-like prose. Rather, the only instructions that can be used are the ones that are identical to the ASCII encoding of the alphabet. For instance, the ASCII encoding of the letter "r" is identical to the binary for the unconditional jmp instruction. Granted, the authors showed that you can do a lot with this limited set of instructions, but I still wouldn't call it arbitrary.
Second, he showed several examples of the sentences created. They make about as much sense as "Lorem ipsum dolor sit amet..." The tight constraints on the instructions that can be encoded into ASCII make crafting decent English syntax nearly impossible. Spam filters based on natural language processing could probably detect and flag them.
While disguising the binary as ASCII is cool, I don't see that it's all that different than other exploits. Once a sentence containing an exploit is detected, you'll have signatures just like any other type of virus/trojan. I highly doubt that contemporary anti-virus scanners stop working on data that looks like ASCII. Rather, they look for tell-tale signs of particular instructions that appear in particular orders, etc.
And, as many others have pointed out, this code is only harmful if it is executed in the right context (i.e., you have a vulnerability to exploit). Disguising the code as ASCII doesn't really make it different than any other type of zero-day attack.
This work was very sophisticated, and there's no way that script kiddies could build something like this. I don't know that more advanced attackers would bother, because I really don't see all that much of a payoff given the amount of work that this attack requires. It's a whole lot easier to take over a vulnerable web server and launch a XSS attack. The incentives simply do not seem to suggest that this technique will become widespread.
So, no, I don't think the sky is falling because of this attack. Having said that, though, this was a very cool piece of work.
Re: (Score:3, Informative)
First, I would not say that they can convert arbitrary shell code to English-like prose. Rather, the only instructions that can be used are the ones that are identical to the ASCII encoding of the alphabet. For instance, the ASCII encoding of the letter "r" is identical to the binary for the unconditional jmp instruction. Granted, the authors showed that you can do a lot with this limited set of instructions, but I still wouldn't call it arbitrary.
According to the PDF it does convert arbitrary shell code. FTA: What follows is a brief description of the method we have developed for encoding arbitrary shellcode as English text... It looks like they can encode anything once they have built an English-like decoder (judging by their language and the 3rd figure).
The tight constraints on the instructions that can be encoded into ASCII make crafting decent English syntax nearly impossible. Spam filters based on natural language processing could probably detect and flag them.
If they were sending SPAM... which they aren't.
You have... (Score:3, Funny)
You have
a virus
Didn't you know?
You shouldn't be
running Windows
Burma Shave
Hello, World! (Score:3, Insightful)
There is a major center of economic activity, such as Star Trek, including The Ed Sullivan Show. The former Soviet Union. International organization participation Asian Development Bank, established in the United States Drug Enforcement Administration, and the Palestinian territories, the International Telecommunication Union, the result of the collapse of large portions of the three provinces to have a syntax which can be found in the case of Canada and the UK, for the carriage of goods were no doubt first considered by the British, and the government, and the Soviet Union operated on the basis that they were the US Navys interpretation of the state to which he was subsequently influenced by the new government was established in 1951, when the new constitution approved it you King, he now had the higher than that the M.G.u, and soul shouters like Diane. There's a mama maggot including the major justifications that the test led to his own. This is usually prepared by the infection of the Sinai to the back and the Star Destroyers in the parliament, by the speed of these books and the revival of environmental problems of their new Arab states of the Arctic as a more and they possess power to the effort she was especially valuable as the Union and that would have said, as to note that the goods, which the night that if ever I rode after the word Father upon His Church to claim that the peace that had permitted him the city are as a hand of one into I thought of Mr. Crow and the Jews by the days of the C.Cs front garden which had first to St Cyriacus. All of a theology in the setting in a human heart as the tale of this day. I have it to friendship and the States that the way the English of the St Lawrence seven miles of an adjutant...
Now, would you have guessed that this is executable machine code (shellcode)? Honestly, it looks more like the garbage that spammers use to defeat statistical analysis (indeed, this is code generated with a similar goal).
(P.S. this particular sample is merely an amalgamation of the code which was reproduced in the paper; it is not complete, and will therefore not execute).
Re: (Score:3, Informative)
They don't mean shell commands. They mean code that exploits a vulnerability in order to start a shell.
Re:The syntax should not matter.. (Score:5, Insightful)
And nothing in their article is helping with that. They assume they are exploiting a software vulnerability. If I know there is a software vulnerability, there are 1 million and 1 less complex ways for me to blow right by any inline scanner. (One stupid enough not to look and see what the actual bytes were anyway)
Parent
Re: (Score:3, Informative)
It's a research paper, not an exploit, not instructions on how to make an exploit, not recommendations on how to make an exploit. God what's with you people on this site, you can't just see something for what it is, you have to see it for how it serves no purpose to you or how you can do it so much better.
If they could exploit a machine by sending a point across, they'd get it past you lot every time, you'd never detect that huh.
Re: (Score:3, Insightful)
There are indeed times when I think that we built the Internet, and that it taught us only one lesson:
I'm right and you're wrong.
This is not quite as concise as "42". Also, a second Internet will have to be built to determine who is "I" and who is "you".
Re:In other news... (Score:5, Informative)
Good job not reading the article.
It's not that shellcode can be written in text and then compiled to an executable form. It's not that shellcode can be compiled to an intermediary form, translated or compiled into machine instructions by a piece of code (this is common in malware now, to pass input restrictions -- as the article says). It's that the executed machine instructions themselves -- the compiled binary data that can be run raw on an x86 processor -- looks like English text.
Parent
Re: (Score:3, Interesting)
I had brought something like this up during an after-work, Friday night beer session back in the late '80s when a co-worker mentioned the odd snippets of text that one would see while examining programs using the debugger. (No... we weren't talking about strings of text defined in the source code.) I wondered whether it was possible to come up with a program whose machine code formed English text that actually perf
Re: (Score:3, Interesting)
No, it translates assembly to different assembly that's also English. This is actually a rather interesting piece of work. They didn't just write a program that converts assembly to English assembly, they wrote one in English assembly.
Re: (Score:3, Informative)
Technically, machine code -- assembly is the pseudo-English text version of machine code.
But otherwise, yes.
Re:In other news... (Score:4, Informative)
Parent
Re:In other news... (Score:4, Interesting)
An assembler/compiler doesn't necessarily use a high-level language input.
In this instance they (as you say) 'takes as input executable machine code and generates executable machine code with a very narrowly-defined statistical property' which tells me they have an assembler that reads executable code and assembles executable code that looks like English text, in other words an assembler.
Parent
This is far more interesting! (Score:5, Interesting)
I for one is very impressed by what they've done, even if it is somewhat similar to what I did nearly 15 years ago:
At that time I wrote what's probably the "best" executable text encoder for MsDos, it uses the absolute minimum possible amount of self-modification (a single 2-byte Jcc opcode) while staying entirely within the MIME text character set, and survives all the most usual forms of reformatting/reflowing of the text. (Replacing CRLF with a single CR (Mac) or LF (unix) or turning each paragraph into a single line.)
The initial bootstrap looks like this:
ZRYPQIQDYLRQRQRRAQX,2,NPPa,R0Gc,.0Gd,PPu.F2,QX=0+r+E=0=tG0-Ju E=
EE(-(-GNEEEEEEEEEEEEEEEF 5BBEEYQEEEE=DU.COM=======(c)TMathisen95
(The uppercase 'E's are my NOP fillers, they execute as INC BP, a register I don't use.)
Terje
PS. Unlike the current guys, I wrote the code above by hand, on paper, during the evenings of a ski vacation. I had brought with me a listing of the ascii encoding of all instructions that would use MIME characters only. :-)
Parent
Re:This is far more interesting! (Score:4, Interesting)
I know, and that's exactly what's makes it so interesting:
They have effectively defined a small subset of the entire instruction set while allowing all other instructions that doesn't produce a side effect which would crash their "real" code.
Terje
Parent
Re:In other news...BAN THE PARENT (Score:5, Informative)
This is the sixth spam message this user has posted, will SLASHDOT please BAN this guy already? Come on.
Parent
Re:In other news...BAN THE PARENT (Score:5, Informative)
This is the sixth spam message this user has posted, will SLASHDOT please BAN this guy already? Come on.
He must be making new logins. I've seen him posting for a few weeks, he surely has more than 6 spams that I've seen alone. Going on that idea... lets see:
http://slashdot.org/~coolforsale117 [slashdot.org]
http://slashdot.org/~coolforsale116 [slashdot.org]
http://slashdot.org/~coolforsale115 [slashdot.org]
http://slashdot.org/~coolforsale114 [slashdot.org]
http://slashdot.org/~coolforsale112 [slashdot.org]
http://slashdot.org/~coolforsale110 [slashdot.org]
No doubt there is a TON of them. So I'd guess they are banning him, he just keeps making new uids (and siphoning a ton of moderation points to keep him marked at troll / offtopic). I know I've used many mod points keeping this bastard down.
Parent
Re: (Score:3, Funny)
Re: (Score:3, Interesting)
It hope none of you are thinking of subscribing coolforsale's email address zminring@gmail.com to a lot of spam lists.
That would be very wrong.
Very very wrong.
Re: (Score:3, Insightful)
Re: (Score:3, Insightful)
It's a shellcode [wikipedia.org]; it's actually written in machine code.
Re:Confused (Score:5, Informative)
TFA uses the security community's special term "(a) shellcode", which means something other than what it sounds like to ordinary programmers.
"A shellcode" is the infection head of an exploit - the thing you try to get to run on the target to make the rest of the exploit work. It's in the machine language of the target, not a shell language.
It's called "a shellcode" because it typically (but not necessarily) tries to sucker the system into launching a shell to run the rest of the exploit. The rest of the exploit may be in a shell language (depending on the shell to interpret it), a machine language executable, etc. Or "the shellcode" may do something else than launch a shell.
This is one of the latter cases. It's a chunk of self-modifying code (due to the limits of what instructions you can get out of English-looking text) that bootstraps its own internals into something that can act as an interpreter (or other executor) for the rest of the English-looking exploit code, then runs though that code and "makes it happen".
You can think of it as a binary executable program that depends on self-modification to get away with consisting only of combinations of bytes that look enough like English to fool spam filters which are trying to recognize executable code.
So it's a very goofy binary and there are no shells or shell languages involved. Instead (if I read this right) the researchers built a very screwy assembler that takes as input an assembler source program and produces as output some VERY screwy machine code that looks like English and ends up doing the same job in a roundabout way, rather than being the direct translation of the assembler code input.
Parent
Re:Confused (Score:4, Informative)
Nope, you're confusing assembly code and shell/machine code, which are two different things.
Assembly is text-based, and is readable for people who know the language. Each operation is a keyword, and some take arguments. It's basically the lightest-weight possible programming language (although it's not really considered a programming language, it's so light weight!) A computer cannot run assembly code directly.
Machine code is what you get if you take the assembly and run it through an assembler to produce code that the computer can understand. The computer can then execute it. It is not human readable unless you've memorized which opcodes correspond to which assembly keywords. Far easier to pipe it through a disassembler to get the assembly code back and read that.
To answer the GP's question this sounds like they mean shell code. It wouldn't be very useful as assembly code anyway. ("To claim your free iPod, run this sentence through masm and run the resulting EXE file.") Most people don't have an assembler and the ones who do aren't usually susceptible to malware anyway.
Parent
Re:That was rather pretty (Score:4, Informative)
Parent