Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!

 



Forgot your password?
typodupeerror
Security IT

How To Guarantee Malware Detection 410

Posted by CmdrTaco
from the pardon-my-skepticism dept.
itwbennett writes "Dr. Markus Jakobsson, Principal Scientist at PARC, explains how it is possible to guarantee the detection of malware, including zero-day attacks and rootkits and even malware that infected a device before the detection program was installed. The solution comes down to this, says Jakobsson: 'Any program — good or bad — that wants to be active in RAM has no choice but to take up some space in RAM. At least one byte.'"
This discussion has been archived. No new comments can be posted.

How To Guarantee Malware Detection

Comments Filter:
  • At least one byte (Score:3, Insightful)

    by BadAnalogyGuy (945258) <BadAnalogyGuy@gmail.com> on Monday March 15, 2010 @11:21AM (#31483136)

    While it might be true that any application will take up at least a byte of memory, there is no reason malware couldn't masquerade as another binary down to the exact number of bytes.

    Hell, Windows is a whole slew of malware that masquerades as the whole OS.

    • by dmgxmichael (1219692) on Monday March 15, 2010 @11:25AM (#31483232) Homepage
      Not a fair comparison. Malware usually does what it's supposed to.
    • by Chris Burke (6130) on Monday March 15, 2010 @11:36AM (#31483436) Homepage

      While it might be true that any application will take up at least a byte of memory, there is no reason malware couldn't masquerade as another binary down to the exact number of bytes.

      Oh see he didn't finish explaining.

      Any program that wants to be resident has to occupy at least one byte of RAM. And that byte should include the Evil Bit, which all malware should set. Then your anti-virus program just checks the Evil Bit and problem solved!

      • Re: (Score:3, Informative)

        by alexhs (877055)

        You've been modded funny, but this is exactly what this "article" is all about.

        If there is no malware in RAM, the results will be the expected result. [...] Or there could be malware in RAM, and the checksum will be wrong. [...] Or malware could divert the read requests [...] . That would result in the right checksum... but a delay.

        Or, there could be malware in RAM, not diverting read requests, and the checksum will be the expected result, and without a delay.

        The only problem with Markus Jakobsson grand theory is that all malware are of that last kind.
        Well, all malware since the memory protection era. I suppose his "product" could work for DOSes (but there is no multitasking there) Windows 3, MacOS9, AmigaOS and some embedded OSes.
        And if the malware does vi

    • by lgw (121541)

      He pushes the hard work off onto an "external verifier" without making it clear how that would itself remain rootkit free.

      If you have a guaranteed-safe external verifier, malware detection and removal isn't rocket science. Oddly, we have the technololgy in place to create a safe verifier: Trusted Coomputing. All we really need to solve the malware problem for good is something like Trusted Computing that we actually trust! An open standard, untainted by DRM, the provides hardware-level cryptographic isol

      • Re: (Score:3, Informative)

        by causality (777677)

        He pushes the hard work off onto an "external verifier" without making it clear how that would itself remain rootkit free.

        If you have a guaranteed-safe external verifier, malware detection and removal isn't rocket science. Oddly, we have the technololgy in place to create a safe verifier: Trusted Coomputing. All we really need to solve the malware problem for good is something like Trusted Computing that we actually trust! An open standard, untainted by DRM, the provides hardware-level cryptographic isolation between the running system and the verifier.

        I'd rather just secure my systems, thanks.

        • Re: (Score:3, Interesting)

          by lgw (121541)

          I'd rather just secure my systems, thanks.

          How would you do that, then? There are so many clever rootkits these days, and no OS is "immune" (although SE Linux may be so if correctly configured, configuration is hard). Hardware-based rootkits (malware in the firmware of peripherals that attack an already kernel-mode driver process) are very dificult to defend against, you can only really hope they get enough bad press before they get you.

          The only way to "just secure your system" against increasingly clever attackers is to have a safe partition, one

    • Re: (Score:3, Insightful)

      by OeLeWaPpErKe (412765)

      There are several weaknesses. Note how he says something about an external verifier, which is delay-sensitive. Note how for the first 2 steps he keeps repeating "malware may of course interfere but this doesn't matter because", and then he stops considering malware interference. That's because at those points, malware interference would be fatal.

      Of course, malware could simply take over the entire procedure, computing the keyed hash itself (a process which can run in a lot less memory : it doesn't actually

      • by Panaflex (13191)

        Oh, and lets consider that a virus writer would know the basic fundamentals of memory management, reporting back less RAM pages than expected. After all, we're depending on the OS memory allocator to provide correct number of pages, paged/nonpaged, etc..

        In other words, we could alter the VM page table and not report all physical pages to the OS / "Guaranteed Virus Detection"

        Another problem to consider is that often programs are compiled with buffer overflow detection, which writes a pattern of bytes before

  • Theory and Reality (Score:5, Insightful)

    by ircmaxell (1117387) on Monday March 15, 2010 @11:21AM (#31483148) Homepage
    In theory, theory and reality are equivalent. In reality, they are quite different...

    Seriously, how could this possibly work for ALL (including undocumented, and hereto unknown) threats? And if it does it by reading straight from RAM (through the kernel), wouldn't a rootkit be able to trivially defeat that?
    • Agreed - couple this with the reality that at least half and probably more of all malware today is packed in with programs the user installed. I don't know how much BearShare and similar crap I've had to clean off relative's comptuers. This technique does nothing to stop malware of that nature - no antivirus technique known can stop the user from installing programs that do stuff other than what they advertise to do.
    • Re: (Score:3, Insightful)

      by goombah99 (560566)

      His whole point was not "this is how you should do it", it was "you could do this, and because you could do this it shows that it's theoretically possible". This is a variant of what is know as a gedanken experiment-- an argument that proves or disproves some fact is true while not actually being somethign you would want to carry out. For example, you could suppose that you could measure the force field is under by running a pole from the earth to the moon and pushing slightly on it. Not that you want

    • ya. coming up with a reliable virus detection scheme for unknown viruses is pretty much in the same area as the halting problem.

      Even detecting polymorphic viruses has been proven to be NP complete.

    • by n1ywb (555767)
      A rootkit that is AWARE of this detection mechanism ought to be able to defeat it easily by just overwriting the computed and expected keys in the detectors memory space with a random number. No delay, the values are equal, so no problem right? Wrong. The only way to make this really work would be for the detector to have direct hardware access to the machine's RAM but be running on a different uninfected machine. That's theoretically possible, but not really practical with out of the box hardware.
      • A rootkit that is AWARE of this detection mechanism ought to be able to defeat it easily by just overwriting the computed and expected keys in the detectors memory space with a random number.

        LOL, good point.

    • by einhverfr (238914)

      I read the article and I wasn't convinced. I don't think one can guarantee malware detection. Any detection approach has false positives and/or false negatives. Typically we err on the side of false negatives, while some other approaches (host-based IDS-type approaches) err on the side of false positives.

      The method addressed here does not deal with all possible attacks, but only the problem of malware interfering with the scan. Hence even with such a mechanism, all you can use it for is guaranteeing the

    • Not only that, but his initial premise is already wrong! Most people conceptualize a program like an application - it's launched, loads into memory, and then does stuff. And while that's typical, it's a grave mistake to think that's the ONLY way to go!

      Off the top of my head, I can think of registering malware as a callack handler for a system event. In this case, you have an infected computer without any code running at all, in a context and namespace different than running applications!

      Winows just wasn't

  • The hard part is actually finding it.
    • It could be in ROM.

      It could be in processor cache.

      It could be in the video card's memory.

      Could it be in pipelined instructions waiting to be executed?

      • by magsol (1406749)
        I would guess, though, that in order to do something we would typically characterize as "malacious" it would have to eventually wind up in RAM (except in the case of ROM...that would really fubar the system).

        Processor cache would eventually result in execution, which takes place in RAM. Video card memory would just do funky things to the display. A pipelined instruction would also eventually end up in RAM once it was executed.

        Perhaps the key here is "actively executing" malware. I suppose it can lay d
    • Re: (Score:3, Funny)

      by dissy (172727)

      The hard part is actually finding it.

      That reminds me of a signature I've seen around here (Sorry, I don't remember who was using it)

      cat /dev/ram | strings | grep llama
      OMG, my RAM is full of llamas!

  • by fuzzyfuzzyfungus (1223518) on Monday March 15, 2010 @11:24AM (#31483188) Journal
    He is indeed selling something [fatskunk.com]...
  • by Lord Grey (463613) * on Monday March 15, 2010 @11:24AM (#31483198)

    Assume now that we have a detection algorithm that runs in kernel mode, and that swaps out everything in RAM. Everything except itself. Well, malware may interfere, of course, as it often does, and remain in RAM. But if we know how big RAM is, we know how much space should be free. Assume we write pseudo-random bits over all this supposedly free space. Again, a malware agent could refuse to be overwritten. It could store those random bits somewhere else instead... like in secondary storage.

    Then, let us compute a keyed hash of the entire memory contents -- both our detection program and all the random bits. Here is what could happen: If there is no malware in RAM, the results will be the expected result. An external verifier checks this, and tells us that the scanned device is clean. Or there could be malware in RAM, and the checksum will be wrong. The external verifier would notice and conclude that the device must be infected. Or malware could divert the read requests directed at the place it is stored to the place in secondary storage where it stored the random bits meant for the space it occupies. That would result in the right checksum... but a delay. This delay would be detected by the external verifier, which would conclude that the device is infected.

    <sarcasm>Punting the problem to an "external verifier" is pretty neat. I wish I could do that with my next hard problem.</sarcasm>

    That whole bit about swapping, though.... If I write malware and hide it somewhere in execution space, do I really care if it gets swapped out? So the code that steals keystrokes or sniffs for credit card numbers doesn't get executed for short while. Big deal. At some point it will get loaded again (if written properly, that is).

    Or am I missing something obvious?

    • Punting the problem to an "external verifier" may or may not be a cop-out depending on the context.

      If you want it to work, with a software update, on today's general purpose x86 office boxes, your "external verifier" might as well be a magic pony that sneezes rainbows and poops out the factors of any arbitrarily large primes that you feed it. Not Happening.

      On the other hand, if your target is "Paranoid embedded architectures, 2-5 years from now" you can posit pretty substantial hardware changes at onl
    • by taustin (171655)

      This proposal isn't to detect what malware is present, or to remove it. It is onl to detect that there is some malware present, which can then lean to more thorough scanning to detect and remove. Knowing that something is there is half the battle.

      • by TheLink (130905)
        > It is only to detect that there is some malware present,

        No, he claimed in the article that:

        > This tells us a few interesting things. We can guarantee detection of malware.
        > And that includes zero-day attacks and rootkits. We can even guarantee that we will
        > detect malware that infected a device before we installed our detection program.

        To me _guaranteeing_ detection of malware (especially zero-day ) is similar to solving the halting problem (without having the source code and knowing all the p
      • by Lord Grey (463613) *

        Detecting the malware depends on the malware trying to stay in memory. My point was that "properly written" malware wouldn't necessarily care if it is was swapped. Allow the swap, get a clean bill of health from the "external verifier," then get reloaded and continue Bad Activities. Downtime for the malware is negligible.

    • Well, wouldn't this require ALL other processes to "sleep" while the check is performed? Sure, Ram is fast, but writing to 4gb of DDR3 would take around 1/3 of a second (excluding the time it took to generate that data and store the hash) considering the peak transfer rate of DDR3 is around 12800 MB/s (Using the best case)... So in reality, you're looking at well over 1/3 of a second (potentially into the seconds. And that's just for writing. You need to swap everything out first. So the whole process
      • by spun (1352)

        A valid criticism. And if the malware is actively resisting the scan, by moving the random bits back in from secondary storage before the hash, the external verifier knows about it because it takes even longer. By design. So, unless you are running a load balanced cluster and can afford to take a server offline for a few minutes when you want to scan, yes, this is a problem with this approach.

    • Punting the problem to an "external verifier" is pretty neat. I wish I could do that with my next hard problem.

      It may be worth doing right. Look for malware from a hypervisor (memory, disk, network, etc.). Running this all inside the insecure machine is just asking for trouble, though, but is the best currently available. But even today there are cpu's shipping without virt support, so this can't be done for every machine yet or for a while. Still, I think many would spend the extra $50 if it worked wel

  • Still a needle (Score:5, Insightful)

    by dmgxmichael (1219692) on Monday March 15, 2010 @11:24AM (#31483200) Homepage

    A needle in a haystack wants roughly the same amount of space as a straw - doesn't make it any easier to find (indeed, that's part of the reason it's so hard to find).

    Even if this technique has merits, it does nothing to correct the primary reason for computer infection - stupid users.

    • by Pojut (1027544)

      Even if this technique has merits, it does nothing to correct the primary reason for computer infection - stupid users.

      As with most things in life, stupidity is the leading cause of problems.

      Except death. I think god has a monopoly on causing death in that department. (Take that last sentence however you will. Just remember: however you take it is how I meant it.)

  • by mangu (126918) on Monday March 15, 2010 @11:24AM (#31483202)

    How about a malware that masquerades as this detector and reports the RAM checksum is OK?

    • by evanbd (210358) on Monday March 15, 2010 @11:53AM (#31483668)

      Detecting a malware detector is just as hard as detecting malware. In general, detecting software of a specific type is halting-equivalent. In practice, the goal is to take shortcuts so that your adversary has a halting-equivalent problem and you don't. At present, the malware authors are winning. If we could force them to detect the malware detectors, that would be a huge advance.

      My skepticism about this is the obvious one: what if the malware just lets itself get swapped out, and relies on stealth to survive the process?

  • "Guarantee" (Score:5, Insightful)

    by Reason58 (775044) on Monday March 15, 2010 @11:27AM (#31483256)
    You can't guarantee anything in security. Especially when a human is involved.
  • And what if the malware lets itself be swapped out of RAM the same as all of the other apps?

    I'd love to have an approach to malware that could always detect unwanted processes, I'm just trying to find holes here.

  • Yeah you can detect that SOMETHING is there, but how do you determine whether that something is supposed to be there or not?

    If you assume all "somethings" are not supposed to be there, you'll have a worse situation than UAC with users being prompted all the time and getting conditioned to click "yes".

    After reading the article, it seems no different from doing an offline scan using ClamAV from a LiveCD except maybe slightly more convenient. You boot a "secure" detection mechanism in place of whatever is nor

  • I note that he seems to have missed a rather obvious possibility: there's malware in RAM, but it allows itself to be swapped out with all the other processes. Why wouldn't it? If it got loaded into RAM once, it'll get loaded again by the same vector. In fact, it has to rely on that happening, since at some point the RAM is going to be physically powered down. There's no point in trying to dig in like a tick.

    So as far as I can see, his magic technique will only catch malware that attempts to protect it

  • is that he's saying, in a round-about way, that if we know what everything on the computer is that isn't Malware, then if it's not what we know, then it must be Malware.

    That and some test for checking behavior.

    The problem which he doesn't seem to resolve is, "How do we know everything that isn't malware?". I mean, I see that he goes on about running this using only in kernel mode, so there should only be kernel memory in RAM, but what if the malware exists hooked in (as many seem to be that I've found) in o
  • Assume now that we have a detection algorithm that runs in kernel mode, and that swaps out everything in RAM. Everything except itself. Well, malware may interfere, of course, as it often does, and remain in RAM. But if we know how big RAM is, we know how much space should be free. Assume we write pseudo-random bits over all this supposedly free space. Again, a malware agent could refuse to be overwritten. It could store those random bits somewhere else instead... like in secondary storage.

    Then, let us compute a keyed hash of the entire memory contents -- both our detection program and all the random bits. Here is what could happen: If there is no malware in RAM, the results will be the expected result. An external verifier checks this, and tells us that the scanned device is clean. Or there could be malware in RAM, and the checksum will be wrong.

    Ah ha! All you need is a kernel-mode algorithm that knows exactly what should be in RAM, at all times! In other words, it has to emulate all of the legitimate software you’ll use, because otherwise how would it tell the difference between legitimate software’s use of RAM and malicious software’s use of RAM?

    What an idiot.

  • either
    * the malware is in the kernel, in which case it can provide a false checksum of the memory to the external verifier
    or
    * the malware is in userspace in which case it gets swapped out, the verifier determines there is no malware in the system, then it gets swapped back in and carries on performing its malicious activities with its user privileges

  • If the malware is /sbin/halt, you've still got a problem.

  • Everyone who claims to solve a long-standing problem "guaranteed", does not know all the possibilities that could thwart their solution. Guaranteed.

  • Easy (Score:2, Flamebait)

    by camperdave (969942)
    If $OS=="Windows" Then print "Malware Detected";
  • Is why didn't Microsoft make the OS files read only way back when?
    Make the user give explicit permission to over right system files?
    It wouldn't make it impossible to get malware but it sure a shooting could make getting ride of it easier.

  • by bmo (77928) on Monday March 15, 2010 @11:33AM (#31483380)

    How about we change things in Windows so it actually prevents infection in the first place?

    1. Educate users. Microsoft does a piss-poor job of this.
    2. STOP DEPENDING ON 3 MAGIC LETTERS TO DETERMINE IF SOMETHING IS CODE OR DATA. COME ON, SERIOUSLY. THIS SHOULD HAVE DIED WITH CP/M.
    3. Kill ActiveX - I know of no legitimate website besides Microsoft.com that requires ActiveX.
    4. If a file comes in from the outside world - STRIP ITS PERMISSION TO EXECUTE. MAKE THE USER UNPACK IT FROM AN ARCHIVE OR SET ITS PERMISSION.

    Really. Seriously.

    No, the above won't cover every situation, but it's a pretty good start.

    --
    BMO

    • No, the above won't cover every situation, but it's a pretty good start.

      You say those as if Microsoft isn't aware of the problems with their design decisions. They by-and-large don't bear the costs of their poor security but would bear additional costs if people had to learn new ways of interacting with files (support costs, engineering, etc.).

      So far they're right - their market position hasn't been adversely affected by the malware crapfest they've foisted onto their users. A cookie for he who figures ou

  • register (Score:5, Interesting)

    by bugs2squash (1132591) on Monday March 15, 2010 @11:34AM (#31483392)
    Some processors may have big enough register sets that malware could reside entirely within the CPU.
  • by nahdude812 (88157) * on Monday March 15, 2010 @11:34AM (#31483414) Homepage

    Sure, malware has to occupy memory. That doesn't mean it has to be its own memory. Buffer overflows are all about corrupting another application's memory space.

    His basic argument is that if you want to scan RAM, the kernel can halt all processing except its RAM scanner, and have a go at the RAM safely. If it's particularly insidious malware, it'll try to hide itself in various ways, one of which would be to masquerade the portion of RAM it was using with something legitimate looking (maybe erase that portion of memory). But you know it did this because you can see that memory which was supposed to be free is no longer free. Except the hardware has no concept of free or occupied memory. It just has memory, and the OS keeps track of what's free and not. The OS - the same space where malware is running.

    OR, the malware could simply not do this, then its behavior is no different from any legitimate program. So how do you detect it now? You still need definitions that say, "When running in memory, this virus looks like X," then look through memory for that pattern.

    Besides, who's to say that the kernel space is guaranteed free of malware itself? Even if you would have successfully identified the threat in RAM, you have no guarantee that the malware hasn't corrupted the identification routine.

    It's like someone came along and said, "Hey, you guys are looking for malware wrong. You have to look for it! And I mean really look for it!"

  • by Dunbal (464142) *

    Someone has discovered the white-list.

    Please take a number and stand behind the perpetual motion people. When I'm done with them, I will explain the few finite set of cases where this method DOES work, and you can assume that in the infinite number of OTHER cases, this method does NOT work.

  • 1) install malware
    2) report that there is malware installed
  • Redeeculous idea. (Score:5, Interesting)

    by Ancient_Hacker (751168) on Monday March 15, 2010 @12:24PM (#31484114)

    I tried reading TFA a few times. First time, utter confusion. Second, third times, no better. I can't make any sense out of these points:

    >1) There are absolutely only three things malware can do when you scan for it. One: be active in RAM, maybe trying to interfere with the detection algorithm. Two: not be active in RAM, but store itself in secondary storage. It cannot interfere with the detection algorithm then, quite obviously. And option number three: erase itself.

    Absolutely, not. There are many other things malware could be doing. Inactive in RAM, compressed and inactive in RAM, encoded as plausible-looking entries in the File Name Table or the Virtual Memory map.

    >2) Any program -- good or bad -- that wants to be active in RAM has no choice but to take up some space in RAM. At least one byte, right?

    No, it could be sleeping, existing only as an entry in the swapped-out process table. Or in unused space below a thread stack.

    >Assume now that we have a detection algorithm that runs in kernel mode, and that swaps out everything in RAM. Everything except itself.

    Whoah there fella. Everything? Are you going to turn off all timers and interrupt enables so their service routines don't get called?
    Hard to do without mucking up all the device drivers. Are you going to swap out the kernel too, as malware is quite capable of infesting kernel space. And what about device drivers? They're constantly mucking with their internal tables and I/O buffers.
    And if you turn off all device drivers, you lose, as there's nothing stopping malware from masquerading as a device driver. Many do.

    >>But if we know how big RAM is, we know how much space should be free.

    Whoa there again, big guy. There are plenty of machines with RAM at places not generally known to the OS, such as video RAM, graphics polygon RAM, network card RAM buffers, and kernel stacks.

    >> Assume we write pseudo-random bits over all this supposedly free space. Again, a malware agent could refuse to be overwritten.

    You don't need a checksum test to do this-- each page of virtual memory has R/W control bits.
    And you're foiled here again, as there are plenty of system areas that are write-protected, such as pre code areas and the VM tables themselves.

    >>Then, let us compute a keyed hash of the entire memory contents -- both our detection program and all the random bits. Here is what could happen: If there is no malware in RAM, the results will be the expected result. An external verifier checks this, and tells us that the scanned device is clean.

    Nooo, that just tells you that either you overwrote the malware, so you'll never find it, or the malware during your two sweeps did not change any RAM contents. Quite possible as most malware just sits around most of the time.

    >> Or there could be malware in RAM, and the checksum will be wrong.

    Well, no, unless you disabled all interrupts and stopped all kernel tasks, there will still be system timers and interrupts and device drivers changing their state in RAM.

    >> The external verifier would notice and conclude that the device must be infected.

    Or some part of the system or some device driver is still running. Huge chance of false positives.

    This essay seems to have been written by someone with only a glancing familiarity with hardware and system software.

  • by dskoll (99328) on Monday March 15, 2010 @01:40PM (#31485364)

    From the Our Solutions [fatskunk.com] page:

    A technique known as software-based attestation can provide an alternative defense against malware by performing infection scans periodically and detect the presence of any program that refuses to be inactivated – as well as any inactivated program that is known to be malicious.

    So, it can detect malware that refuses to be inactivated which is a tiny (vanishingly-tiny?) percentage of malware, as well as inactivated software that is known to be malicious (eg, because of a known virus signature.)

    So what's the advantage over signature-based virus-scanners? Well, you get to detect completely hypothetical software that (somehow) refuses to allow the kernel to swap it out (and how that is possible is never explained) at the cost of hugely-expensive computations.

    Great.

The tree of research must from time to time be refreshed with the blood of bean counters. -- Alan Kay

Working...