Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!

 


Forgot your password?
Close
typodupeerror
×
Security

Known-Good MD5 Database 309

Posted by timothy from the damn-good-idea dept.
bgp4 writes "Have you ever examined a system you thought was broken into but you weren't sure? If only you had run an integrity verification program like osiris or Tripwire first you could have figured out what programs had been changed. In an effort to help out in the instances when you can't answer the question "what was this like before?" we've constructed a searchable database of MD5 and SHA-1 hashes for files in many standard operating systems. You can search using the filename or the checksum and see if you have a trojaned binary or an overactive imagination. Currently at knowngoods.org we have many FreeBSD, OS X, Linux, and Solaris installations checksummed and cataloged. If you have other programs or distributions you would like to see in the database, please let us know."
This discussion has been archived. No new comments can be posted.

Known-Good MD5 Database More

Known-Good MD5 Database

Comments Filter:

  • Yes, in fact, I have! (Score:4, Funny)

    by Anonymous Coward on Monday December 09, 2002 @12:33AM (#4841830)
    Have you ever examined a system you thought was broken into but you weren't sure?
    Just about every time I've broken into a system! :)

    • Re:Yes, in fact, I have! (Score:5, Interesting)

      by Anonymous Coward on Monday December 09, 2002 @03:11AM (#4842394)
      Have to AC this one....

      [ This is a story about why getting "good" checksums to start with is very important. ]

      On a related topic: Ever examined a system you didn't think was broken into, and were sure?

      The sysadmins at my old school did. And they were wrong.

      You see, they connected a new box, the replacement main server, to the LAN, and used an easily-guessable password convention for staff accounts, PRIOR TO RUNNING TRIPWIRE on it. Seems "someone" got in and changed a few key binaries, THEN the admins ran Tripwire. Periodically, when the system got munged and a restore was required, they'd restore the original tapes, Tripwire would yell about a few binaries (including some innocuous distractors), and the admins would dutifully go to backups, find the modified binaries and restore them, figuring they had to be right, because of course, they matched the Tripwire signatures.

      Ya gotta love self-repairing back doors when you're a student at the mercy of admins who work 9-5 M-F, NFS and lpd subsystems that croak only after 10pm or on weekends, and newbies who fill up file systems.

      The local 3-person student root cabal used these back doors for several years, until the machine was replaced. AFAIK, the admins never knew. They had spent much of my undergrad time trying to find SOMETHING I'd done, to punish me for, so if they'd known about this...

  • What about source builds? (Score:5, Insightful)

    by Anonymous Coward on Monday December 09, 2002 @12:33AM (#4841832)
    Wouldn't this be useless to anybody that builds from source?
    • If the distributor of your source was compromised to give out a file containing a trojan or other nasty surprise, then no, it isn't useless.

    • Re:What about source builds? (Score:5, Insightful)

      by Cerlyn ( 202990 ) on Monday December 09, 2002 @12:45AM (#4841905)

      Indeed; the capability of such a system is a bit limited with operating systems like FreeBSD, which actively *encourage* their users to build/rebuild from sources. IIRC, FreeBSD actually only gives intermediate security updates in source code format so you have to compile them (not too hard: cd /usr/src ; make buildworld).

      So, recording the checksum to /bin/ls, etc. is a bit flawed in that when I do a "make buildworld", my custom configuration parameters from /etc/make.conf get used, overriding CPU type, if Xfree86 is installed, etc. Since my system's parameters likely will not match FreeBSD's master build system, there is a high chance that the checksums after I do a rebuild are significantly different.

      But for non-source distributions (Redhat, etc.) this concept is excellent, assuming that no one compromises the database or the OS kernel. Unfortunately, no database checksummer will ever counteract the case when the OS kernel itself is compromised, potentially returning one file when scanned and another when executed.

      Still, it wouldn't hurt for them to record source file checksums as well; after all, having an independant checksumming group would require them to be compromised as well as the FTP network, making an attacker's life harder.

      • Re:What about source builds? (Score:4, Insightful)

        by pVoid ( 607584 ) on Monday December 09, 2002 @01:02AM (#4841981)
        Indeed.

        In fact, this system would be best suited for systems which aren't OSS... such as windows =)

        crowd boos... stones and rotten tomatoes fly as author runs for cover

        :)

        • Actually, I know Windows 2000 Professional has a similar system. I've recently been installing/reinstalling a few things and suddenly a box popped up saying something like "Windows File Integrity Checker. Windows has detected that vital system files have been modified and to ensure stability needs to restore these files from the Windows 2000 Professional CD". I'm not sure which files it checks or how, but I do know it has got a least "a" level of checking inbuilt.

          • Re:What about source builds? (Score:4, Informative)

            by pVoid ( 607584 ) on Monday December 09, 2002 @01:59AM (#4842197)
            Yes, the Win32 PE format (portable executable) has a checksum field which is 'normally' not used.

            It *is* checked for *some* critical system images however... I know for sure that some files in /system32 (so called 'KnownDlls') are among this list.

            Note though, that this checksum is to prevent accidental data corruption and not maintain system security integrity; since the checksum field is actually in the file itself, it can be updated after a virus/haxxor has patched the target file.

        • A database of known good files for Windows is being built by the National Institute for Standards in Technology. It's called the National Software Reference Library [nist.gov] and it costs $90 for an annual, agency-wide subscription.

          There are some significant problems with their database however:

          1. It's huge. They have a single giant flat file with SHA-1, MD5, MD4, and CRC-32 values. Right now the file, when uncompressed, is over 1GB.
          2.Over 42% of the entries are duplicates. I found this out by running sort -Uf on it.
          3. Many of the files were hashed before installation. The MD5s for these files often change the installation process.

      • Still, it wouldn't hurt for them to record source file checksums as well;
        Right. In a source distribution, checksum the source files and whatever binaries you started with (gcc etc) and what's the problem?
      • Unfortunately, no database checksummer will ever counteract the case when the OS kernel itself is compromised, potentially returning one file when scanned and another when executed.
        Not if you execute your md5sum or other checksum program in a trusted environment, e.g., after booting a rescue system from CD/DVD-ROM. If you suspect that your system has been compromised, you probably wouldn't want to run any executables directly on that system.
    • Wouldn't this be useless to anybody that builds from source?

      Even worse, if you use, say, FreeBSD, and you build from some cvs tag other than RELENG_4_7_RELEASE or so (I use RELENG_4), chances are you've got quite a few small deltas dotted around the system -- something like this would need to track md5 changes of not just releases, but of the -STABLE branch (at every commit) to be useful to me -- and you've still got the security branches (RELENG_4_[1-7]) to worry about.

      It's hard, even without getting into handling the various differences compiling can introduce -- compilation date timestamps, alternate build options, etc.

  • What?! No Windows? (Score:2, Insightful)

    by Anonymous Coward
    We need file verification, too! Probably more so with some of the Windows/IE vulnerabilities.
    • Windows doesn't really have a good system of labeling releases, and I'm sure that the people running this website don't wanna do this for every service pack available for most microsoft products.
    • We need file verification, too! Probably more so with some of the Windows/IE vulnerabilities.

      Don't worry, you'll have that soon. It's called Palladium.

      As my grandmother used to say: "Be careful about what you wish for, because your wishes might come true". Wise woman.
  • What if someone hacked into the MD5 database and changed the entries? :-)

  • This is one of those things... (Score:3, Insightful)

    by carl67lp ( 465321 ) on Monday December 09, 2002 @12:35AM (#4841844) Journal
    This is the type of thing that you'd ask "Why didn't they do this sooner?" -- it's just that logical of an idea.

    Absolutely fabulous, wonderful! The real trick, though, is to build up trust in your database so that those searching it will be sure that the checksums are actually correct--you know, rather than buying a burglar alarm from the robber himself. Thus, I doubt you'd be able to take submissions from users right away--at least without a competent staff checking to make sure they're correct.

  • Cool? (Score:2, Interesting)

    by kir ( 583 )
    You know, this is sort of cool... until it gets hacked (cracked... whatever) and then your entire OS looks bad. Wait. That is COOL!

  • ooooo nifty (Score:5, Insightful)

    by netwiz ( 33291 ) on Monday December 09, 2002 @12:37AM (#4841859) Homepage
    I've been wondering when something along these lines would be available.

    [devil's advocate] However, how do we know that the pregenerated checksums are correct? Who watches the watchers? [/devil's advocate]

    Yah, yah, I know, the easiest way is to inspect the source for the minicompiler, the main compiler, and the program by hand, then build all of them step-by-step until you're done, then use the final binary to generate your hash. I wonder, tho, how much drift might there be in using a pre-built compiler (say I D/Led the binaries for GCC and the libraries to go with it). One tiny change in machine state (or any other number of things I would suppose) could result in the final binary being a single byte off, and the whole thing's a wash.

    Granted, I may be talking out of my ass here, could someone w/ some hard-core coding knowledge or CS experience expound on the above?

  • What about Windows OS? (Score:5, Insightful)

    by scubacuda ( 411898 ) <scubacuda@gmai[ ]om ['l.c' in gap]> on Monday December 09, 2002 @12:41AM (#4841871)
    I didn't see the ability to search for Windows MD5 hashes.

    Considering its history of vulnerabilities, I'd think that this would be pretty important...

  • Compromised /bin/md5 (Score:5, Informative)

    by Cadre ( 11051 ) on Monday December 09, 2002 @12:41AM (#4841877) Homepage

    What they don't say and what a lot of security folks forget to do is that they can't check your checksums of binaries on the same box. You need to copy the files to another box and check the checksums there with a known good version of your checksumming binary. The local version of your checksumming binary could have been compromised.

    Heck, the utilities you used to pull the binary off the machine in question could have been compromised and may not be actually copying the binary in question, but a good version of the binary. The only way to check this would be to mount the drive on another machine and check it there... And if people aren't doing that (which it's a pain in the ass) all this website is going to do is give people a false sense of security.

    • Heck, the utilities you used to pull the binary off the machine in question could have been compromised and may not be actually copying the binary in question, but a good version of the binary. The only way to check this would be to mount the drive on another machine and check it there... And if people aren't doing that (which it's a pain in the ass) all this website is going to do is give people a false sense of security.>Heck, the utilities you used to pull the binary off the machine in question could have been compromised and may not be actually copying the binary in question, but a good version of the binary.

      Other replies have mentioned that it might make more sense to boot off known clean read-only media, on which you also have a copy of your checksum utility.

      That said, this still provides a false sense of security. The only way to be absolutely certain that your binaries have not been compromised is the following technique:

      Have all your code written by hermit programmers. They must develop their OS and all programming tools (compilers, etc.) by themselves, on a computer that has no connection to the outside world. Taking an OS from another hermit programmer is also acceptable, as long as it is conveyed by hand from one to the other.

      You must know and trust all of the hermit programmers.

      The hermits must live, eat, and sleep in giant vaults designed to provide physical security to them and their computers. They definitely will not have telephones.

      They must develop applications from scratch--no outside data may be allowed to contaminate their pristine systems. Source code may be imported, as long as it is delivered in hard copy form and hand keyed by someone who is very security conscious.

      The hermits must hand deliver the binaries of applications to you. You should have already received a copy of their pristine OS by this method.

      Presto! Completely secure binaries. No trojans. No false sense of security.

      Oh, unless someone finds a buffer overrun that your hermits missed. Then some kiddie will own your box. Damn.

      • What if the hermits' computer components have backdoors that automatically insert backdoors into everything written on them?

        You'd have to have sterile hermits manufacturing CDs out of their own feces and urine (sterile) and burning code on them with laser pointers manufactured from the same source with machines made out of (you guessed it!) poop and piss.

        Now you know why I hate those filthy asshole hermits.

      • OK, I'll admit I'm not 100% sure about Richard Stallman's personal life, but from what I've heard, I think I can trust /usr/bin/gcc :)

    • "False" senses of security (Score:5, Insightful)

      by Hizonner ( 38491 ) on Monday December 09, 2002 @01:45AM (#4842145)
      Spoken like a true second-year student.

      The reality of the matter is that, while it certainly would be possible for somebody to gag a machine to evade all your wascally checksumming tricks, they frequently don't do so. And when they do it, there's the usual arms-race lag between the time when a new method of checking comes out and when they update their tools to evade it. And there's a cost to them for each defense they evade; if you want to avoid every defense you ever hear of, you basically have to roll your own rootkits, which is a huge time investment.

      And a kiddie who's out there collecting hundreds of boxes has no particular incentive to be anal about holding onto yours.

      ... and everybody makes mistakes. Yes, you're right, looking at checksums gives you absolutely no security against omniscient adversaries with infinite resources. Luckily, real adversaries are not omniscient and have limited resources. Yes, you'll even miss some of the real adversaries. You'll also catch some. Probably a lot. Nothing is perfect. Deal with it.

      Fucking pompous amateurs.

      • Re:"False" senses of security (Score:2, Insightful)

        by Anonymous Coward
        He may be a second year student, but he's right. You don't check a potentially compromised system with itself! Mount the drive on a trusted system and check it there. This isn't even hard. If you suspect a breakin, schedule a half-hour of downtime and boot from a trusted CD, like Knoppix or the live filesystem that comes with Slackware, and check your HD from there. Simple!

        • If you suspect a breakin, schedule a half-hour of downtime and boot from a trusted CD, like Knoppix or the live filesystem that comes with Slackware, and check your HD from there.

          And if the BIOS is trojaned?

      • The reality of the matter is that, while it certainly would be possible for somebody to gag a machine to evade all your wascally checksumming tricks, they frequently don't do so.

        No kidding. I still haven't even seen people updating the RPM md5sums, and you'd think that's something that rootkits would like to do.

        Sure, if you know your system has been compromised, you want to take it down and do a check with known-safe binaries, kernel, etc. But in the real world you can't do that daily on a production box, so checksumming on a live box is a reasonable solution.
    • You need to copy the files to another box and check the checksums there with a known good version of your checksumming binary

      You're being ridiculously pedantic about the theoretical limits of security, yet you naiively trust tar/dd/cp/NFS to copy the files correctly? You trust the drive firmware? The machine's BIOS? The CPU?

      Either take the security argument to its true limits, or realize that practical choices need to be made and be quiet.

    • And what if you have a compromised kernel? Even if the binary on the disk is 'clean', and md5's ok on any box, when loaded and run on the compromised system any code could be run.

      But how can the kernel be compromised by just d/ling a binary you ask?...Exactly, it can't, the same with md5. If someone has managed to compromise md5 on your machine, youve got more problems than not being able to verify d/ld files, your machine has already been rooted.
      • youve got more problems than not being able to verify d/ld files, your machine has already been rooted.

        Wouldn't that be the whole point to running a checksum scan on your system's binaries?

        *scan* Oh good, everything's fine.

        -OR-

        The hell? I've been r3wt3d! d00d!

        I don't think this is in reference to just verification of "d/ld" files, but a method for scanning your already existing system for problems.

        The most secure thing I can think of to do would be have a server box (a nice high powered full-throttle beast, imagine a beowulf cluster of THESE) and a security check box (P3 166 with 64MB RAM/8GB HD is plenty). Have a cron-triggered script and a checksumming verification program on the security box which in the middle of the night, or even at random times whenever, will bring up eth0 on the security box, mount the server's disks over NFS, check them, and then bring down eth0 so as to isolate the security box. Such a thing would probably work better with a cluster of servers, say a load-balancing system of web servers, so that only one box out of the total whole had to be down at once and the rest could still be functioning, while the box being checked is isolated by a controlable router or something. Of course that's gotta be secure, too....ARG headache

  • Polymorphic files (Score:5, Informative)

    by cperciva ( 102828 ) on Monday December 09, 2002 @12:42AM (#4841884) Homepage
    There is one problem with this: Some files are going to be different every time they are compiled. In particular, quite a few files include time stamps.

    A few months ago I put together a list of the "polymorphic" files in FreeBSD 4.6:

    /kernel, /boot/loader, and /boot/pxeboot all contain user, host, time, and date stamps, as expected.


    All .a files (126 in /usr/lib, one in /usr/libdata/perl/5.00503/mach/auto/DynaLoader) contain indices of .o files, including seconds-since-epoch stamps

    User, host, time, and date stamps are found in /etc/mail/freebsd.cf /usr/sbin/named /usr/libexec/named-xfer

    Time and date stamps are found in: /usr/bin/suidperl /usr/bin/ntpq /usr/sbin/ntp(d|date|dc|timeset|trace) /usr/sbin/isdn(d|debug|monitor|phone|telctl) /usr/libdata/perl/5.00503/mach/perllocal.pod

    Date stamps are found in: /usr/sbin/ppp /var/db/port.mkversion /usr/share/doc/usd/(07.mail|13.viref|18.msdiffs|19 .memacros|20.meref)/paper.ascii.gz (once you ungzip them) /usr/share/perl/man/man3/(Config|DynaLoader).3.gz (once you ungzip them)

    Files which are always the same size, but have randomized contents: /usr/share/games/fortune/*.dat /var/games/phantasia/void


    These files are always going to set off alarms if you've upgraded-by-source. (On the other hand, if a file *not* on this list has a different checksum, it probably just means that you've applied a security patch.)

  • md5sum Binary Might Be Trojaned (Score:5, Informative)

    by John Hasler ( 414242 ) on Monday December 09, 2002 @12:46AM (#4841911) Homepage
    Boot from a known good floppy or CD to check your md5sums.

  • Filtered as a "Hacking" site (Score:4, Interesting)

    by KidSock ( 150684 ) on Monday December 09, 2002 @12:51AM (#4841932)
    Mu corporate www proxy filters this site as category "Hacking".

  • config files (Score:5, Funny)

    by Erpo ( 237853 ) on Monday December 09, 2002 @12:55AM (#4841949)
    This is great for precompiled binaries, but it won't work so well for config files - they're different from system to system. I have a better solution:

    Anyone who wants to make sure their important config files haven't been changed by an intruder can email them to me, and I'll hold on to them for safe keeping. /etc/passwd and /etc/shadow are especially likely to be modified, so I'd recommend sending those right away.
  • have some Ajax (TM), you can get paranoid without even smoking anything!

  • Relief (Score:3, Funny)

    by eyeball ( 17206 ) on Monday December 09, 2002 @12:58AM (#4841965) Journal
    Oh good, the md5 hash for my /sbin/md5 binary matches the signature found on known-goods. Now I can sleep at night. oh, wait...

  • What about focusing on files that are routinely replaced with trojans, rootkits, etc.?

    I'm not saying NOT to do the rest of the files, just that these (I'd think) would be the files that you'd want to check FIRST before the rest of the system.

    Perhaps a separate section featuring these targeted areas?

  • do any of the distributions allow for doing something like this via apt/dpkg?

    likely only handle part of the .rpm/.deb but it seems like something workable for the binaries that are installed.

  • Something's wrong here (Score:5, Insightful)

    by phr2 ( 545169 ) on Monday December 09, 2002 @01:08AM (#4842011)
    If we need an external database of md5's to authenticate so many different files, that means that md5's weren't really the right authentication method to begin with. It's better to use digital signatures.

    The fancy way to do that is with an Authenticode-like system for signing files. Distro maintainers would sign the files in their distros, and users could also sign their own files. A simpler way would be to just have a big, signed list of md5's in some file that tripwire checks against. Tripwire would check the signature on the file before believing the md5's in it. Or the list could contain individual signatures per file instead of just hashes.

    A centralized md5 database doesn't feel so right with the free software spirit, which says (legitimate) users could modify the files at any time, or just recompile them with a slightly different compiler, etc.

    • And what's to prevent an intruder from adding a trojan to the signature-checking program/library?

      Chicken-and-egg...
      • Hopefully enough other people would review it.

        Perhaps some sort of rating system?

      • Re:Something's wrong here (Score:3, Interesting)

        by phr2 ( 545169 )
        Nothing stops an intruder from trojaning the known-good-retrieval program either.

        Basically to be really careful, you have to do the checks offline, on a separate computer, i.e. not relying on executables running on a system that's been exposed to attackers.

        This is the kind of thing that the Palladium hardware should be able to help with. What Microsoft wants to do with it is evil, but it's capable of being used for good purposes too.

    • A simpler way would be to just have a big, signed list of md5's in some file that tripwire checks against. Tripwire would check the signature on the file before believing the md5's in it.

      Note: this is exactly what tripwire already does. Except that it also stores other file attributes as well.
    • A MAC is probably a better idea. It's basically a hash that requires a secret key to compute. You could keep a database of known-good MACs on your hard disk, and if you suspect a crack, boot from a CD and verify them. This way you just need a special password to update or verify the database (although if you suspect the database checker has been trojaned, you'd want to boot from a read-only medium to check it). I don't see the point of a centralized database here, especially with so many people rebuilding from source. Oh yeah, and a MAC works on Windows (or, hey, MacOS) too.
    • Every package in the current Red Hat Linux is signed using GPG, and IIRC this has been the case for a few years now. Most other Linux distros also sign their packages.

  • Debian / debsums (Score:5, Informative)

    by zsazsa ( 141679 ) on Monday December 09, 2002 @01:11AM (#4842016) Homepage
    Debian has this built into the OS with debsums [debian.org].

    It does require a legit dpkg database (and md5sum, and the debsums program itself...) but it's a nice tool.

  • This sounds nice, but I see problems as installs move from the "100% installed code" to the "patch of the week" versions. Especially when you have to do custom builds of the software.

    Are you running BIND, Apache, wu-ftpd, or (shudder) Sendmail? If you are, your system won't be entirely in their shiny dbase for more than a month (probably more like a week) after you install. And if _you_ don't update it, someone will be kind enough to "update" some file for you soon enough...

    As a test, I checked /bin/ps on a few local systems. (If you don't know why I started with this one, you will. Probably sooner than you'd wish to.)

    From the dbase:

    RH 7.1 - MD5: ac0b58050deb21db1ed701277521320b
    RH 7.3 - MD5: 6d3abf4efc9235e4eb5dc540d61d42fa

    My systems:

    #1 - MD5: ac0b58050deb21db1ed701277521320b
    #2 - MD5: ac0b58050deb21db1ed701277521320b
    #3 - MD5: 9724525265900e5f9020de3b431425b1
    #4 - MD5: 881c7af31f6f447e29820fb73dc1dd9a
    #5 - MD5: 6d3abf4efc9235e4eb5dc540d61d42fa

    Binary compatibility is seen for systems 1, 2, and 5, but the RH7.2 system (#4) doesn't match. System #3 is a Gentoo system, which is probably the most secure, but also the least likely to ever match with their list. I guess that's the peril of compiling your own code.

  • Use BITZI too! (Score:3, Informative)

    by aminorex ( 141494 ) on Monday December 09, 2002 @01:15AM (#4842028) Homepage Journal
    I'd rather see everyone using bitzi.com, since it's
    goal is to gather metadata for *every* file in the
    universe, and keep the data free, supported by a
    related business model (and a viable, sustainable
    support mechanism is GOOD), but I support this
    project too, because choice and freedom are goods.
    Therefore, I urge everyone to submit metadata
    to both projects.

    If you only submit to one, however, please submit
    to bitzi, because it provides an automation API,
    and uses better hashes.

    Note that I have no affiliation with the Bitzi company.

  • Another Resource (Score:5, Informative)

    by Taim ( 1157 ) on Monday December 09, 2002 @01:19AM (#4842048)
    NIST (The National Institute of Standards and Technology) currently has a program to provide this service, though largely focused on Microsoft OSes and associated apps. It may be found here: National Software Reference Library [nist.gov]

    The complete list of software they've checksummed can be found here: Software Listing [nist.gov] or you can use their search engine if you're looking for a specific application here: Search Engine [nist.gov]
  • Something to combat md5sum itself from being cracked. Perhaps a statically compiled binary that you can download with the program of your choice. Then rootkits would have to modify every program that can download a file, or the kernel. The best system would be a nice bootable CD that would scan all known file system types for files that have md5 sums of known bad files, not search for files and make sure they have a md5 sum of a good file. Then root kits will have to rely on a compiler or append random bytes to the end of the files.

    Well this gets so complicated that by the time you've thought it all out, you really need virus scanner technology to thwart root kits. Maybe a kernel patch could run a virus scan on executable files? It would be quite difficult to tamper with the actual running kernel in memory without causing the system to lock or reboot, thus giving away that the system is being tampered with. Assuming root kits are distributed in source form, you'll need heuristic scanning to find them. This means false positives and manual overides by the system administrator.

  • Combination solution. (Score:3, Interesting)

    by pr0ntab ( 632466 ) <pr0ntab.gmail@com> on Monday December 09, 2002 @01:21AM (#4842059) Journal
    Ideally, a simple tool should be developed that does the following:

    Compare the MD5sums of critical files to a recent known "snapshot" of the system on RO media, which only indexes files that were changed and reconciled. Perhaps there is a list of files of which only certain byte ranges (perhaps just executable ELF sections) are checked, are some are omitted. (Other slashdotters mention caches/timestamps in certain relevant files that screw up checksums). You would have a whitelist (files which must match), then a graylist (files which meet byte-range criteria), and perhaps even a blacklist that prevents files that would normally be flagged to be ignored.

    In checking full file checksums, those not explicitly listed above would fallback to a check using a HTTP get request conforming to this helpful document [knowngoods.org] these guys have offered.

    And to those who were asking about other distributions: they are looking for people willing to work with them to add new distros/architectures to their database.

  • Bleah (Score:4, Informative)

    by digitaltraveller ( 167469 ) on Monday December 09, 2002 @01:31AM (#4842094) Homepage
    NIST [nist.gov] does this too. For a different reason though. To help forensic examiners eliminate non-important data in a suspect's computer. They use 4 different hash algorithms (MD5, SHA-1, CRC32, and one other), so good luck finding a collision for all 4. They were giving out copies of the CD-hashdb at an InfoSec conference I was at recently.

  • Excellent! (Score:4, Interesting)

    by defile ( 1059 ) on Monday December 09, 2002 @01:43AM (#4842137) Homepage Journal

    Now I can add a compromised md5sum to my rootkit which uses values from this site.

    Go team!

  • Er, rpm -V? (Score:5, Informative)

    by SlashChick ( 544252 ) <erica@noSpam.erica.biz> on Monday December 09, 2002 @01:49AM (#4842161) Homepage Journal
    For Red Hat-based systems, at least, rpm -V will do pretty much exactly what you're looking for.

    From the man page for rpm:

    The general form of an rpm verify command is

    rpm {-V|--verify} [select-options] [--nodeps] [--nofiles] [--nomd5] [--noscripts]

    Verifying a package compares information about the installed files in the package with
    information about the files taken from the package metadata stored in the rpm database. Among other things, verifying compares the size, MD5 sum, permissions, type, owner and group of each file. Any discrepencies are displayed. ... The (mnemonically emBoldened) character denotes failure of the corresponding --verify test:

    S file Size differs

    M Mode differs (includes permissions and file type)

    5 MD5 sum differs

    D Device major/minor number mis-match

    L readLink(2) path mis-match

    U User ownership differs

    G Group ownership differs

    T mTime differs

    So while that's a bit cryptic, a shell script run once every x days (30? 14?) should tell you what files have changed. All you would have to do is run rpm -qa to grab a list of the packages in your system, and then loop through the list and run rpm -V for each RPM returned.

    For instance, running rpm -V on two common packages, Apache and PHP, shows me the following:
    # rpm -V php
    S.5....T c /etc/php.ini
    (php.ini has changed... which in this case means I've tweaked some of PHP's default settings.)

    # rpm -V apache
    S.5....T c /etc/httpd/conf/httpd.conf
    missing /var/www/html/index.html
    missing /var/www/html/poweredby.png
    (Okay, I've changed httpd.conf, again pretty much a given, and I've removed a couple of the default files.)

    I guess this website seems pretty unneeded to me. Granted, the above is just for RPM-based systems, but I'm sure Debian and ports have similar options. And to the people who have installed from source and say "What about me?", I say, first, never underestimate the power of a package management system, and second, check out CheckInstall [asic-linux.com.mx], which allows you to create an RPM or DEB just by saying "checkinstall" instead of "make install". If you feel you must compile from source, checkinstall is a necessity! Using checkinstall gives you all the benefits of a package management system while still allowing for the flexibility that compiling from source provides.

    Between checkinstall and up2date, I'm a very happy Red Hat customer. I just wish more people knew about some of the truly powerful things in package management systems (such as the verify command detailed above.) Package management systems are there for a reason. Use them! :)

    • Re:Er, rpm -V? (Score:3, Insightful)

      by Nicolas MONNET ( 4727 )
      Usually crackers don't think of altering the RPM database containing the MD5 hashes -- it's happened to me during a Bind compromise --, but there's nothing that would prevent them from doing so ... so you need an external database.

    • Re:Er, rpm -V? (Score:3, Informative)

      by D0wnsp0ut ( 321316 )
      For Red Hat-based systems, at least, rpm -V will do pretty much exactly what you're looking for.

      For those who pointed out that the RPM database is a local database, remember, you can get those MD5 hashes from 2 other sources:

      • your installation medium (CD)
      • from Red Hat's web site itself

      Of course, using the web site is going to be a lot more labor intensive, but it is available. Writing a script to compare hashes computed using RPMs from the CD image and comparing them to your installed binaries should be a piece of cake (using the --dump option to -ql).

  • The problem with this is that it assumes that binaries never legitimately change. That is false in Mac OS X. The Mach-O file format allows for "pre-binding", where the linker tries to resolve imported functions and data before the app is loaded, and, if successful, writes the offsets to the file.

    I'm not familiar with the Mach-O file format, but I'd guess that the changes are confined to a small part of the file. But if you could just checksum the code sections, that might work.

    On the other hand, talking about libraries makes me think. Don't forget to check the libraries. If I trojan libc, I'll be getting all kinds of control while leaving the program binaries unmodified.

    rlwimi

  • how does this work? (Score:5, Funny)

    by thogard ( 43403 ) on Monday December 09, 2002 @01:55AM (#4842181) Homepage
    Ok, lets see if I've been hacked...
    $ md5 /dev/null
    d41d8cd98f00b204e9800998ecf8427e

    So I put d41d8cd98f00b204e9800998ecf8427e in the search engine and it came up with 560 hits (compared with 3170 from google).

    Now it appears that someone replaced my /dev/null with /private/var/servermgrd/servermgr_dirserv.lock from Mac OS X. What a bummer and its a brand new system too...

    Does the database have a way to flag files as being bad? Sa

    When I put in 3ac9bc346d736b4a51d676faa2a08a57
    I should get back:
    *** Trojaned openssh-3.4p1.tar.gz ****

    One thing that could make this useful would be a dns like interface...
    host 3ac9bc346d736b4a51d676faa2a08a57.knowngoods.org || echo bad
    • >When I put in 3ac9bc346d736b4a51d676faa2a08a57
      I should get back:
      *** Trojaned openssh-3.4p1.tar.gz ****

      How the bloody hell will it know that? Just changing a couple of bytes in a file will completely change the checksum. There's no way of telling what the file _should_ be from the checksum.

  • Solaris Fingerprint Database (Score:5, Informative)

    by nbvb ( 32836 ) on Monday December 09, 2002 @02:08AM (#4842231) Journal
    Sun already provides this for Solaris.

    http://sunsolve.Sun.COM/pub-cgi/fileFingerprints .p l

    It contains information for:

    Operating Systems

    Solaris SPARC - 2.0, 2.1, 2.2, 2.3, 2.4, 2.5, 2.5.1, 2.6, Solaris 7 and Solaris 8
    Solaris x86 - 2.1, 2.4, 2.5, 2.5.1, 2.6, Solaris 7 and Solaris 8
    Solaris PPC - 2.5.1
    Trusted Solaris SPARC - 2.5, 2.5.1 and 7
    Trusted Solaris 7 x86
    Most CDs bundled with Solaris 2.6 and later.

    Patches

    Nearly all released Solaris patches, including all SunSolve CDs to date. (4.0.11)
    All Solaris 2.6/7 Maintenance updates.
    All patches available from SunSolve.

    Unbundled Products

    Around 150 CDs with unbundled products are included. If you are missing any particular product, please feel free to send email and we will try to include it as soon as possible.
  • A MAC is like a hash with a secret key; the hash depends on both the file and the key. That way you can keep a copy of the database locally and an attacker can't change it to cover his tracks like he could an MD5 database if you kept it locally (of course he could change the verify program if that were local...but probably not without changing the file size).

    This would allow for protection of files not in an online database (compiled from source, for example) using only local files.

    You can use a block cipher chaining mode (don't remember which one) as a MAC, or use say AES_k(MD5(file)), or, IIRC you can use MD5(k_1 file k_2) where k_1 and k_2 are different secret keys (check this out before using, lots of constructions like this are totally insecure), or you can use something designed as a MAC (eg RIPEMD). Any of these could be run from a shell script to quickly verify all binaries (or whatever you were protecting).

  • Personally.. (Score:3, Interesting)

    by shepd ( 155729 ) <slashdot.org@gmai l . c om> on Monday December 09, 2002 @02:22AM (#4842281) Homepage Journal
    I like this [chkrootkit.org] utility. It's pretty handy, although probably not as effective as this database, unless you're running slackware, or another popular, but undatabased distro. :-)

  • What about AIDE? (Score:4, Interesting)

    by strobert ( 79836 ) on Monday December 09, 2002 @02:34AM (#4842302) Homepage
    the poster mentions Tripwaire, but what about AIDE [cs.tut.fi]?
    In additon to being a proper Open Source project, it allows for features that (last I heard at any rate) tripwire doesn't support, like a centralized checksum DB. That feature alone makes the tool superior (IMHO). For example it makes the verification process a lot nicer (intruder can't courrpt the local md5sum's because there aren't any).

  • Versions? (Score:4, Insightful)

    by tconnors ( 91126 ) on Monday December 09, 2002 @03:05AM (#4842374) Homepage Journal
    OK - debian seemed to have one version there - r5, whatever that is. How does it handle apt-get upgrades? If r5 is reffering to something like stable, then even stable changes over time (contrary to what some poeple think ;-). So do they take the checksums from a machine that was just apt-get upgraded last night, or what? If they mean an actual yearly or half yearly release, who on this world does not apt-get upgrade when there is a security fix released? So your system sure as hell aint going to match theirs.

    Then I can't imagine how you would be able to automate this, so it checks all the binaries in /bin /sbin, /usr/sbin etc - do they have some alternative to HTTP for their database?

    Doesn't seem overly useful to me....

    • Source distros! (Score:3, Insightful)

      by PigleT ( 28894 )
      "Doesn't seem overly useful to me...."

      Nor to me, for a different reason: what about those of us with CFLAGS= set to various strange funky optimizations in Gentoo? What about the Ports system in FreeBSD, similarly?

      This thing does not have the potential to spread to all distributions or all unixen.

      What about historical storage? Are they really proposing to store an md5sum for /bin/* /usr/bin/* for all packages for all distributions for all releases, or when do older things get purged?

      Seems mad to me. Would be better off staying with AIDE instead, IMO.

  • Straightforward solution (Score:4, Interesting)

    by zunger ( 17731 ) on Monday December 09, 2002 @03:05AM (#4842375)
    I found a fairly straightforward solution to this problem. I wrote a small wrapper around a known-good md5 function, compiled it and placed it in a nonstandard location. (Thus it doesn't have a widely recognizeable filesize or md5 to be detected and stomped) Then I wrote a simple shell script which checksums various critical files on a regular basis and tests the MD5 values against a record it keeps, again in a private location. Whenver a change happens, it sets off alarm bells all over the place, both in syslogs and on the console.

    On top of this I stuck in one small bit of shell script that allowed me to modify a file myself without setting off alarms - it simply recalculated the md5 value and updated the record files.

    I suppose this is theoretically vulnerable to an attacker reading through /etc/crontab, then checking each local shell script for a sensor and carefully overwriting my own nonstandard code - but if any attacker has that much free time on his hands, there's a limit to how much of a sensor I can implement.

    The nice thing about this code is that it also implicitly tests for corruption of critical files after fsck-triggering events like kernel panics or total power failures. (That's actually what prompted its initial writing) And it's remarkably trivial to implement, even more so if one simply copies an off-the-shelf md5 binary rather than compiling one's own wrapper.
  • What about user/machine specific or configurable scripts? Those that are created during setup, and can't be check-summed.

    These files can contain trojan too, and most "modern" trojans are written in some interpreted script, not in C. How about detecting this?

  • I was dissappointed to see no OpenBSD database. :-(

Slashdot Top Deals

He has not acquired a fortune; the fortune has acquired him. -- Bion

Close