Stories
Slash Boxes
Comments
Search

News for nerds, stuff that matters

Slashdot Log In

Log In

Create Account  |  Retrieve Password

Data Loss Bug In OS X 10.5 Leopard

Posted by kdawson on Mon Nov 05, 2007 06:33 PM
from the first-do-no-harm dept.
Bug Businesses OS X Operating Systems Apple
An anonymous reader writes "Leopard's Finder has a glaring bug in its directory-moving code, leading to horrendous data loss if a destination volume disappears while a move operation is in progress. This author first came across it when Samba crashed while he was moving a directory from his desktop over to a Samba mount on his FreeBSD server."
+ -
story

Related Stories

Submission: Massive Data Loss Bug in Leopard by Anonymous Coward
This discussion has been archived. No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.

Data Loss Bug In OS X 10.5 Leopard 50 Comments More /

 Full
 Abbreviated
 Hidden
More
Loading... please wait.

  • That's silly. (Score:5, Funny)

    by ackthpt (218170) * on Monday November 05 2007, @06:36PM (#21248189) Homepage Journal

    Normally while moving you ensure the copy completed before deleting the original. Apple must be using some discount programmers.

    • Ah, the "outsourcing" coding model.. (Score:5, Funny)

      by eniac42 (1144799) on Monday November 05 2007, @06:53PM (#21248409) Journal
      Advert on Amazon Mechanical Turk:
      Write OS-X compatible application to Move a file between two filesystem devices..
      Time Allotted:: 6 hours. Reward: $10.00..
    • Apple must be using some discount programmers

      Of course not! Don't be a troll.

      Everyone knows that Apple's products Just Work, and that's no different in this case. The files were moved just like you asked, and if you can't find them. well, that's not Apple's fault, is it? You don't blame the contractor who built you home when you lose your keys, do you?

      In any case, you should be using Shadow Copy...er...Time Machine which would have protected you from going and losing track of your own files.

      • Temporal problem (Score:5, Funny)

        by Roger W Moore (538166) on Monday November 05 2007, @09:59PM (#21250203) Journal
        In any case, you should be using Shadow Copy...er...Time Machine which would have protected you from going and losing track of your own files.

        Great! So now not only don't we know where our data is but when it is. Perhaps in a week or two's time the data will materialize in the folder it was supposed to be moved to with an accompanying "whorping" sound coming from the speakers?

      • Re:That's silly. (Score:5, Informative)

        by Hatta (162192) on Monday November 05 2007, @06:59PM (#21248483) Journal
        Why don't you just use rsync [anu.edu.au]?

        • Re:That's silly. (Score:5, Informative)

          by Frothy Walrus (534163) on Monday November 05 2007, @07:04PM (#21248575)
          Oh yeah, rsync. Is that one still broken on 10.5? Apple's build of rsync on 10.4 consistently choked on a few files when my dad started using it on his Mac Pro.

            • Re:That's silly. (Score:5, Funny)

              by delire (809063) on Tuesday November 06 2007, @06:01AM (#21252805)

              resource fork... from Apple's rsync manpage: -E --extended-attributes copy extended attributes, resource forks
              With cryptic commands just to copy files Apple will ALWAYS have ~.001% market share. When will Apple get it into their head that programmers shouldn't design user interfaces?

              Why doesn't Apple just copy the way Ubuntu does it and get with the 2000's?

              Pride is what it is. That damned pride!

      • Re:That's silly. (Score:5, Insightful)

        by Taagehornet (984739) on Monday November 05 2007, @07:55PM (#21249115)

        this is not what current generation of typical user would do, but I believe they should be educated on this anyway

        Reeducate the user, you say. Surely you must be joking, right?

        Let's ignore for a moment that Leopard may have a few bugs that will have to be ironed out. That's only to be expected with *_any_* newly released OS and the reason why no sane person would ever dare to update the OS on a mission critical machine within the first few months of the release.

        However, if you can't rely on your OS to perform a simple file move without risking data corruption, then the right solution is definitely not to verify every single operation by hand. Automating tedious tasks is exactly what computers do best, and that the OS ensures the integrity of the copy before throwing away the original is definitely something you should expect.

      • Re:That's silly. (Score:5, Informative)

        by ackthpt (218170) * on Monday November 05 2007, @07:19PM (#21248739) Homepage Journal

        So what happens if you're moving a 120GB folder one directory level on a 150GB disk?

        Typically if you are moving within the same logical device the file pointer is moved and no copying need take place.

        When moving to another device your code reads and writes, within a loop and traps exceptions (such as the device suddenly vanished, where the OS should raise an exception and your application traps it.) A wide variety of errors could occur while moving and in the event any of them happen the user should be notified in an appropriate manner and the original data not deleted.

        I've written a number of applications which moved files or data between databases and it's fundamental your application is on the watch for any problems. Not to have an exception raised or to trap any and all, well, that's simply an inexcusable lapse.

        This sort of thing is extremely critical if you happend to be defragmenting a disk drive. Long before Macs and PCs we had to defrag our mainframe drives and the applications which did the work were quite careful. Often the best practice, if you had the resource of a second drive, was to simply defrag to a new drive then re-assign the new dist as the original.

        • Re:That's silly. (Score:5, Informative)

          by Matt Perry (793115) on Tuesday November 06 2007, @01:00AM (#21251417)

          I think you misunderstand something.
          No, I understand perfectly.

          I know for a fact that I've successfully canceled FTP-moves on Windows (brand); I also know for a fact that I've successfully canceled FTP-moves on all modern instances of the Windows OS with various FTP clients (programs).
          I'm not talking about other clients. I'm talking about using Windows XP as the client.

          What program were you using that fragged your FTP server directory exactly?
          Windows XP, service pack 1.

          I suspect you're a fud-fucking idiot
          You need to work on your social skills.

          although at the moment, I'm interested to hear if there's more than fabrication here. Pray tell: how can I reproduce this bug?
          You might need to use WinXP service pack 1. I'm unable to reproduce it now on my home machine which has SP2 installed. At work we've just upgraded to service pack 2 within the last six months. Anyway, about a year ago I was trying to move nearly 40 gigs of data from a Solaris server at another site to my local machine. The Solaris server was an FTP server and that's the only access I had to it. I opened Internet Explorer and typed in ftp://servername [servername] and was prompted for a username and password which I entered. Then I saw the files in the explorer view just like I see files on my local machine. Since I was moving so much data I decided to do it one top-level directory at a time. To move the files I selected the folder, dragged it to a folder on my local machine, and held down shift when I let go of the mouse pointer so it would move instead of copy. I noticed that Windows went through three steps when it moved from the FTP site:
          1. A dialog appears with the title Moving... and it says it's calculating how much time it'll take. This took a while itself.
          2. Then the dialog says it's moving the files and the progress meter shows the progress
          3. After the progress meter fills up, it goes to empty again and then says it's deleting the files it moved
          I did this with most of the folders until I reached one that had about 18GB of data in it. I performed the same procedure as above to start the move and after it was copying files for about a minute or two I realized that I was moving them to the wrong location (my desktop instead of a directory on my C drive). At work our desktops are retargeted to a file server and not our local hard drives. I knew that by copying to the desktop I would hit my quota limit on the server whereas if I had copied to the C drive like I did with the other folders I had no limit aside from the free hard drive space. Not copying there in the first place was just a mistake. I clicked cancel to stop the transfer. When I did that, the dialog that had the progress bar changed to the empty progress bar and said it was now deleting. By the time I realized what was going on, it had deleted that folder and all of its content from the FTP site.

          Now this could have been some weird bug or interaction between the fact that I was using a machine with SP1 instead of SP2, that my desktop and profile were retargeted to another machine, or that I was moving so much data. It wasn't a lot of files as these were data files for desktop publish programs for some brochures and catalogs, along with large print-ready images. I don't know what items worked together to cause the problem. I do know that it did happen and that I had to deal with the IT group at our other site (that had the Solaris server), open the helpdesk ticket to get them to restore the files from backup, wait for them to get around to it, etc. It was a huge pain and delayed what I was working on, causing grief for myself and my internal customer. One thing's for sure; I'll never use the built-in Windows FTP client again.

  • Tiger has this problem as well!!! (Score:5, Informative)

    by itsybitsy (149808) on Monday November 05 2007, @06:36PM (#21248191)
    I lost a huge amount of data being MOVED (not copied) from one volume into a virtual volume DMG file. Lost and gone forever, lots of important files. What happened? I simply bumped the laptop Mac Book Pro during the move... zap... gone forever. The DMG file was blank! Yes, complely zero bytes except for a bit of header non-file data. It sucks bad.

  • You just don't get it... (Score:5, Funny)

    by juanfgs (922455) on Monday November 05 2007, @06:42PM (#21248275) Homepage
    This is the new Leopard "iLostMyFrigginFiles" feature, next version they will add a badass black hole effect when it does that!

  • Par for the course? (Score:5, Informative)

    by GoRK (10018) <johnlNO@SPAMblurbco.com> on Monday November 05 2007, @06:45PM (#21248311) Homepage Journal
    No offense meant here, but normal move/copy operations are traditionally highly destructive events on MacOS anyway. For instance there is absolutely no simple way to merge two folders contents together on the mac. If you drag a folder called "Documents" into your home directory and click on "OK", the Mac OS will happily delete your entire documents folder. I was reminded of this enormous frustration while recovering from some multi-volume backups recently, having to resort to an obscure OS X commandline tool 'pax' and Leopard's newfound support of hardlinks to make some simple file copies play nice and not unnecessarily consume 3 times the disk space they should have.

    For all of the flack the Windows file copy interface gets, it is both safer and more flexible than trying to use the Finder: an interface that makes file management so stupefying it becomes impossible.

    • Re:Par for the course? (Score:5, Interesting)

      by Blakey Rat (99501) on Monday November 05 2007, @07:04PM (#21248553)
      That's left over from the original spatial Finder design in Mac Classic. Apple hasn't really decided whether they want to get rid of the spatial interface, so instead they've made this horrible frankenstein half-spatial, half-browser interface which pretty much everybody hates.

      Doing a "replace" for that operation makes sense in a spatial system because all spatial icons are treated the same way. You'd wouldn't expect dragging a Word file named "happy.doc" into a folder already containing a "happy.doc" to perform a merge operation; so why would you expect that with a folder in the same situation?

      That said, if you've never used Mac Classic, you'd think OS X has nothing but a browser interface, in which case all metaphors and ideals are out the damned window, and the OS might as well do a merge operation. Since you most likely came from Windows, or a Linux environment ripped-off from Linux, you'd expect dragging identically-named folders together to do a merge operation because that's what you're used to.

      Apple needs to make up its mind what Finder is. It gets worse and worse every version.

    • Re:Par for the course? (Score:5, Insightful)

      by nine-times (778537) <nine.times@gmail.com> on Monday November 05 2007, @07:09PM (#21248635) Homepage

      If you drag a folder called "Documents" into your home directory and click on "OK",

      To be fair, I don't think it asks you whether it's ok to move that directory. It will warn you that it's going to replace that folder, and the buttons will either say, "Replace" or "Stop". It's not that ambiguous.

      The only thing that makes it problematic is if you're accustomed to working in a file manager that will automatically merge directories, then you might think it's going to merge when it's actually going to replace. I would say that neither behavior is "wrong", but you certainly can get unhappy results if you're expecting one behavior and get another.

      Honestly, it took me a little while to get used to it, but now that I expect it, it's fine. Usually, if I'm doing anything complicated with copying/moving lots of stuff recursively, I'm going to want to use a command line anyhow. In the command-line, "cp" and "mv" work in normal unix fashion.

    • Re:Par for the course? (Score:4, Insightful)

      by node 3 (115640) on Monday November 05 2007, @07:20PM (#21248765)

      For instance there is absolutely no simple way to merge two folders contents together on the mac.
      Open a folder, "Select All", drag into destination folder.

      If you drag a folder called "Documents" into your home directory and click on "OK", the Mac OS will happily delete your entire documents folder.
      It does state this is going to happen in the window with the OK button.

      I was reminded of this enormous frustration while recovering from some multi-volume backups recently, having to resort to an obscure OS X commandline tool 'pax' and Leopard's newfound support of hardlinks to make some simple file copies play nice and not unnecessarily consume 3 times the disk space they should have.
      If you were going with the command line, you could have just used "cp" or "rsync". Your mention of hardlinks is perplexing, though. OS X has supported hard links forever. It recently added support for hard linking folders (extending hard links beyond the standard). Since you're comparing with Windows, does Windows somehow know if you are copying a file that's identical to one that already exists, and makes a hard link? I'm fairly sure it doesn't.

      For all of the flack the Windows file copy interface gets, it is both safer and more flexible than trying to use the Finder
      How so? This bug aside, I don't see how it's safer *or* more flexible. The difference--the *only* difference--here is whether folders replace or merge. Windows isn't more flexible, as it does one way, not both. As for safer, they both tell you when something destructive is going to happen.

      the Finder: an interface that makes file management so stupefying it becomes impossible.
      Impossible? Really? All because it replaces folders (and tells you, with a chance to abort) instead of merging them?

      • Re:Par for the course? (Score:5, Insightful)

        by JunoonX (1171445) on Monday November 05 2007, @07:07PM (#21248607)
        When two folders, both named "Documents", where one is dragged and dropped into the home directory containing another "Documents" folder, Windows prompts if you want to replace content from the dropped folder on to the one being dropped on. At this point, if any files with same name are encountered, they will be replaced with the one from the first directory; however, all other files in folder will stay intact.

      • Re:Par for the course? (Score:5, Informative)

        by TheNetAvenger (624455) on Monday November 05 2007, @08:16PM (#21249347)
        I haven't noticed different behavior in any version of Windows. How do you merge similarly named folders during a copy/move in Windows? In my experience you get the same "Do you want to replace this?" type prompt you get in OS X.


        Serious answer...

        XP offers a basic, do you want to replace folder and a basic do you want to replace files. (Very generic, but more than OS X does)

        Vista on the other hand, asks if you want to replace/merge folder and then if conflicting files are found in the folders it asks you if you want to overwrite the files, don't overwrite them, or create a second copy of the file in the same folder. File by File and Folder by Folder if you want, or you can hit a check box to mimic your response for all file and folders if you are moving a lot of files/folders.

        It also will show you the two versions of the file or folder so you can make a decision based on which files/folders are newer, and you even get a thumbnail of the file for documents and pictures to be sure you are keeping the one you want.

        (Very simple interface, but more has the functionality of the power geek that was always left to using tools like XCopy in the past.)

        This is one of the 'little' Vista features that doesn't get any press, but is a lifesaver for people that move around a lot of data, as you can merge and update folders and files much easier.

        Stuff like this is the reason I said if MS did a 'new features' list like the pety list Apple did with their 300 new features, Vista would have several thousand new features to list.

        (Again MS's marketing sucks, since most people don't even know stuff like this exists in Vista, and it is both powerful, easy, and just works.)

  • Take advantage of Time Machine (Score:4, Funny)

    by tji (74570) on Monday November 05 2007, @06:49PM (#21248363)

    Not to be glib, but.. This would be a great demonstration of the value of "Time Machine" backups. Time Machine is not perfect, but it is a good start on a backup system well integrated into the OS. The example problem, data loss, would be really easily recovered via Time Machine.

    Beyond the basics that every decent backup app does, the things I like about Time Machine are:

    - Integration into Applications. For example: "Show me what my iTunes library or iPhoto library looked like last Thursday"

    - Integration into OS install. In the case of disk failure, recovery to previous state is simple - rather than multi-step with a separate backup app.

    Some things that need improving:

    - Better handling of file exceptions. I keep work data in encrypted disk volumes (DMGs). If I change one byte, the whole huge file needs to be backed up as each change is detected (generating MANY copies of that big DMG). The only other choice is to say "ignore this file/directory". Same thing applies to any large file, such as a VMware VM file. A better option would be to say "Back this file up, but only keep 'n' versions".

    - Time Machine has gotten twice, pegging the CPU/fans on my MacBook Pro.

  • Wow (Score:5, Interesting)

    by Zebra_X (13249) on Monday November 05 2007, @06:52PM (#21248393)
    Unbelieveable. Forgot to check the result of the copy operation eh? So basically this is a catastropic defect for people who deal with very large media files to and from remote stores or people who deal with virtual machine images.

    Back in the day when I used to use my mac I dropped a directory (A) into another directory (B) but there was an existing directory (C) with the same name as (A). The finder asked me something, I clicked OK. I was dismayed to find that the dialog had asked me "Would you like to replace directory C, with A?" - Why on earth would that ever be the default option for a directory move? From the users perspective you aren't really moving the directory, the intention is to move the files, thus the sane response would be to merge A with C not replace it.

    Whatever.

  • "haha" (Score:5, Interesting)

    by MutantEnemy (545783) on Monday November 05 2007, @07:31PM (#21248879) Homepage

    Why is every destructive computer bug that happens tagged with "haha"?

    Data Loss Bug In OS X 10.5 Leopard
    bug, macosx, apple, haha

    Symantec Updates Cause Chaos in China
    haha, security, bug, windows, feature

    Banner Ad on Myspace Serves Adware to 1 Million
    haha, myspace, pwnd, security, adware

    Ubuntu May Be Killing Your Laptop's Hard Drive
    linux, haha, storage, bug, spam

    Islamists exploit buffer overflow, hack U.S. nuclear command; world doomed
    eschaton, religion, waronterror, haha

    OK, I made one of those up. But it doesn't even matter what OS or company is responsible for the problem - whoever makes the tags seems to take great delight in all computer snafus. How does the tagging system work anyway? It's always been mysterious to me.

  • I had this happening to me with WinXP aswell (Score:5, Informative)

    by Lisandro (799651) on Monday November 05 2007, @09:08PM (#21249805)
    That's it, moving a file, network failure and poof, moved files deleted from the source. Most probably a bizarre coincidence, but after it happened a few times i now copy and then delete when working on a SMB network.

    (Or maybe Samba has something to do with it?)

    • Wait... (Score:5, Funny)

      by ackthpt (218170) * on Monday November 05 2007, @06:40PM (#21248241) Homepage Journal

      It's kind of strange that this didn't come up while people were beta testing OS X 10.5. Samba is used in many places. I hope they get it fixed soon.

      Exactly which decade did you fall into your recently awkened from coma in? Testing? Testing? Nobody tests anything anymore, they just go play with all the new toys and stare at the eye-candy. Actual mundane, humdrum testing? That's an SEP if ever I didn't see one.

    • Re:Wierd (Score:5, Funny)

      by cnettel (836611) on Monday November 05 2007, @06:40PM (#21248253)
      All fanboys were just happy with how blazing fast file copy was compared to Vista. The non-fanboys tried to check the real size of the dir by copying it to a Vista machine afterwards, but the progress bar got stuck on 413 hours left and counting, so they couldn't file the bug in time.

    • Re:Terrible bug (Score:5, Insightful)

      by Liquidrage (640463) on Monday November 05 2007, @06:43PM (#21248281)
      Yes, that's easily avoidable. If you know it exists. And hopefully the first time you get bitten by this it isn't something critical. Would be garbage to have to work around that one.

      TFA looked decent in it's details. Even step by step recreation. But it's a pretty serious bug, that as you mention, *needs* to be fixed quick and I didn't see any other sources confirming it.

    • Re:Terrible bug (Score:4, Interesting)

      by El Lobo (994537) on Monday November 05 2007, @07:00PM (#21248507)

      but I'm sure Apple will have this fixed pretty quick.
      Well, the problem is, this bug exists even in TIGER and has been repported many times! And no, not fixed yet. Abble is a coorporation like any other, and not the superpower that some users seem to think they are.

      • Re:Terrible bug (Score:5, Insightful)

        by Knara (9377) on Monday November 05 2007, @07:11PM (#21248653)

        You're asking if a bug wherein entire folder hierarchies can go *poof* in the event a network share drops should be considered critical? Are you serious?

      • Re:Terrible bug (Score:5, Insightful)

        by TheNetAvenger (624455) on Monday November 05 2007, @08:02PM (#21249207)
        is this even that critical? Doesn't leopard have the time machine in it? Can't you just go back and get your files out of the time machine if they were that important?


        Only if a backup of the files was run, which is a requirement of Time Machine.

        If this was Vista, then there would be a good chance to use 'previous versions' to recover the folder data, as it does not 'solely' rely on external backup for timeline file recovery. Vistas use volume level file version snapshots(a feature of NTFS that HPS+ doesn't support), so there are backups even on the drive if it hasn't ever been backed up.

        (Remember Time Machine usually only runs once an hour, and all versioning or changes made in that hour are never kept or tracked.)

        -PS Not trying to troll, but this is a perfect example of the difference between Vista's previous versions and Apple's Time Machine I have tried to point out in the past.

        Vista does both volume level backups and external backups, unlike just external backups like Time Machine does.

        See why IT people prefer Vista's method, even if a backup hasn't run that hour, even users themseleves can access files and folders rather easily that got deleted or changed. And since this has been around Since Windows 2003 Server, in corporate environments, even XP users accessing 2003 servers have had this feature for over 4 years now.

        Vista's claim to fame is that it enables these features on the local hard drive, and also integrates with the Vista backup system, so file versions appear from the backups in addition to the 'snapshot versions' on the main hard drive.

        • That's missing the point... (Score:5, Interesting)

          by megaduck (250895) <dvarvel&hotmail,com> on Monday November 05 2007, @10:48PM (#21250541) Journal
          Speaking as one of those IT people, NTFS is probably one of the coolest pieces of software ever to come out of Redmond. ACLs, alternate data streams, directory junctions, single-instance stores, shadow copy, the list of useful features is huge. Even more surprisingly, it works pretty much as advertised. Frickin' cool.

          There's another angle, though. On paper, Vista's NTFS-based backup technology walks all over Time Machine. However, the USABILITY of Vista's technology is crap. This morning, I enabled Time Machine by plugging in a USB drive and clicking "Use as Backup Disk" when prompted. To do restores, I launch the cleverly named "Time Machine" application. I've already used it twice today just because it's fun to watch the spacey animations.

          Compare that to Vista's clunky "Backup and Restore Center", which you have to use if you want to backup your files on an alternate volume. I guarantee you that using "Backup and Restore Center" is beyond most average users. Sure, it might be "better", but what good is it if it never gets used?

    • Re:I don't understand (Score:5, Informative)

      by t35t0r (751958) on Monday November 05 2007, @07:43PM (#21249007)
      NFS write failure on Linux 2.4, check your data is gone.

      Uhh no. In linux mv's always make sure the data moved then delete the original file (as far back as 2.4). You can test this by dd'ing a large file (use if=/dev/urandom), then run a md5sum on it, then do a mv within the same drive, over nfs, over samba/cifs, to floppy (not sure what happens here because linux caches writes to floppies until umount), to usb drive, whatever. While the mv is in operation just pull the plug on the system (target or source system). Your old file will still be there!

      • Re:I don't understand (Score:5, Funny)

        by myowntrueself (607117) on Monday November 05 2007, @07:39PM (#21248967)
        I agree. I always copy then delete, especially when dealing with network shares.

        Despite its many shortcomings, Windows ME (*NOT* the more recent ME2) had this truly wonderful feature where if you delete something from a network share it would *copy* the data across the network into your trash folder.

        Really handy when you delete 10G of data on a network share and your local hard drive has 5G available and you are on a 10mbps network.

All trademarks and copyrights on this page are owned by their respective owners. Comments are owned by the Poster. The Rest © 1997-2009 Geeknet, Inc.