Saturday, 8 March 2008

Time Machine Wireless Backup Continued...

This post is now out of date!

Check out the latest info on my main wireless Time Machine post.

Old stuff below for historical record only...

So I promised in my previous post that I would give an update on how I got my wireless Time Machine backups working from a Mac running 10.5.2. I’ve also found a useful utility to control how often Time Machine runs (and found out where the appropriate property list files are kept) and I understand a little more why Apple have made this whole shebang an unsupported option…

Setting up from 10.5.2

The setup process is a little convoluted, but it’s not too hard to follow. What you do need, however, is a second Mac on which you can share some space using Apple’s built-in file sharing.

The first trick is to create your initial Time Machine backup on the second Mac, and then copy that backup across to the unsupported share. Once it’s there, Time Machine will happily accept it and continue backing up.

The second trick is to make that initial backup as small as possible so you don’t have to wait for the setup or copy a huge amount from the second Mac to your wireless backup repository. Michael on pastebin came up with the idea of changing your Time Machine preferences to exclude your local hard disks, but to include a very small mounted disk image instead.

You can create this image using Disk Utility (use the “Mac OS Extended (Journaled)” format) and once mounted, go to the Time Machine preferences and remove it from the “Do not back up” list under the Options… button. While you’re here, add all your local hard disks to the list (just while you create the initial backup).

Set Time Machine going, backing up that empty disk image onto the share on the other Mac (it should only take a few minutes). When it’s done, turn Time Machine off for the moment and copy the files that Time Machine created across to your wireless backup. Be careful when you do so — if you’re copying from HFS to a different disk format, you’ll need to use tar as follows:

tar -czpf backup.tgz *.sparsebundle

This will create the file backup.tgz which you can copy across onto your wireless backup server. Once there, run:

tar -xzvf backup.tgz

…and you should end up with both the sparse bundle folder that you copied, and a ._file of the same name that gives Mac OS X some information about the directory.

Now you can go back to the Time Machine preferences and point it at your wireless server where you copied the files (you have already activated the unsupported network drives haven’t you?). Go into the options and remember to remove your local hard disks from the “Do not back up” list, and you can unmount and delete that empty disk image now as well. Your wireless Time Machine should now be ready to go — just be patient for the first backup.

Control how often Time Machine runs

I’ve finally found a utility that claims to control the frequency and times of the backup run — TimeMachineEditor. The previous application I found, Time Machine Scheduler, wouldn’t even install on my Mac, let alone run, and it seemed to be running its own scheduler rather than tweaking the settings of the existing one.

By looking through the binary of TimeMachineEditor, I also found the property lists that it works on: they’re at /System/Library/LaunchDaemons/ and /System/Library/LaunchDaemons/ I’m not sure if this app tweaks these settings or also runs its own scheduler, but it’s happy to install and run.

Caveat Maxtor

So Felix on Apple’s discussion list has documented one reason why Apple has made backing up to an SMB share an unsupported option. When disk space runs out, Time Machine deletes old backups to make room for the new ones. This is by design — the system automatically thins out hourly backups older than a day, daily backups older than a week and weekly backups as needed to maintain space for new backups.

However, Felix noted that when Time Machine was supposed to be cleaning up space, not only could it not reclaim any space, but it then went on to delete all his old backups without telling him. He put this down to a bug in the sparse bundle space recovery code — the old backups were being deleted but the sparse bundle wasn’t getting any smaller. Indeed, when he compacted the sparse bundle by hand, he got loads more disk space back.

I’m wary of this issue as I’ve noticed exactly the same thing in my own logs. I’m going to keep checking the status of my backups as the disk fills up (though there’s plenty of space at the moment!) and will try to run hdiutil compact <sparse bundle> in time to prevent backups going missing!

Update after applying latest Time Machine system upgrade

I just ran the Time Machine and AirPort Updates v1.0 and this problem has been fixed! My logs now show that when there isn't enough space on the disk, Time Machine will delete the least important backup then eject the disk and compact it to recover the free space before trying again. This is fantastic news and the reports that Apple has enabled Time Machine on USB disks attached to Airport Extreme base stations give me much more confidence in using a third party NAS box. More details in my latest post.


Colton said...

What about those of us without a second Mac?

Adam Cohen-Rose said...

I saw a post somewhere (can't find it now) in which someone started the backup and copied the resulting sparsebundle file before the backup finished with an error. Somehow this got round the error -- I'm not sure how and I didn't try it, as it sounded even more dodgy that my current system!

Alternatively, borrow someone else's Mac for a few minutes to get yourself set up :-)

David said...

I took your "copy the sparsebundle file" bit of info and ran with it. Actually, you don't need to copy the file, just its name. Once you have the name, use disk utility to create a new sparsebundle at the size you need, which will necessarily limit the amount of your back up that can be used.

I just copied the resulting dmg file to you back up (I just dropped it on, even though it's a different hierarchy) and Time Machine was able to find it and started working.

I'm using a MacBook 2.0 Ghz, Leopard 10.5.2 backing up to an HP MediaVault over a wireless G network (Microsoft MN-720 base station I've had for years, and it's been flawless).

Thanks for the help.

Kevin Murphy said...

Just to clarify, does David's sparsebundle-with-defined-size approach get around the problem described by Felix?

Adam Cohen-Rose said...

David's approach gets past the creation of the sparsebundle on 10.5.2 -- cheers David! However, the problem that Felix described is that Time Machine won't reclaim space when deleting old backups (the sparsebundle doesn't get smaller even though files have been removed from it).

Anonymous said...

It seems like every post I read this is the same result in the end. The disk fills up and all backups are lost. I am curious if this is due to the underlying partition not being an HFS+ Partition? I have a linux box I may slap another drive in it and see what happens when it fills up. I will just create a small partition as HFS+ and try and mount it and then share it out using AFP/Avahi. Just curious if anyone else has tried this with any luck?

Adam Cohen-Rose said...

I suspect that the underlying partition format is irrelevant as it's the handling of the sparsebundle that is the issue here. Having said that, please post any results you get as I'm sure people will find them useful!

Has anyone tried the Time Machine and Airport Update with SMB shares? Does this solve the problem with reclaiming space in sparsebundles?

Adam Cohen-Rose said...

See update at the end of the blog post -- Time Machine will now reclaim space in sparsebundles, so it won't delete all your backups when it runs out of room.

Valentin Starke said...

Thank you for your detailed explanations. I have a request for additional information: I have a G5 with a big hard-drive attached to it via firewire. When activating file sharing I can access the hard drive from my MBP Laptop using my wireless network. However, the hard-drive is not proposed as a destination for Time Machine on the MBP. Of course I have activated the support for "UnsupportedNetworkVolumes" but I seem to missing something. Thanks if you can help. Valentin.

Anonymous said...

My Network: I have a server with 400GB drive space running Linux, and a 2 VMWare guests (iFolder and a Cyrus Mail Server)
From here I share a work data drive, and a media drive via SMB so that my wife Vista PC can attach and so I can use my XP tablet and my Mac Mini.with this setup am I better off attaching the external drive to my linux server and backing up to a new SMB share of that disk + local linux backups or can I get timemachine to back up mounted SMB volumes from my Mac? and attach the disk via firewire to my mini?


Adam Cohen-Rose said...

@Valentin: Your setup is slightly different than the one I know about -- you can share your firewire drive over AFP and get Time Machine to find it more easily that way. Try adding an empty file called at the root of your firewire drive and see if that helps.

@Anonymous: I don't quite understand what you want to achieve here. With your existing system and no additional firewire drive, you can back up your mini using Time Machine to an SMB share on your Linux server. Check out my main Time Machine over SMB entry for details. Alternatively, you could attach a new firewire drive to your mini and Time Machine will just work.

Anonymous said...

For what it's worth, I've got an IOCell 351UNE personal NAS device that I wanted to try out. My current time machine backup is a 1TB drive in an ICY Dock firewire enclosure.

Since I had read that TimeMachine wouldn't allow you to create a disk on a NAS device but would use it if you created it locally and copied it over, I decided to try a quick test.

I took my existing TimeMachine disk out of the ICY Dock and put it in the IOCell. I had already installed the IOCell app that let me mount it. It came right up and started working without any other changes. I can view my backup history all the way back to December 08. I forced a current backup and verified that the files are out there.

The IOCell enclosure is currently $65 from It has a USB interface so you can configure TimeMachine to use it the normal way. After that, you can just plug it into the network, installed the Mac drivers and you're in business.

For people who don't have a second Mac, it might be an easier and fairly cheap option.