Backup Strategy for users' $HOME

Rolling snapshots

Each thumper will loop over all users' file systems and create a new snapshot every 6 hours. Old snapshots will be kept for at least a week, depending on disk space maybe even longer. If a user accidentally deletes a file which was present when an older snapshot has been created, she can retrieve the file by looking under her ~/.zfs directory and copy the needed file back to her "standard" view of her file system.

Backup to remote system

Independent of the snapshot creation, a backup script will run on every Sun thumper (the physical place of the $HOME file systems) and rsync the data from the latest snapshot to a remote Sun thumper. This will ensure that in the event of a total system failure, we should still be able to have most data available on a different physical machine.

Gory details

Steps to perform

The backup script needs to perform the following simple steps:

1. loop over all local users on this box -> $USER
2. locate latest snapshot of $USER
3. decide which server is the target server for the backup
4. rsync latest local snapshot to target machine
5. in case of a fatal error, mail admins, otherwise update web page/motd
6. finish loop

result message

Number of files: 25437390
Number of files transferred: 6123656
Total file size: 2841932645178 bytes
Total transferred file size: 224324409610 bytes
Literal data: 224324409610 bytes
Matched data: 0 bytes
File list size: 1142309559
File list generation time: 16784.539 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 225769269238
Total bytes received: 135500641

sent 225769269238 bytes  received 135500641 bytes  2609730.19 bytes/sec
total size is 2841932645178  speedup is 12.58

-- CarstenAulbert - 25 Nov 2009