Backup Strategy for users' $HOME
Rolling snapshots
Each thumper will loop over all users' file systems and create a new snapshot every 6 hours. Old snapshots will be kept for at least a week, depending on disk space maybe even longer. If a user accidentally deletes a file which was present when an older snapshot has been created, she can retrieve the file by looking under her
~/.zfs
directory and copy the needed file back to her "standard" view of her file system.
Backup to remote system
Independent of the snapshot creation, a backup script will run on every Sun thumper (the physical place of the
$HOME
file systems) and rsync the data from the latest snapshot to a remote Sun thumper. This will ensure that in the event of a total system failure, we should still be able to have most data available on a different physical machine.
Gory details
The backup script needs to perform the following simple steps:
- loop over all local users on this box ->
$USER
- locate latest snapshot of
$USER
- decide which server is the target server for the backup
- rsync latest local snapshot to target machine
- in case of a fatal error, mail admins, otherwise update web page/motd
- finish loop
result message
Number of files: 25437390
Number of files transferred: 6123656
Total file size: 2841932645178 bytes
Total transferred file size: 224324409610 bytes
Literal data: 224324409610 bytes
Matched data: 0 bytes
File list size: 1142309559
File list generation time: 16784.539 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 225769269238
Total bytes received: 135500641
sent 225769269238 bytes received 135500641 bytes 2609730.19 bytes/sec
total size is 2841932645178 speedup is 12.58
--
CarstenAulbert - 25 Nov 2009