Clean the local scratch space
Sometimes you want to get rid off all temporary stuff you left on any compute node (e.g. you just finished a big search, all results are stored in your home and you just want to be nice to other users and give them more space to breath).
The brutal way
Just run this from any head node (atlas1, atlas2, ...) - CAVEAT
this will delete all files under your local directory on any compute node!
dsh -aMF50 'cd /local/user/`whoami` && ls | xargs -r rm -rf'
What will this do?
dsh will launch 50 parallel ssh sessions (
-F50) to all compute nodes (stored under
-a) and will display the output of any command prefixed with the node's name (
- on each node, the shell is instructed to change into the directory
`whoami` will return automatically your user name and relieve me from writing this tutorial once for every user we have
- in this directory it will call
ls and pipe the output to
xargs. If there is any input it will call
rm -rf with the maximal number of arguments possible and repeat this until all files and directories are gone.
The more cautious way
dsh -aMF50 'find /local/user/`whoami` -mtime +1 -type f -mindepth 1 | xargs -r rm -f; \
find /local/user/`whoami` -mtime +1 -type d -mindepth 1 | xargs -r rmdir'
This command will also run 50 parallel "threads" on the cluster, while first finding and removing all files (
which are at least 1 day old (
). The second part will then delete all directories which are empty and also more than a day old (Thanks ChrisP
for the suggestion)
- 15 Apr 2010
| Title || Clean the local scratch directories of compute nodes |
| Description || With a simple command you can clean your scratch space on all Atlas' compute nodes |
| Tags || clean, dsh, xargs |
| Category || User |