Some things to be aware of when running Octave scripts on ATLAS:

Octave path searching

When an Octave script calls a non-built-in function, Octave will look through, first the directory containing the script, and then $OCTAVE_PATH for any file (presumably with a .m extension) that contains the definition of that function. In doing so, Octave (at least as of version 3.2.4) will access every file of every directory in the search path. Let's say you're running 10000 jobs, each generating 3 files (condor stdout, condor stderr, plus a result file), and these files are in the same directory as your script. If your script then calls a non-built-in Octave will then spend >99.9% of its time looking through 30000+ files trying to find that function, and your jobs will grind to a halt - and you'll be seriously hammering your home filesystem!! (The author speaks from experience here!)

So, it's important to make sure that none of the directories in Octave's search path (particular the directory containing your script) contains a large number of files. The easiest way to do this is to send all job files (condor stdout, condor stderr, results file) into subdirectories of the directory you're running jobs from. Here's an toy setup:

$ ls

$ cat job.sub
universe = vanilla
executable = octave
arguments = -qfH ./script.m --output results/results.dat.$(Process)
log = /local/user/$USER/job.log
output = condor/condor.out.$(Process)
error = condor/condor.err.$(Process)

-- KarlWette - 01 Nov 2011

DocumentationForm edit

Title Running Octave scripts on ATLAS
Description Some tips and pitfalls when running Octave scripts on ATLAS
Tags octave
Category User
Topic revision: r6 - 10 Feb 2012, ArthurVarkentin
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Foswiki? Send feedback