Carsten's collected bits and pieces

salt server attachments

redirect

systemctl stop salt-minion && cd /etc/salt/ && mv pki/minion/minion_master.pub pki/minion/minion_master.pub.saltmine && sed -i 's/saltmine/salt-server-carsten/' minion.d/* && systemctl start salt-minion

and back again:

systemctl stop salt-minion && cd /etc/salt &&mv pki/minion/minion_master.pub.saltmine pki/minion/minion_master.pub && sed -i 's/salt-server-carsten/saltmine/' minion.d/* && systemctl start salt-minion

Debian live

log into cowbuilder on bob,
apt-get install live-build
mkdir /live
cd /live
lb config --apt-recommends false --architectures amd64 --clean --memtest none --verbose --bootstrap-flavour minimal
lb build

Currently this results in a 120 MB Wheezy ISO, todo: testing and customizing

E@H post-run file re-organization

Eisntein@home result files are organized in tarballs which are written daily. Each tarball contains results called 'canonical' (first choice) or 'valid' (second choice). Ideally both files should be the same as seen by the project specific validator. Tarball sizes vary a lot, e.g. for S6Bucket they can range from a few hundred kByte up to several GByte in size.

Re-organizing the results into frequency bands should enable post processing to be much smoother than before, thus this task.

To achieve this, the following steps are performed:

  1. parse command line options and act...
  2. read state file - in case previous run failed
  3. iterate over list of found tarballs
    1. unpack tarball to /dev/shm
    2. iterate over result file
      1. FrCheck result file
      2. append lines from result file to f-based results
  4. close all open file handles/flushing results to disk

These steps are performed by a script which can be found here.

This should be followed by an independent check (Bernd did this for S6Bucket) to ensure ALL candidates are retained.

Finally, all bands created should be sorted by the most useful column(s) and compressed (gzip) to save space and necessary IO during file reads.

apt-get --yes --no-install-recommends install devscripts 
cd /data/s6bucket/bin
rm -rf /dev/shm/EatH-* 
rm -f /data/result/* /data/state.*
git pull
screen -S S5GC1 -d -m bash -c "annotate-output ./convert2freqbased.pl --verbose --noprompt --input=/data/work --output=/data/result --state=/data/state.$(hostname) --freqstart=50.1174637425192 --band=0.05 --label="S5GC1_%08.3f.band" --validator=/data/s6bucket/bin/validator_test_S5GC1B | tee /data/S5GC1.log"

set up working pool for ra05..ra17 (say on ra05 in /data/tmp/):

mkdir -p /data/tmp; cd /data/tmp/
find /home/einstein/UWM-backup/S5GC1/results/ -type f -name "canonical*tar" | rl | split -d -l $((270/12)) - ra
for i in $(seq -w 12 -1 0); do mv ra$i ra$(printf "%02d" $((5+${i#0}))); done
for i in *; do scp $i $i:/data/; ssh $i "cd /data/work; rm -f /data/work/*; cat /data/$i | xargs -n 1 ln -s"; done

My Prefs

  • Set NOWYSIWYG = 0
  • Set SKIN = natedit,pattern

UserForm edit

FirstName Carsten
LastName Aulbert
OrganisationName Albert-Einstein-Institute
OrganisationURL http://www.aei.mpg.de/
Profession SysAdmin, Physicist
Country Germany
State
Address Callinststraße 38
Location Hannover
Telephone +49 511 762 17185
VoIP
InstantMessaging (IM)
Email carsten.aulbert@aei.mpg.de
HomePage
Comment
Topic attachments
I Attachment Action Size Date Who Comment
carsten_aulbert.vcfvcf carsten_aulbert.vcf manage 430 bytes 08 Sep 2008 - 18:35 CarstenAulbert VCF
Topic revision: r33 - 26 May 2020, CarstenAulbert
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Foswiki? Send feedback