Simple E@H disk layout tests

Summary

Table with timing results (time in seconds, lower is better):

setup file #Sorted ascending size/file [kB] zfs2lc128 zfs2lc4 zfs2lc1024 zfs2lc1024a
O1 tar 23488 780.0 27.8 53.2 73.4 74.3
FGRPB1 tar 28195 0.98 2.4 2.8 2.4 2.6
BRP6 tar 48759 4,9 3.5 3.9 3.5 3.4
S6CasA tar 74347 141.1 27.9 34.5 24.0 23.9
create fan-out - - 1.4 1.3 1.3 1.2
BRP6 mv - - 3.0 3.5 3.1 3.0
FGRPB1 mv - - 2.9 2.7 2.8 3.3
O1 mv - - 2.7 3.5 3.4 3.0
S6CasA mv - - 7.0 7.0 6.7 6.4
hardlink A->B - - 17.8 18.9 16.2 17.5
tarball create - - 305.7 376.8 151.2 151.3
remove - - 735.2 1118.9 59.3 57.5
untar - - 13.7 38.5 13.3 14.7

Setup & Results

zfs2lc128

ZFS with 2 disks per mirror, cache and log device on NVME, 128kB record size

Create 26 fan out dirs: Runtime: 1.4s
/srv/BRP6_canonical_results_2016_01_20.tar - 230 MiB - 48759 files:
     tar: Runtime: 2.7s
     umount: Runtime: 0.8s
Moving 48759 files into fan-out:
     mv: Runtime: 2.0s
     umount: Runtime: 1.0s
/srv/FGRPB1_canonical_results_2016_03_23.tar - 27 MiB - 28195 files:
     tar: Runtime: 1.5s
     umount: Runtime: 0.9s
Moving 28195 files into fan-out:
     mv: Runtime: 2.1s
     umount: Runtime: 0.8s
/srv/O1AS20-100I_canonical_results_2016_03_30.tar - 17882 MiB - 23488 files:
     tar: Runtime: 25.9s
     umount: Runtime: 1.9s
Moving 23488 files into fan-out:
     mv: Runtime: 1.9s
     umount: Runtime: 0.8s
/srv/S6CasA_canonical_results_2014_01_21.tar - 10245 MiB - 74347 files:
     tar: Runtime: 25.7s
     umount: Runtime: 2.2s
Moving 74347 files into fan-out:
     mv: Runtime: 6.1s
     umount: Runtime: 0.9s
hard linking A to B: Runtime: 17.8s
Creating single tarball in-place of fanned-out files (10% of all): Runtime: 305.7s
Remove everything: Runtime: 735.2s
Untar large tarball again: Runtime: 13.7s

zfs2lc4

ZFS with 2 disks per mirror, cache and log device on NVME, 4kB record size

Create 26 fan out dirs: Runtime: 1.3s
/srv/BRP6_canonical_results_2016_01_20.tar - 230 MiB - 48759 files:
     tar: Runtime: 3.0s
     umount: Runtime: 0.9s
Moving 48759 files into fan-out:
     mv: Runtime: 2.5s
     umount: Runtime: 1.0s
/srv/FGRPB1_canonical_results_2016_03_23.tar - 27 MiB - 28195 files:
     tar: Runtime: 1.6s
     umount: Runtime: 1.2s
Moving 28195 files into fan-out:
     mv: Runtime: 1.8s
     umount: Runtime: 0.9s
/srv/O1AS20-100I_canonical_results_2016_03_30.tar - 17882 MiB - 23488 files:
     tar: Runtime: 44.8s
     umount: Runtime: 8.4s
Moving 23488 files into fan-out:
     mv: Runtime: 2.8s
     umount: Runtime: 0.7s
/srv/S6CasA_canonical_results_2014_01_21.tar - 10245 MiB - 74347 files:
     tar: Runtime: 28.8s
     umount: Runtime: 5.7s
Moving 74347 files into fan-out:
     mv: Runtime: 6.2s
     umount: Runtime: 0.8s
hard linking A to B: Runtime: 18.9s
Creating single tarball in-place of fanned-out files (10% of all): Runtime: 376.8s
Remove everything: Runtime: 1118.9s
Untar large tarball again: Runtime: 38.5s

zfs2lc1024a

ZFS with 2 disks per mirror, cache and log device on NVME, 1MB record size

Create 26 fan out dirs: Runtime: 1.3s
/srv/BRP6_canonical_results_2016_01_20.tar - 230 MiB - 48759 files:
     tar: Runtime: 2.7s
     umount: Runtime: 0.8s
Moving 48759 files into fan-out:
     mv: Runtime: 2.1s
     umount: Runtime: 1.0s
/srv/FGRPB1_canonical_results_2016_03_23.tar - 27 MiB - 28195 files:
     tar: Runtime: 1.5s
     umount: Runtime: 0.9s
Moving 28195 files into fan-out:
     mv: Runtime: 2.1s
     umount: Runtime: 0.7s
/srv/O1AS20-100I_canonical_results_2016_03_30.tar - 17882 MiB - 23488 files:
     tar: Runtime: 72.2s
     umount: Runtime: 1.2s
Moving 23488 files into fan-out:
     mv: Runtime: 2.6s
     umount: Runtime: 0.8s
/srv/S6CasA_canonical_results_2014_01_21.tar - 10245 MiB - 74347 files:
     tar: Runtime: 22.8s
     umount: Runtime: 1.2s
Moving 74347 files into fan-out:
     mv: Runtime: 6.0s
     umount: Runtime: 0.7s
hard linking A to B: Runtime: 16.2s
Creating single tarball in-place of fanned-out files (10% of all): Runtime: 151.2s
Remove everything: Runtime: 59.3s
Untar large tarball again: Runtime: 13.3s

zfs2lc1024

ZFS with 2 disks per mirror, cache and log device on NVME, 1MB record size, atime off

Create 26 fan out dirs: Runtime: 1.2s
/srv/BRP6_canonical_results_2016_01_20.tar - 230 MiB - 48759 files:
     tar: Runtime: 2.7s
     umount: Runtime: 0.7s
Moving 48759 files into fan-out:
     mv: Runtime: 2.0s
     umount: Runtime: 1.0s
/srv/FGRPB1_canonical_results_2016_03_23.tar - 27 MiB - 28195 files:
     tar: Runtime: 1.6s
     umount: Runtime: 1.0s
Moving 28195 files into fan-out:
     mv: Runtime: 2.5s
     umount: Runtime: 0.8s
/srv/O1AS20-100I_canonical_results_2016_03_30.tar - 17882 MiB - 23488 files:
     tar: Runtime: 73.1s
     umount: Runtime: 1.2s
Moving 23488 files into fan-out:
     mv: Runtime: 2.4s
     umount: Runtime: 0.6s
/srv/S6CasA_canonical_results_2014_01_21.tar - 10245 MiB - 74347 files:
     tar: Runtime: 22.5s
     umount: Runtime: 1.4s
Moving 74347 files into fan-out:
     mv: Runtime: 5.8s
     umount: Runtime: 0.8s
hard linking A to B: Runtime: 17.5s
Creating single tarball in-place of fanned-out files (10% of all): Runtime: 151.3s
Remove everything: Runtime: 57.5s
Untar large tarball again: Runtime: 14.7s

Templates


Topic attachments
I Attachment Action Size Date Who Comment
bench.shsh bench.sh manage 2 K 07 Apr 2016 - 06:21 CarstenAulbert simple benchmark script
This topic: ATLAS > WebHome > EatHDiskConfigTests2
Topic revision: 08 Apr 2016, CarstenAulbert
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Foswiki? Send feedback