Simple E@H disk layout tests

Summary

Table with timing results (time in seconds, lower is better):

setup file # size/file [kB] zfs2lc128 zfs2lc4 zfs2lc1024 zfs2lc1024a
create fan-out - - 1.4 1.3 1.3 1.2
BRP6 tar 48759 4,9 3.5 3.9 3.5 3.4
BRP6 mv - - 3.0 3.5 3.1 3.0
FGRPB1 tar 28195 0.98 2.4 2.8 2.4 2.6
FGRPB1 mv - - 2.9 2.7 2.8 3.3
O1 tar 23488 780.0 27.8 53.2 73.4 74.3
O1 mv - - 2.7 3.5 3.4 3.0
S6CasA tar 74347 141.1 27.9 34.5 24.0 23.9
S6CasA mv - - 7.0 7.0 6.7 6.4
hardlink A->B - - 17.8 18.9 16.2 17.5
tarball create - - 305.7 376.8 151.2 151.3
remove - - 735.2 1118.9 59.3 57.5
untar - - 13.7 38.5 13.3 14.7

Setup & Results

zfs2lc128

ZFS with 2 disks per mirror, cache and log device on NVME, 128kB record size

Create 26 fan out dirs: Runtime: 1.4s
/srv/BRP6_canonical_results_2016_01_20.tar - 230 MiB - 48759 files:
     tar: Runtime: 2.7s
     umount: Runtime: 0.8s
Moving 48759 files into fan-out:
     mv: Runtime: 2.0s
     umount: Runtime: 1.0s
/srv/FGRPB1_canonical_results_2016_03_23.tar - 27 MiB - 28195 files:
     tar: Runtime: 1.5s
     umount: Runtime: 0.9s
Moving 28195 files into fan-out:
     mv: Runtime: 2.1s
     umount: Runtime: 0.8s
/srv/O1AS20-100I_canonical_results_2016_03_30.tar - 17882 MiB - 23488 files:
     tar: Runtime: 25.9s
     umount: Runtime: 1.9s
Moving 23488 files into fan-out:
     mv: Runtime: 1.9s
     umount: Runtime: 0.8s
/srv/S6CasA_canonical_results_2014_01_21.tar - 10245 MiB - 74347 files:
     tar: Runtime: 25.7s
     umount: Runtime: 2.2s
Moving 74347 files into fan-out:
     mv: Runtime: 6.1s
     umount: Runtime: 0.9s
hard linking A to B: Runtime: 17.8s
Creating single tarball in-place of fanned-out files (10% of all): Runtime: 305.7s
Remove everything: Runtime: 735.2s
Untar large tarball again: Runtime: 13.7s

zfs2lc4

ZFS with 2 disks per mirror, cache and log device on NVME, 4kB record size

Create 26 fan out dirs: Runtime: 1.3s
/srv/BRP6_canonical_results_2016_01_20.tar - 230 MiB - 48759 files:
     tar: Runtime: 3.0s
     umount: Runtime: 0.9s
Moving 48759 files into fan-out:
     mv: Runtime: 2.5s
     umount: Runtime: 1.0s
/srv/FGRPB1_canonical_results_2016_03_23.tar - 27 MiB - 28195 files:
     tar: Runtime: 1.6s
     umount: Runtime: 1.2s
Moving 28195 files into fan-out:
     mv: Runtime: 1.8s
     umount: Runtime: 0.9s
/srv/O1AS20-100I_canonical_results_2016_03_30.tar - 17882 MiB - 23488 files:
     tar: Runtime: 44.8s
     umount: Runtime: 8.4s
Moving 23488 files into fan-out:
     mv: Runtime: 2.8s
     umount: Runtime: 0.7s
/srv/S6CasA_canonical_results_2014_01_21.tar - 10245 MiB - 74347 files:
     tar: Runtime: 28.8s
     umount: Runtime: 5.7s
Moving 74347 files into fan-out:
     mv: Runtime: 6.2s
     umount: Runtime: 0.8s
hard linking A to B: Runtime: 18.9s
Creating single tarball in-place of fanned-out files (10% of all): Runtime: 376.8s
Remove everything: Runtime: 1118.9s
Untar large tarball again: Runtime: 38.5s

zfs2lc1024a

ZFS with 2 disks per mirror, cache and log device on NVME, 1MB record size

Create 26 fan out dirs: Runtime: 1.3s
/srv/BRP6_canonical_results_2016_01_20.tar - 230 MiB - 48759 files:
     tar: Runtime: 2.7s
     umount: Runtime: 0.8s
Moving 48759 files into fan-out:
     mv: Runtime: 2.1s
     umount: Runtime: 1.0s
/srv/FGRPB1_canonical_results_2016_03_23.tar - 27 MiB - 28195 files:
     tar: Runtime: 1.5s
     umount: Runtime: 0.9s
Moving 28195 files into fan-out:
     mv: Runtime: 2.1s
     umount: Runtime: 0.7s
/srv/O1AS20-100I_canonical_results_2016_03_30.tar - 17882 MiB - 23488 files:
     tar: Runtime: 72.2s
     umount: Runtime: 1.2s
Moving 23488 files into fan-out:
     mv: Runtime: 2.6s
     umount: Runtime: 0.8s
/srv/S6CasA_canonical_results_2014_01_21.tar - 10245 MiB - 74347 files:
     tar: Runtime: 22.8s
     umount: Runtime: 1.2s
Moving 74347 files into fan-out:
     mv: Runtime: 6.0s
     umount: Runtime: 0.7s
hard linking A to B: Runtime: 16.2s
Creating single tarball in-place of fanned-out files (10% of all): Runtime: 151.2s
Remove everything: Runtime: 59.3s
Untar large tarball again: Runtime: 13.3s

zfs2lc1024

ZFS with 2 disks per mirror, cache and log device on NVME, 1MB record size, atime off

Create 26 fan out dirs: Runtime: 1.2s
/srv/BRP6_canonical_results_2016_01_20.tar - 230 MiB - 48759 files:
     tar: Runtime: 2.7s
     umount: Runtime: 0.7s
Moving 48759 files into fan-out:
     mv: Runtime: 2.0s
     umount: Runtime: 1.0s
/srv/FGRPB1_canonical_results_2016_03_23.tar - 27 MiB - 28195 files:
     tar: Runtime: 1.6s
     umount: Runtime: 1.0s
Moving 28195 files into fan-out:
     mv: Runtime: 2.5s
     umount: Runtime: 0.8s
/srv/O1AS20-100I_canonical_results_2016_03_30.tar - 17882 MiB - 23488 files:
     tar: Runtime: 73.1s
     umount: Runtime: 1.2s
Moving 23488 files into fan-out:
     mv: Runtime: 2.4s
     umount: Runtime: 0.6s
/srv/S6CasA_canonical_results_2014_01_21.tar - 10245 MiB - 74347 files:
     tar: Runtime: 22.5s
     umount: Runtime: 1.4s
Moving 74347 files into fan-out:
     mv: Runtime: 5.8s
     umount: Runtime: 0.8s
hard linking A to B: Runtime: 17.5s
Creating single tarball in-place of fanned-out files (10% of all): Runtime: 151.3s
Remove everything: Runtime: 57.5s
Untar large tarball again: Runtime: 14.7s

Templates


Topic attachments
I AttachmentSorted ascending Action Size Date Who Comment
bench.shsh bench.sh manage 2 K 07 Apr 2016 - 06:21 CarstenAulbert simple benchmark script
Topic revision: r3 - 08 Apr 2016, CarstenAulbert
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Foswiki? Send feedback