Playing with Multithreading support and tuning ZSTD_c_nbWorkers (cf. https://facebook.github.io/zstd/zstd_manual.html#chapter4)
Select how many threads will be spawned to compress in parallel.
When nbWorkers >= 1, triggers asynchronous mode when invoking ZSTD_compressStream*() :
ZSTD_compressStream*() consumes input and flush output if possible, but immediately gives back control to caller,
while compression is performed in parallel, within worker thread(s).
(note : a strong exception to this rule is when first invocation of ZSTD_compressStream2() sets ZSTD_e_end :
in which case, ZSTD_compressStream2() delegates to ZSTD_compress2(), which is always a blocking call).
More workers improve speed, but also increase memory usage.
Default value is `0`, aka "single-threaded mode" : no worker is spawned,
compression is performed inside Caller's thread, and all invocations are blocking */
I've benched and compared compression efficiencies between mod_zstd and zstd.exe cmd line.
Here are the results on a 16 core machine (with a 156MiB XML file)
- zstd -15 -f 156.kml -o 156.zst}
ZSTD_NBTHREADS=0 20.70'
ZSTD_NBTHREADS=1 19.38'
ZSTD_NBTHREADS=2 11.61'
ZSTD_NBTHREADS=16 4.1'
- Mod_zstd with ZSTD_c_compressionLevel=15
ZSTD_c_nbWorkers=0 21.002994'
ZSTD_c_nbWorkers=1 6.658820'
ZSTD_c_nbWorkers=2 6.414540'
ZSTD_c_nbWorkers=16 6.748982
✅ For zstd.exe, time seems coherent and relative ty to the number of thread.
❓ For mod_zstd implementation, it sounds like multithreading parameter give on/off results :
- off with 0
- on with any other values.
Does I missed something?
Playing with Multithreading support and tuning ZSTD_c_nbWorkers (cf. https://facebook.github.io/zstd/zstd_manual.html#chapter4)
I've benched and compared compression efficiencies between
mod_zstdandzstd.execmd line.Here are the results on a 16 core machine (with a 156MiB XML file)
✅ For
zstd.exe, time seems coherent and relative ty to the number of thread.❓ For
mod_zstdimplementation, it sounds like multithreading parameter give on/off results :Does I missed something?