Skip to content

Reusing context for compression #7

Description

@nono303

Actually, ZSTD_createCCtx() is called on compress_filter (output filter registered function)
This implies that the zstd context is created and destroyed on each request - which might not be optimized for performance.

https://facebook.github.io/zstd/zstd_manual.html#chapter4

When compressing many times,
  it is recommended to allocate a context just once,
  and re-use it for each successive compression operation.
  This will make workload friendlier for system's memory.
  Note : re-using context is just a speed / resource optimization.
         It doesn't change the compression ratio, which remains identical.
  Note 2 : In multi-threaded environments,
         use one different context per thread for parallel execution.

Also Cf. facebook/zstd#2107

So, my question is, could the zstd context can be instantiated one time by "worker ❓" and reused for all compression process.
⚠ Must be thread safe in case of concurrent compression in a given worker

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requesthelp wantedExtra attention is needed

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions