Skip to content

feat(response): add zstd compression for responses#4215

Open
dave-atx wants to merge 1 commit intominiflux:mainfrom
dave-atx:zstd
Open

feat(response): add zstd compression for responses#4215
dave-atx wants to merge 1 commit intominiflux:mainfrom
dave-atx:zstd

Conversation

@dave-atx
Copy link
Copy Markdown
Contributor

@dave-atx dave-atx commented Apr 5, 2026

Add zstd as the preferred HTTP response compression algorithm using klauspost/compress. zstd is substantially faster than brotli with similar compression ratios.

Since this introduces a new dependency, it also swaps out stdlib compress/gzip and compress/flate for the klauspost drop-in replacements for better performance.

@jvoisin
Copy link
Copy Markdown
Collaborator

jvoisin commented Apr 6, 2026

zstd will eventually land in go's stdlib. I think we can wait, as we already have gzip and flate via the stdlib, and brotli via an external dependency (that I'd love to remove once we have zstd).

I don't think it's worth adding an external dependency for zstd: we're not in a hurry to add zstd support, adding another external library for nice-to-have-but-not-that-significant gains isn't worth it in my opinion, and brotli is currently widely supported while zstd isn't.

@dave-atx
Copy link
Copy Markdown
Contributor Author

dave-atx commented Apr 6, 2026

@jvoisin absolutely fair points.

What do you think about replacing the brotli lib with this zstd lib now? zstd has support in all current major browsers now, so only users on older browsers would get downgraded to gzip in the near term. It's also trivial to add zstd support to the fetcher side once the library is in.

@jvoisin
Copy link
Copy Markdown
Collaborator

jvoisin commented Apr 6, 2026

Sounds like a good idea to me, as the brotli implementation "was translated from the reference implementation with c2go". Good point on the fallback to gzip.

On the other hand, I think it would be preferable to keep klauspost/compress confined to zstd (and not replace the deflate/gzip) implementation, to reduce miniflux' exposure/usage to third-party code. Moreover, compiling it with noasm and nounsafe would help make me sleep soundly at night, as I value very much go memory-safety :)

Oh and also, klauspost/compress looks maintained and used, so at least if it has horrible issues, we won't be alone firefighting.

@fguillot what do you think?

@dave-atx dave-atx force-pushed the zstd branch 4 times, most recently from 3b13ec2 to cb52f62 Compare April 10, 2026 18:20
@dave-atx
Copy link
Copy Markdown
Contributor Author

@jvoisin thank you for the feedback! Incorporated your suggestions.

As it stands, this PR:

  1. Adds zstd support for responses
  2. Adds zstd support for the fetcher
  3. Removes brotli support and the brotli library

Add zstd compression for HTTP responses and zstd decompression for
feed fetching using klauspost/compress. Remove brotli support. Build
with noasm,nounsafe tags to force pure-Go mode for klauspost/compress.
@fguillot
Copy link
Copy Markdown
Member

@fguillot what do you think?

I would prefer to wait for zstd to be included in the Go standard library rather than adding another dependency and increasing complexity. By then, zstd usage will likely be more widespread across HTTP servers.

Also, this pull request removes brotli support from the feed fetcher. Compression is used not only for the Miniflux web UI, but also when fetching and scraping websites. At this point, brotli appears to be more widely used than zstd for content served by HTTP servers and CDNs in the wild.

https://almanac.httparchive.org/en/2025/cdn#compression

compression stats

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

3 participants