Skip to content

ProxyCached loses transfer compression on both hops after #304 #305

Description

@andrew

#304 made the ProxyCached path request Accept-Encoding: identity so Go stops transparently decompressing responses and the metadata cache stores upstream bytes verbatim. That is correct for the signed-index ecosystems it targeted, but it also drops transfer compression on the proxy-to-upstream hop for every ProxyCached caller.

Once #254 lands this applies to the Homebrew API. formula.jws.json is currently 33,669,153 bytes with Accept-Encoding: identity and 5,107,462 bytes with gzip, so each metadata refresh pulls ~32 MB instead of ~5 MB. Setting verbatim=false for Homebrew would restore the compressed upstream fetch but the proxy would then cache and serve the ~32 MB decoded body to every client, which is worse for a shared cache.

To keep both hops compressed the verbatim path needs to request gzip explicitly rather than identity. Setting Accept-Encoding explicitly already disables transparent decompression, so the compressed bytes and the Content-Encoding header would be cached and replayed exactly as #304 does today. The open question is client behaviour: a cached Content-Encoding: gzip response is served regardless of what the client sent in Accept-Encoding, so apt, apk, dnf and brew all need to handle that header on the affected endpoints. If any of them do not, the fix is per-ecosystem (verbatim becomes an enum: identity, gzip, transparent) rather than a global switch.

Related: proxyMetadataStream (the cache_metadata=false branch) has the same forced identity and would need the same treatment.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions