#304 made the ProxyCached path request Accept-Encoding: identity so Go stops transparently decompressing responses and the metadata cache stores upstream bytes verbatim. That is correct for the signed-index ecosystems it targeted, but it also drops transfer compression on the proxy-to-upstream hop for every ProxyCached caller.
Once #254 lands this applies to the Homebrew API. formula.jws.json is currently 33,669,153 bytes with Accept-Encoding: identity and 5,107,462 bytes with gzip, so each metadata refresh pulls ~32 MB instead of ~5 MB. Setting verbatim=false for Homebrew would restore the compressed upstream fetch but the proxy would then cache and serve the ~32 MB decoded body to every client, which is worse for a shared cache.
To keep both hops compressed the verbatim path needs to request gzip explicitly rather than identity. Setting Accept-Encoding explicitly already disables transparent decompression, so the compressed bytes and the Content-Encoding header would be cached and replayed exactly as #304 does today. The open question is client behaviour: a cached Content-Encoding: gzip response is served regardless of what the client sent in Accept-Encoding, so apt, apk, dnf and brew all need to handle that header on the affected endpoints. If any of them do not, the fix is per-ecosystem (verbatim becomes an enum: identity, gzip, transparent) rather than a global switch.
Related: proxyMetadataStream (the cache_metadata=false branch) has the same forced identity and would need the same treatment.
#304 made the
ProxyCachedpath requestAccept-Encoding: identityso Go stops transparently decompressing responses and the metadata cache stores upstream bytes verbatim. That is correct for the signed-index ecosystems it targeted, but it also drops transfer compression on the proxy-to-upstream hop for everyProxyCachedcaller.Once #254 lands this applies to the Homebrew API.
formula.jws.jsonis currently 33,669,153 bytes withAccept-Encoding: identityand 5,107,462 bytes withgzip, so each metadata refresh pulls ~32 MB instead of ~5 MB. Settingverbatim=falsefor Homebrew would restore the compressed upstream fetch but the proxy would then cache and serve the ~32 MB decoded body to every client, which is worse for a shared cache.To keep both hops compressed the verbatim path needs to request
gzipexplicitly rather thanidentity. SettingAccept-Encodingexplicitly already disables transparent decompression, so the compressed bytes and theContent-Encodingheader would be cached and replayed exactly as #304 does today. The open question is client behaviour: a cachedContent-Encoding: gzipresponse is served regardless of what the client sent inAccept-Encoding, soapt,apk,dnfandbrewall need to handle that header on the affected endpoints. If any of them do not, the fix is per-ecosystem (verbatimbecomes an enum:identity,gzip,transparent) rather than a global switch.Related:
proxyMetadataStream(thecache_metadata=falsebranch) has the same forcedidentityand would need the same treatment.