Skip to content

HBASE-30395 Reduce heap buffer copy in decompression - #8665

Open
Huginn-kio wants to merge 1 commit into
apache:masterfrom
Huginn-kio:HBASE-27049
Open

Huginn-kio wants to merge 1 commit into
apache:masterfrom
Huginn-kio:HBASE-27049

Conversation

@Huginn-kio

@Huginn-kio Huginn-kio commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

HBASE-30395

Summary

Avoid an intermediate heap buffer when reading decompressed stream data into an on-heap ByteBuff.

Root cause

The stream fallback in HFileBlockDefaultDecodingContext uses BlockIOUtils.readFullyWithHeapBuffer, which allocates a temporary heap buffer and copies each read into the destination ByteBuff. This adds an avoidable copy for on-heap destinations.

Fix

  • Read directly into the backing array when the destination ByteBuff exposes one.
  • Preserve the existing temporary-buffer fallback for direct or non-array ByteBuff instances.
  • Use the new helper for compressed stream fallback and uncompressed stream reads.
  • Add regression coverage for both the direct-read and fallback paths.

Compatibility

  • Public API impact: none; BlockIOUtils remains internal.
  • Configuration impact: none.
  • Persisted file format impact: none.
  • Wire format impact: none.

Testing

  • mvn -pl hbase-server -am -Dtest=TestBlockIOUtils -DfailIfNoTests=false -DskipITs test
  • mvn -pl hbase-common -DskipTests -Dcheckstyle.includes="**/BlockIOUtils.java,**/HFileBlockDefaultDecodingContext.java" checkstyle:check
  • mvn -pl hbase-server -DskipTests -Dcheckstyle.includes="**/TestBlockIOUtils.java" checkstyle:check

@Huginn-kio Huginn-kio changed the title HBASE-27049 Reduce heap buffer copy in decompression HBASE-30395 Reduce heap buffer copy in decompression Sep 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant