Skip to content

feat: add xor filter - #208

Draft
tisonkun wants to merge 1 commit into
mainfrom
codex/xor-filter
Draft

feat: add xor filter#208
tisonkun wants to merge 1 commit into
mainfrom
codex/xor-filter

Conversation

@tisonkun

Copy link
Copy Markdown
Member

Summary

  • add immutable XorFilter construction and lookup with 8- and 16-bit fingerprints behind the xor feature
  • support both ordinary Rust Hash values and precomputed 64-bit hashes, with duplicate removal and bounded seeded construction retries
  • add family 22 serialization, defensive deserialization, public API documentation, README/changelog entries, and dedicated behavior/serialization tests

Design

The implementation follows the three-partite peeling construction from Graf and Lemire's xor-filter paper directly, without adding a third-party runtime dependency. XorFilterBuilder::build returns Result because construction is deliberately bounded at 100 seed attempts instead of risking an unbounded loop. The default base construction seed is deterministic, and callers can provide another seed when needed.

Serialization follows the 3-long family 22 layout proposed in apache/datasketches-java#740. Tests use images generated by that Java implementation and verify that Rust produces byte-for-byte identical images for the same values and explicit seed. The initial API intentionally exposes only the 8- and 16-bit variants defined by that portable format.

Deserialization validates the family, version, fingerprint width, hash count, signed lengths, indexable capacity, item count, and payload length before constructing a filter.

Testing

@tisonkun
tisonkun marked this pull request as draft August 23, 2026 20:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant