Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions vortex-duckdb/cpp/table_function.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -208,6 +208,28 @@ duckdb_state register_table_function(DatabaseInstance &db, LogicalType parameter
fn.get_partition_stats = get_partition_stats;
fn.get_multi_file_reader = get_multi_file_reader;

/**
* duckdb's serialization is broken. If you don't set serialize/deserialize
* callbacks, duckdb serializes only the internal state which doesn't work
* for Vortex if you have filters pushed down. Worse, duckdb uses this
* information for CommonSubplanOptimizer which then merges different
* Vortex scans (with different filters pushed down) into one scan in tpcds.
*
* However, this is a regression on q15 and such where we do have
* completely equal scans which can't be merged. This is a reasonable price
* for correctness.
*
* Very unexpectedly verify_serialization doesn't do any verification but
* disables serialization at all.
*/
fn.verify_serialization = false;
fn.serialize = [](auto &, auto, auto &) {
throw NotImplementedException("Can't serialize Vortex state");
};
fn.deserialize = [](auto &, auto &) -> unique_ptr<FunctionData> {
throw NotImplementedException("Can't deserialize Vortex state");
};

try {
auto &system_catalog = Catalog::GetSystemCatalog(db);
auto data = CatalogTransaction::GetSystemTransaction(db);
Expand Down
Loading
Loading