[FLINK-40356][table] Add new MapFromEntries function by VasShabu · Pull Request #28948 · apache/flink

VasShabu · 2026-08-10T15:50:29Z

What is the purpose of the change

The Purpose of this change was to add a new builtin function, MapFromEntries Function to Flink

Brief change log

-Added mapFromEntries functionality
-Wired mapFromEntries function to flink

Verifying this change

Please make sure both new and modified tests in this PR follow the conventions for tests defined in our code quality guide.

(Please pick either of the following options)

This change is already covered by existing tests, such as the ones located in

to run test you need to run this command:
./mvnw -o -pl flink-table/flink-table-planner -Dtest='JsonFunctionsITCase' -Dsurefire.failIfNoSpecifiedTests=false -Dcheckstyle.skip=true -Dspotless.check.skip=true -Drat.skip=true -e -Denforcer.skip=true test

Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): (no)
The public API, i.e., is any changed class annotated with @Public(Evolving): (yes / no)
The serializers: (don't know)
The runtime per-record code paths (performance sensitive): (don't know)
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (no)
The S3 file system connector: (no)

Documentation

Does this pull request introduce a new feature? (yes)
If yes, how is the feature documented? (sql_fuunctions.yml,javadocs,pythonDocs)

Was generative AI tooling used to co-author this PR?

Yes (please specify the tool below)

Generated-by: Claude Opus 4.8

flinkbot · 2026-08-10T15:59:46Z

CI report:

a7cc5f6 Azure: FAILURE
d292ae6 Azure: PENDING

Bot commands

The @flinkbot bot supports the following commands:

@flinkbot run azure re-run the last Azure build

- Pre-size the map since we already know the input - Fill the key and value arrays in one loop instead of using streams

gustavodemorais · 2026-08-17T13:15:12Z

+                                DataTypes.ARRAY(entryType),
+                                DataTypes.ARRAY(nestedEntryType),
+                                DataTypes.ARRAY(entryType))
+                        // duplicate keys: the last value wins


The whole EqualityAndHashcodeProvider/ObjectContainer machinery exists for non-trivial key types, but no case has duplicate STRING, ROW or ARRAY keys - the generated equality/hashcode path is never exercised. Please add one, plus an entry with a NULL value.

gustavodemorais · 2026-08-17T13:26:41Z

Thanks @VasShabu, first pass above

…,null keys

VasShabu · 2026-08-18T12:12:07Z

Thanks @VasShabu, first pass above

Thankyou @gustavodemorais have made the changes as per your request, please let me know if you have any feedback.

Thanks
Vas

snuyanzin · 2026-08-22T19:00:37Z

+
+/** A {@link MapData} backed directly by a key array and a value array. */
+@Internal
+public class MapDataContainer implements MapData {


Why do we need it as a separate file using only in one place?

btw there are already 2 similar inner classes in other classes?
Can we reuse instead?

for instance: extract once and reuse

+1 both MapFromArraysFunction and MapDataForMapFromArrays have the same logic.

MapFromArraysFunction needs a small adjustment. MapUnionFunction could use MapContainer as-is. I think all three could share this one util

I think I should unify this MapDataContainer class for other functions in its own PR simply because one of the guidelines in the PR descriptions is that

Each pull request should address only one issue, not mix up code from multiple issues

So I think it is best to create a new PR to unify the Container classes

Thoughts?
@gustavodemorais @snuyanzin

you can do this in same PR, however different commit

gustavodemorais

Hey @VasShabu, thanks for working on this. Some suggestions

gustavodemorais · 2026-08-24T10:57:17Z

+
+/** A {@link MapData} backed directly by a key array and a value array. */
+@Internal
+public class MapDataContainer implements MapData {


+1 both MapFromArraysFunction and MapDataForMapFromArrays have the same logic.

MapFromArraysFunction needs a small adjustment. MapUnionFunction could use MapContainer as-is. I think all three could share this one util

snuyanzin

since the AI was used, it should be used Generated-by: instead of Co-authored-by: as mentioned in Apache recommendations https://www.apache.org/legal/generative-tooling.html

also the doc about this is in https://github.com/apache/flink/pull/27776/changes#r2979585031

VasShabu · 2026-08-25T09:03:23Z

since the AI was used, it should be used Generated-by: instead of Co-authored-by: as mentioned in Apache recommendations https://www.apache.org/legal/generative-tooling.html

also the doc about this is in https://github.com/apache/flink/pull/27776/changes#r2979585031

I have already set it as Generated-by in the pr description

snuyanzin · 2026-08-25T09:22:32Z

I have already set it as Generated-by in the pr description

look at this commit 9d25cbe
and you will see it is not true

VasShabu · 2026-08-25T09:44:50Z

I have already set it as Generated-by in the pr description

look at this commit 9d25cbe and you will see it is not true

I see will fix that.

…pe access - Reformat the MAP_FROM_ENTRIES docs to the fenced sql / "-- Returns" style across sql_functions.yml, its zh mirror, BaseExpressions javadoc and the pyflink docstring - Reword the non-array argument error to quote 'input' and report the actual type - Read the entry field types via DataType.getFieldDataTypes() instead of the generic getChildren() Generated-By: Claude Opus 4.8 <noreply@anthropic.com>

…Arrays

[FLINK-40356][table] Add new MapFromEntries function

c434e72

snuyanzin reviewed Aug 11, 2026

View reviewed changes

Comment thread ...me/src/main/java/org/apache/flink/table/runtime/functions/scalar/MapFromEntriesFunction.java Outdated

[FLINK-40356][table] Simplify how MapFromEntries builds the map

efb0bda

- Pre-size the map since we already know the input - Fill the key and value arrays in one loop instead of using streams

gustavodemorais reviewed Aug 17, 2026

View reviewed changes

[FLINK-40356][table] refined documentation, added testcases for array…

6dd2822

…,null keys

snuyanzin reviewed Aug 22, 2026

View reviewed changes

gustavodemorais reviewed Aug 24, 2026

View reviewed changes

VasShabu force-pushed the MapFromEntriesImplementation branch from 2425e71 to 9d25cbe Compare August 24, 2026 16:18

snuyanzin requested changes Aug 24, 2026

View reviewed changes

VasShabu force-pushed the MapFromEntriesImplementation branch from 9d25cbe to 0fc0fdd Compare August 25, 2026 10:04

[FLINK-40356][table] Unified mapDataContainer across MapUnion,MapFrom…

a7cc5f6

…Arrays

gustavodemorais reviewed Aug 25, 2026

View reviewed changes

Comment thread ...rg/apache/flink/table/types/inference/strategies/ArrayOfEntriesArgumentTypeStrategyTest.java Outdated

Comment thread ...va/org/apache/flink/table/types/inference/strategies/ArrayOfEntriesArgumentTypeStrategy.java Outdated

Comment thread docs/data/sql_functions.yml Outdated

[FLINK-40356][table] Solved test cases not passing + 'NULL' nit

d292ae6

Uh oh!

Conversation

VasShabu commented Aug 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What is the purpose of the change

Brief change log

Verifying this change

Does this pull request potentially affect one of the following parts:

Documentation

Was generative AI tooling used to co-author this PR?

Uh oh!

flinkbot commented Aug 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

CI report:

Uh oh!

Uh oh!

Uh oh!

Uh oh!

gustavodemorais Aug 17, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

gustavodemorais commented Aug 17, 2026

Uh oh!

VasShabu commented Aug 18, 2026

Uh oh!

snuyanzin Aug 22, 2026

Choose a reason for hiding this comment

Uh oh!

snuyanzin Aug 23, 2026

Choose a reason for hiding this comment

Uh oh!

snuyanzin Aug 23, 2026

Choose a reason for hiding this comment

Uh oh!

gustavodemorais Aug 24, 2026

Choose a reason for hiding this comment

Uh oh!

VasShabu Aug 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

snuyanzin Aug 24, 2026

Choose a reason for hiding this comment

Uh oh!

gustavodemorais left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

gustavodemorais Aug 24, 2026

Choose a reason for hiding this comment

Uh oh!

snuyanzin left a comment

Choose a reason for hiding this comment

Uh oh!

VasShabu commented Aug 25, 2026

Uh oh!

snuyanzin commented Aug 25, 2026

Uh oh!

VasShabu commented Aug 25, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

VasShabu commented Aug 10, 2026 •

edited

Loading

flinkbot commented Aug 10, 2026 •

edited

Loading

VasShabu Aug 24, 2026 •

edited

Loading

gustavodemorais left a comment •

edited

Loading