PyDABs: support interface{} fields (cluster_policies, dashboards, genie_spaces) - #6585
Open
Sankalp-Mittal wants to merge 15 commits into
Open
PyDABs: support interface{} fields (cluster_policies, dashboards, genie_spaces)#6585Sankalp-Mittal wants to merge 15 commits into
Sankalp-Mittal wants to merge 15 commits into
Conversation
interface{} fields will be generated as `Any`. There is no schema to
normalize against, so _transform must return the value unchanged;
otherwise it falls through to the "Unexpected type" error. Validation of
the value (string or map) stays on the Go side in the config mutators.
Co-authored-by: Isaac <no-reply@databricks.com>
Model interface{} fields (schema ref #/$defs/interface) as typing.Any,
following the same PRIMITIVES + RENAMES pattern used for time.Time:
- PRIMITIVES treats "interface" as a terminal type, so get_package returns
None (no package split crash, no bogus AnyParam) and the reachability
walk stops there.
- RENAMES maps it to "Any"; the generated-module import header gains Any
(lint trims it where unused).
- _synth_scalar emits a fixed inline dict for interface fields, so a future
required interface field has a value instead of crashing the generator.
No generated output changes yet; the resources with these fields are still
denylisted.
Co-authored-by: Isaac <no-reply@databricks.com>
These three resources were denylisted only because they carry interface{}
fields (definition, policy_family_definition_overrides,
serialized_dashboard, serialized_space). Now that interface{} generates as
Any, remove them from the denylist and regenerate.
The denylist itself is kept (empty) as the escape hatch for future
resources that hit an unmodellable type.
Co-authored-by: Isaac <no-reply@databricks.com>
…paces
Each fixture covers both authoring forms of the interface{} field: a YAML
instance sets it as an inline map, a Python instance sets it as a
serialized JSON string. The golden shows both converging to a JSON string
(the Go config mutator marshals the map; the string passes through).
Satisfies test_python_support_coverage, which requires one acceptance
fixture per PyDABs resource.
Co-authored-by: Isaac <no-reply@databricks.com>
Document that a fixture for a resource with an interface{}/Any field must
exercise both authoring forms (inline map in YAML, serialized JSON string
in Python), so every future interface type is covered the same way.
Co-authored-by: Isaac <no-reply@databricks.com>
PR link to be appended when the PR is opened. Co-authored-by: Isaac <no-reply@databricks.com>
Regenerated public_api.txt to include add_cluster_policy / add_dashboard / add_genie_space and the cluster_policy_mutator / dashboard_mutator / genie_space_mutator decorators. Co-authored-by: Isaac <no-reply@databricks.com>
Passing typing.Any as the `cls` argument (and as Variable.type) is intentional in these tests but not expressible in the type signature. Co-authored-by: Isaac <no-reply@databricks.com>
Main renamed the generated TestCase to ResourceTestCase; regenerate the three new resources' test-case files so they match. Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
A dict and a serialized JSON string take different _transform paths, so an acceptance fixture must exercise both through add_<singular> (two Python instances). Covering only one leaves the other path silently brittle. The YAML instance exercises the loader, not the Python Any path, so it does not substitute for either. Co-authored-by: Isaac <no-reply@databricks.com>
Each fixture now adds a third instance (my_<name>_3) that passes the interface field as an inline dict through add_<singular>, alongside the existing serialized-string instance. The golden shows the dict marshalled to the same compact JSON string, exercising the Any passthrough end to end. Co-authored-by: Isaac <no-reply@databricks.com>
Spread the inline-dict instances across lines (trailing commas keep them exploded through ruff) so the structured-dict case reads as visibly distinct from the flat serialized-string case. No behavior change; goldens are unchanged. Co-authored-by: Isaac <no-reply@databricks.com>
Sankalp-Mittal
marked this pull request as ready for review
September 9, 2026 13:19
Contributor
Approval status: pending
|
Collaborator
Integration test reportCommit: 7803a1a
Top 6 slowest tests (at least 2 minutes):
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds PyDABs (Python) support for Go
interface{}fields and onboards the three resources that were denylisted solely because of them: cluster policies, dashboards, and Genie spaces (definition/policy_family_definition_overrides,serialized_dashboard,serialized_space).How
interface{}(schema ref#/$defs/interface) is modelled astyping.Any, following the samePRIMITIVES+RENAMESpattern already used fortime.Time:"interface"is a terminal primitive renamed toAny, soget_packagereturnsNone(no crash, no bogusAnyParam) and the import header gainsAny._transformpasses anAny-typed value through unchanged; there is no schema to normalize against. Validation (string or map) stays on the Go side in the existingconfigure_*config mutators._synth_scalaremits a fixed value for interface fields.The
RESOURCE_DENYLISTis kept (now empty) as the escape hatch for future resources that hit an unmodellable type.Acceptance coverage
Each new resource has a
<plural>-support/fixture that exercises both authoring forms of the interface field: an inline map (YAML instance) and a serialized JSON string (Python instance). The golden shows them converging — the map is marshalled to a JSON string, the string passes through. This convention is documented inacceptance/bundle/python/README.mdand thepydabs-acceptance-testsrule so future interface types are covered the same way.Testing
databricks_testsunit suite (incl._transformAny passthrough and the coverage guard), pyright, and the three acceptance tests all pass.pydabs-codegenis idempotent on top ofmain.This pull request and its description were written by Isaac.