Skip to content

THRIFT-4623 Fix python complex nested struct includes - #3753

Open
thomasbruggink wants to merge 1 commit into
apache:masterfrom
thomasbruggink:fix-recursive-complex-python-includes
Open

THRIFT-4623 Fix python complex nested struct includes#3753
thomasbruggink wants to merge 1 commit into
apache:masterfrom
thomasbruggink:fix-recursive-complex-python-includes

Conversation

@thomasbruggink

Copy link
Copy Markdown
Contributor

Fix the bug where the following:

struct C {
    1: i32 value
}

struct A {
    1: C nested = {}
}

struct B {
    1: A itm = {}
}

would generate the following invalid python code:

class C:
    thrift_spec = None

    def __init__(self, value=None):
        self.value = value

class A:
    thrift_spec = None

    def __init__(self, nested=C()):
        if nested is self.thrift_spec[1][4]:
            nested = C()
        self.nested = nested

class B:
    thrift_spec = None

    def __init__(self, itm=A()):
        if itm is self.thrift_spec[1][4]:
            itm = A()
        self.itm = itm

The problem with the above code is that self.thrift_spec was initialized and set after the module was loaded. In python this would work when B is instantiated but since B immediately invokes A() it happens before the thrift_spec is set and this would result in TypeError: 'NoneType' object is not subscriptable. By first initializing a temporary placeholder with a unique address to compare to called _THRIFT_DEFAULT the generated code would look like:

_THRIFT_DEFAULT = object()

class C:
    def __init__(self, value=None):
        self.value = value

class A:
    def __init__(self, nested=_THRIFT_DEFAULT):
        if nested is _THRIFT_DEFAULT:
            nested = C()

class B:
    def __init__(self, itm=_THRIFT_DEFAULT):
        if itm is _THRIFT_DEFAULT:
            itm = A()

Here the instantiation of the default object is defered until after the module has been fully loaded resulting in the correct value being assigned.

  • Did you create an Apache Jira ticket? (Request account here, not required for trivial changes)
  • If a ticket exists: Does your pull request title follow the pattern "THRIFT-NNNN: describe my issue"?
  • Did you squash your changes to a single commit? (not required, but preferred)
  • Did you do your best to avoid breaking changes? If one was needed, did you label the Jira ticket with "Breaking-Change"?
  • If your change does not involve any code, include [skip ci] anywhere in the commit message to free up build resources.

@mergeable mergeable Bot added python compiler build and general CI cmake, automake and build system changes labels Aug 28, 2026
Fix the bug where the following:
```
struct C {
    1: i32 value
}

struct A {
    1: C nested = {}
}

struct B {
    1: A itm = {}
}
```
would generate the following invalid python code:
```
class C:
    thrift_spec = None

    def __init__(self, value=None):
        self.value = value

class A:
    thrift_spec = None

    def __init__(self, nested=C()):
        if nested is self.thrift_spec[1][4]:
            nested = C()
        self.nested = nested

class B:
    thrift_spec = None

    def __init__(self, itm=A()):
        if itm is self.thrift_spec[1][4]:
            itm = A()
        self.itm = itm
```
The problem with the above code is that `self.thrift_spec` was initialized and set after the module was loaded.
In python this would work when B is instantiated but since B immediately
invokes A() it happens before the thrift_spec is set and this would
result in `TypeError: 'NoneType' object is not subscriptable`.
By first initializing a temporary placeholder with a unique address to
compare to called `_THRIFT_DEFAULT` the generated code would look like:
```
_THRIFT_DEFAULT = object()

class C:
    def __init__(self, value=None):
        self.value = value

class A:
    def __init__(self, nested=_THRIFT_DEFAULT):
        if nested is _THRIFT_DEFAULT:
            nested = C()

class B:
    def __init__(self, itm=_THRIFT_DEFAULT):
        if itm is _THRIFT_DEFAULT:
            itm = A()
```
Here the instantiation of the default object is defered until after the
module has been fully loaded resulting in the correct value being
assigned.
@thomasbruggink
thomasbruggink force-pushed the fix-recursive-complex-python-includes branch from 8049ffd to bb4bf37 Compare August 28, 2026 14:17
@Jens-G

Jens-G commented Sep 11, 2026

Copy link
Copy Markdown
Member

Code review

No blocking issues found. Checked for bugs and CLAUDE.md compliance.

A few suggestions, below the bar for an issue but verified:

  • A small formality for when you next push: CONTRIBUTING.md asks for a colon after the ticket id in the commit subject (THRIFT-4623: Fix python complex nested struct includes, the PR title can follow suit) and a Client: py line below it.

thrift/CONTRIBUTING.md

Lines 18 to 27 in bb4bf37

1. All pull requests should contain a single commit per issue, or we will ask you to squash it.
1. The pull request title must begin with the Jira THRIFT ticket identifier if it has an associated ticket, for example:
THRIFT-9999: an example pull request title
1. Commit messages must follow this pattern for code changes (deviations will not be merged):
THRIFT-9999: [summary of fix, one line if possible]
Client: [language(s) affected, comma separated, for example: "cpp,erl,perl"]

  • type now goes through get_true_type(), and the same variable feeds the unchanged immutable branch below it. With -gen py:enum, a field typed as a typedef of an enum, in an exception or a python.immutable struct, now gets the enum coercion (x if hasattr(x, 'value') else Color.__members__.get(x)) instead of a plain assignment, so an int passed there is stored as None. Non-typedef enum fields already behave that way, so this may be acceptable, but it is a behaviour change outside the struct-default fix. A separate variable for the new check would avoid it.

// Initialize fields
t_type* type = get_true_type((*m_iter)->get_type());
if (!type->is_base_type() && !type->is_enum() && (*m_iter)->get_value() != nullptr) {
indent(out) << "if " << maybe_escape_identifier((*m_iter)->get_name()) << " is _THRIFT_DEFAULT:" << '\n';
indent_up();
indent(out) << maybe_escape_identifier((*m_iter)->get_name()) << " = " << render_field_default_value(*m_iter)
<< '\n';
indent_down();
}
if (is_immutable(tstruct)) {
if (gen_enum_ && type->is_enum()) {
indent(out) << "super(" << maybe_escape_identifier(tstruct->get_name()) << ", self).__setattr__('"
<< (*m_iter)->get_name() << "', " << maybe_escape_identifier((*m_iter)->get_name())
<< " if hasattr(" << maybe_escape_identifier((*m_iter)->get_name()) << ", 'value') else "
<< type_name(type) << ".__members__.get(" << maybe_escape_identifier((*m_iter)->get_name()) << "))" << '\n';
} else if (gen_newstyle_ || gen_dynamic_) {

  • With -gen py:type_hints, the new default produces signatures like nested: typing.Optional[C] = _THRIFT_DEFAULT. A type checker rejects that, because object() is neither C nor None. Runtime behaviour is not affected.

result << " = ";
if (tfield->get_value() != nullptr) {
t_type* type = get_true_type(tfield->get_type());
if (!type->is_base_type() && !type->is_enum()) {
result << "_THRIFT_DEFAULT";
} else {
result << render_field_default_value(tfield);
}
} else {

🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

build and general CI cmake, automake and build system changes compiler python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants