feat: matrix build python-version: ['3.11', '3.12']

[b-long]

This PR is serving two goals:

1. It regression tests bind: make _gopy_clear_go_tls opt-in, off by default #398 , by enabling Python 3.12 builds
2. It identifies and fixes (commit d41db33 ) issue Complex-returning functions crash under CPython 3.12 #400

While we shouldn't combine the two, I'm doing so to be pragmatic and move quickly (for now). Depending on feedback, I may split this PR into 2 separate parts.

Fixes: #400

[satarsa]

Thanks for merging #398! Wiring up the 3.12 matrix is exactly the coverage this needed, I'm happy to have a look. I've reproduced #400 locally end-to-end before writing this, so below is what I've found, a confirmation that your fix is correct, and one structural observation you may want to weigh for later.

As I now understand, #400 is a real bug, fully independent of #398. Your PyGILState_Ensure / Release around PyComplex_FromDoubles is correct and I verified it fixes the crash. The root cause is that gopy releases the GIL (PyEval_SaveThread) around the whole Go wrapper body, and the complex conversion is the one path that allocates a Python object inside that GIL-released window. That's why only complex crashes and, e.g., int or bytes do not.

Environment

Gentoo, glibc, system CPython 3.14.6, go 1.26.4, gcc 16, x86_64. Note 3.12 is only the floor: this class of crash is the glibc + CPython-3.12+ layout, not a specific minor, so 3.14 reproduces #400 identically. Your CI hitting it on 3.12 and me hitting it on 3.14 are the same bug.

1. #400 is real and orthogonal to #398

I built gopy from master-with-#398 (clear off by default) and ran a small probe package, each function in its own process (a fault shows up as a non-zero/signal exit):

function	Go->Py conversion path	result (clear off, no complex fix)
Add(i, j int) int	C side (pybindgen)	OK, 5
GetString() string	C side, C.CString	OK, hello
RealOf(complex128) float64	Go side, PyComplex_AsCComplex (read, no alloc)	OK, 3.0
GetBytes() []byte	C-side handle (go.Slice_byte)	OK
bytes(GetBytes())	Go side, PyBytes_FromStringAndSize	OK, b'\x01\x02\x03\x04'
MayFail(int) (int, error)	value + raise	OK / RuntimeError
Comp64Add(complex64) complex64	Go side, PyComplex_FromDoubles (alloc)	SIGSEGV

So with the TLS clear already off, complex still crashes while everything else is fine; this is not #395 wearing a complex hat, it is a distinct bug. Confirmed crash signature matches your report:

Fatal Python error: Segmentation fault
Current thread (most recent call first):
  File ".../simple.py", line 107 in Comp64Add

2. Root cause: allocation inside gopy's GIL-released window

The generated wrapper releases the GIL for the entire body:

//export probe_Comp64Add
func probe_Comp64Add(i *C.PyObject, j *C.PyObject) *C.PyObject {
	_saved_thread := C.PyEval_SaveThread()        // GIL released, current tstate nulled on this thread
	defer C.PyEval_RestoreThread(_saved_thread)   // re-acquired on return
	return complex64GoToPy(probe.Comp64Add(complex64PyToGo(i), complex64PyToGo(j)))
	//     ^^^^^^^^^^^^^^^^ PyComplex_FromDoubles allocates here, GIL-released -> null tstate
}

On CPython 3.12+, object allocation dereferences the current thread state (allocator / GC bookkeeping), so allocating with the GIL released and tstate == NULL faults. This explains the whole table above:

• int / float / bool / string are marshaled on the C side by pybindgen, where the GIL is still held -> fine.
• []byte comes back as a handle (C side); only an explicit bytes(x) hits the Go-side PyBytes_FromStringAndSize: and that runs as its own exported call with the GIL held, so it's fine too.
• PyComplex_AsCComplex (reading a complex arg) runs in the released window but only reads two doubles, no allocation -> fine.
• PyComplex_FromDoubles (building the complex result) is the one Go-side allocation inside the released window -> crash.

3. As I see your fix is correct and verified

PyGILState_Ensure() re-acquires the GIL and binds a valid tstate for exactly the allocation, Release() drops it again. I rebuilt gopy with your two-function patch and:

Comp64Add(1+2j, 3+4j) -> (4+6j)     # exit 0
RealOf(3+4j)          -> 3.0        # exit 0
Add(2,3)              -> 5          # exit 0

It also matches gopy's own precedent: the Python-callback path in bind/symbols.go already wraps its C-API calls in PyGILState_Ensure / Release for the same reason. So this is consistent with the existing design, not a one-off.

4. Two a little stinky places, they're optional and may be for later

1. The arg-read side is still GIL-released. complex64PyToGo / complex128PyToGo (PyComplex_AsCComplex) touch a PyObject while the GIL is released. It doesn't fault (no allocation), but strictly, touching any PyObject without the GIL is UB. So the complex round-trip as a whole runs Python-C-API-against-released-GIL; the allocation is just the part that visibly crashes.

2. The tuple builder. bind/symbols.go has a Go-side PyTuple_New / gopy_build_* path. I could not get it onto a live wrapper path from ordinary signatures (plain multi-return like (int,int) is rejected: "second result value must be of type error"; (T, error) returns value+raise, no tuple), so I could not make it crash. But if any construct emits that builder inside the released window, it's the same bug.

The minimal, robust generalization would probably be to not PyEval_SaveThread for wrappers whose signature marshals *C.PyObject on the Go side (complex today), or to keep the GIL for the marshalling prologue/epilogue and only release around the pure-Go call. Your targeted PyGILState_Ensure is a perfectly good fix for the crash right now; the above is just the fuller shape of it if you ever want to close the window entirely.

[b-long]

@satarsa Thank you so much for the detailed feedback 🙇

I stacked a PR on top of this one, and I wonder if you'd be generous enough to look at it as well: #401

One thing to note: I tested locally (with the changes from PR 401) and reverting d41db33 on top of changes in 401. I expected that would be enough to resolve the original segfault, since 401 moves the complex arg marshalling ahead of PyEval_SaveThread — but it isn't. TestBindSimple segfaults on the very first Comp64Add call, same fault signature as the original report.

Tracing it: 401 only fixes the argument side (your point 1 above — complex64PyToGo/128PyToGo now run before the GIL is released). The crash you fixed in d41db33 is on the return side. complex64GoToPy/128GoToPy get called from inside return complex64GoToPy(pkg.Comp64Add(...)), and since PyEval_RestoreThread is only deferred, Go evaluates that whole expression — including the allocation — before the deferred restore runs. So your PyGILState_Ensure/Release there is still the only thing protecting it; 401 doesn't touch or replace it.

So it looks like your fix and the two commits in 401 are covering two different, adjacent windows (return-side allocation vs. argument-side read, plus the tuple builder from your point 2) rather than one overlapping one. As far as I can tell, we need all of the above 🙂