concurrency fails w/o warmup

[bjarthur]

trying to recapitulate the multithreading example in the docs, and on ubuntu and macos i have to call systemsleep normally beforehand or submitting it to the pool will hang:

Python 3.14.0 | packaged by conda-forge | (main, Oct 12 2025, 19:48:32) [Clang 20.1.8 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from concurrent.futures import ThreadPoolExecutor, wait
>>> from juliacall import Main as jl
>>> pool = ThreadPoolExecutor(4)
>>> jl.Libc.systemsleep(1)   # w/o this line the next line hangs
0
>>> f = pool.submit(jl.Libc.systemsleep, 5)
>>> f
<Future at 0x106203e00 state=finished returned int>
>>> import juliacall
>>> juliacall.__version__
'0.9.28'
>>> jl.VERSION
Julia: v"1.12.0"

[bjarthur]

here's another similar example. without executing jl.mwejl2(10,10) first, the call to submit hangs. curiously, mwejl() does not need to be warmed up.

Python 3.13.2 | packaged by conda-forge | (main, Feb 17 2025, 14:10:22) [GCC 13.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from concurrent import futures
>>> from juliacall import Main as jl
>>> jl.seval("""
...     function mwejl()
...         [x^2*y for x=1:10, y=1:10]
...     end
...     function mwejl2(sx, sy)
...         [x^2*y for x=1:sx, y=1:sy]
...     end
...     """)
... 
Julia: mwejl2 (generic function with 1 method)
>>> def mwepy(z):
...   print('1')
...   d = jl.mwejl()
...   print('2')
...   d2 = jl.mwejl2(10, 10)
...   print('3')
...   return (d,d2)
... 
>>> exc = futures.ThreadPoolExecutor(max_workers=4)
>>> jl.mwejl2(10,10)       # the next line hangs w/o this line
Julia:
10×10 Matrix{Int64}:
   1    2    3    4    5    6    7    8    9    10
   4    8   12   16   20   24   28   32   36    40
   9   18   27   36   45   54   63   72   81    90
  16   32   48   64   80   96  112  128  144   160
  25   50   75  100  125  150  175  200  225   250
  36   72  108  144  180  216  252  288  324   360
  49   98  147  196  245  294  343  392  441   490
  64  128  192  256  320  384  448  512  576   640
  81  162  243  324  405  486  567  648  729   810
 100  200  300  400  500  600  700  800  900  1000
>>> f = exc.submit(mwepy, 0)
1
2
 3
>>> f.result()
(Julia:
10×10 Matrix{Int64}:
   1    2    3    4    5    6    7    8    9    10
   4    8   12   16   20   24   28   32   36    40
   9   18   27   36   45   54   63   72   81    90
  16   32   48   64   80   96  112  128  144   160
  25   50   75  100  125  150  175  200  225   250
  36   72  108  144  180  216  252  288  324   360
  49   98  147  196  245  294  343  392  441   490
  64  128  192  256  320  384  448  512  576   640
  81  162  243  324  405  486  567  648  729   810
 100  200  300  400  500  600  700  800  900  1000, Julia:
10×10 Matrix{Int64}:
   1    2    3    4    5    6    7    8    9    10
   4    8   12   16   20   24   28   32   36    40
   9   18   27   36   45   54   63   72   81    90
  16   32   48   64   80   96  112  128  144   160
  25   50   75  100  125  150  175  200  225   250
  36   72  108  144  180  216  252  288  324   360
  49   98  147  196  245  294  343  392  441   490
  64  128  192  256  320  384  448  512  576   640
  81  162  243  324  405  486  567  648  729   810
 100  200  300  400  500  600  700  800  900  1000)
>>> jl.VERSION
Julia: v"1.12.0"
>>> import juliacall
>>> juliacall.__version__
'0.9.28'

[bjarthur]

i'm guessing the compilation is allocating, as disabling GC with jl.GC.enable(False) obviates the need for warmup.

[cjdoris]

AFAIU any Julia code that yields will hang when run from a Python thread other than the main one. I think it's because Julia doesn't know about the thread so the task scheduler never returns there.

Presumably this is the same issue - compilation probably yields.

There's a C function to let Julia adopt a foreign thread but I didn't have any luck using it the one time I tried.