Rework execnet onto a trio-native core (sync facade, uv-provisioned workers, execnet.trio)

.claude/settings.json

@@ -0,0 +1,7 @@
+{
+  "permissions": {
+    "allow": [
+      "Bash(uv run pytest:*)"
+    ]
+  }
+}

.gitignore

@@ -20,3 +20,4 @@ include/
 .vagrant.d/
 .config/
 .local/
+.claude/settings.local.json

.pre-commit-config.yaml

@@ -31,5 +31,6 @@ repos:
     -   id: mypy
         additional_dependencies:
           - pytest
+          - trio
+          - hypothesis
           - types-pywin32
-          - types-gevent

CHANGELOG.rst

@@ -2,6 +2,34 @@
 ------------------
 
 * `#380 <https://github.com/pytest-dev/execnet/pull/380>`__: Add support for Python 3.13 and 3.14, and drop EOL 3.8 and 3.9.
+* Trio host-thread Message IO for local ``popen`` + import bootstrap (coordinator and
+  worker). Adds a hard ``trio`` dependency. Disable with ``EXECNET_TRIO_HOST=0``.
+  Other gateway types and greenlet execmodels keep the legacy thread path.
+* Removed ``execnet.script.shell``, an interactive remote prompt that injected its own
+  source into the pre-Trio socket server. The Trio socket server never execs an incoming
+  source line, so the module could no longer work against it.
+* Removed ``execnet.script.quitserver``, which shut a socket server down by sending it
+  ``"raise KeyboardInterrupt"`` to exec. It relied on the same removed handshake.
+* Removed ``execnet.script.loop_socketserver``, a restart loop around a sibling
+  ``socketserver.py`` file. The socket server serves connections in a loop itself
+  (``--once`` opts out), and the sibling-file path never resolved for an installed
+  execnet anyway.
+* Removed ``execnet.script.socketserverservice``, the pywin32 Windows service wrapper.
+  Wrap the ``execnet-socketserver`` console command with a service host such as NSSM
+  instead; the socket gateway example documents how.
+* Moved the socket server to ``execnet._socketserver`` and removed the now empty
+  ``execnet.script`` package. The ``execnet-socketserver`` console command is unchanged
+  and remains the supported way to run it.
+* Removed ``Gateway.reconfigure`` and ``Channel.reconfigure`` along with the
+  ``py2str_as_py3str`` / ``py3str_as_py2str`` arguments of ``gateway_base.loads`` and
+  ``gateway_base.load``. They configured string coercion between Python2 and Python3
+  peers, which execnet can no longer have: ``py2str_as_py3str`` was already unreachable
+  because only a Python2 serializer emits the opcode it gates. The ``RECONFIGURE``
+  message code stays reserved but is no longer sent or handled.
+* The serializer dropped the retired ``PY2STRING`` and ``UNICODE`` opcodes and renamed
+  ``PY3STRING`` to ``STRING``. Values dumped by execnet running on Python2 no longer
+  load. Opcode bytes are unchanged for every type that survives.
+
 
 2.1.2 (2025-11-11)
 ------------------

doc/basics.rst

@@ -43,8 +43,8 @@ Examples for valid gateway specifications
   ``default`` via SSH through Vagrant's ``vagrant ssh`` command. It supports
   the same additional parameters as regular SSH connections.
 
-* ``popen//python=python2.7//nice=20`` specification of
-  a python subprocess using the ``python2.7`` executable which must be
+* ``popen//python=python3.13//nice=20`` specification of
+  a python subprocess using the ``python3.13`` executable which must be
   discoverable through the system ``PATH``; running with the lowest
   CPU priority ("nice" level).  By default current dir will be the
   current dir of the instantiator.
@@ -57,12 +57,10 @@ Examples for valid gateway specifications
   same interpreter as the one it is initiated from and additionally
   remotely sets an environment variable ``NAME`` to ``value``.
 
-* ``popen//execmodel=eventlet`` specifies a subprocess that uses the
-  same interpreter as the one it is initiated from but will run the
-  other side using eventlet for handling IO and dispatching threads.
-
 * ``socket=192.168.1.4:8888`` specifies a Python Socket server
-  process that listens on ``192.168.1.4:8888``
+  process that listens on ``192.168.1.4:8888``.  Such a server can be
+  started with the ``execnet-socketserver`` console command, e.g. run
+  anywhere with ``uvx --from execnet execnet-socketserver :8888``.
 
 .. versionadded:: 1.5
 
@@ -88,10 +86,6 @@ a channel object whose symmetric counterpart channel
 is available to the remotely executing source.
 
 
-.. method:: Gateway.reconfigure([py2str_as_py3str=True, py3str_as_py2str=False])
-
-    Reconfigures the string-coercion behaviour of the gateway
-
 .. _`Channel`:
 .. _`channel-api`:
 
@@ -138,30 +132,29 @@ processes then you often want to call ``group.terminate()``
 yourself and specify a larger or not timeout.
 
 
-threading models: gevent, eventlet, thread, main_thread_only
+threading models: thread, main_thread_only
 ====================================================================
 
 .. versionadded:: 1.2 (status: experimental!)
 
-execnet supports "main_thread_only", "thread", "eventlet" and "gevent"
-as thread models on each of the two sides.  You need to decide which
-model to use before you create any gateways::
+execnet supports "thread" and "main_thread_only" as thread models on
+each of the two sides.  You need to decide which model to use before
+you create any gateways::
 
     # content of threadmodel.py
     import execnet
-    # locally use "eventlet", remotely use "thread" model
-    execnet.set_execmodel("eventlet", "thread")
+    # locally use "thread", remotely use "main_thread_only" model
+    execnet.set_execmodel("thread", "main_thread_only")
     gw = execnet.makegateway()
     print (gw)
     print (gw.remote_status())
     print (gw.remote_exec("channel.send(1)").receive())
 
-You need to have eventlet installed in your environment and then
-you can execute this little test file::
+You can execute this little test file::
 
     $ python threadmodel.py
-    <Gateway id='gw0' receive-live, eventlet model, 0 active channels>
-    <RInfo 'numchannels=0, numexecuting=0, execmodel=thread'>
+    <Gateway id='gw0' receive-live, thread model, 0 active channels>
+    <RInfo 'numchannels=0, numexecuting=0, execmodel=main_thread_only'>
     1
 
 How to execute in the main thread
@@ -219,23 +212,53 @@ configure a tracing mechanism:
 
 .. _`dumps/loads`:
 .. _`dumps/loads API`:
+.. _`serialization`:
 
-Cross-interpreter serialization of Python objects
+Sending objects over a channel
 =======================================================
 
-.. versionadded:: 1.1
+A channel carries only **simple builtin data**: ``None``, ``bool``,
+``int``, ``float``, ``complex``, ``bytes``, ``str`` and arbitrarily nested
+``list`` / ``tuple`` / ``set`` / ``frozenset`` / ``dict`` of those -- plus
+**channel references**, which arrive as channels on the peer.  That is the
+entire contract.
+
+execnet does **not** pickle and does **not** encode rich objects for you:
+arbitrary instances, functions, ``datetime``, dataclasses, pydantic models,
+numpy arrays, enums, etc. have no wire representation.  This is deliberate;
+encoded / rich-object channels are out of scope for execnet.
+
+Sending an unsupported value raises ``DumpError`` (a subclass of
+``DataFormatError``); a corrupt or protocol-mismatched payload on receive
+raises ``LoadError``.  These signal a **caller error to resolve** -- reduce
+the value to simple data before sending -- not a transport failure.  The
+standalone serializer itself is an internal implementation detail
+(``execnet.gateway_base``) and is not part of the public API.
+
+Encode rich objects yourself
+-------------------------------------------------------
+
+Turning a rich object into simple data (and back) is the caller's job.  Use
+an established encoding mechanism rather than expecting the channel to do it:
+
+- **pydantic**: ``model.model_dump(mode="json")`` reduces a model to simple
+  data (``datetime`` -> ISO string, ``UUID`` / ``Enum`` / ``Decimal`` ->
+  primitives); ``Model.model_validate(...)`` rebuilds it on the other side.
+  ``TypeAdapter`` covers non-model types.
+
+  ::
 
-Execnet exposes a function pair which you can safely use to
-store and load values from different Python interpreters
-(e.g. Python2 and Python3, PyPy and Jython). Here is
-a basic example::
+      channel.send(model.model_dump(mode="json"))
+      # peer:
+      model = MyModel.model_validate(channel.receive())
 
-    >>> import execnet
-    >>> dump = execnet.dumps([1,2,3])
-    >>> execnet.loads(dump)
-    [1,2,3]
+- **pytest** does exactly this above execnet: pytest-xdist ships
+  ``TestReport`` objects with the ``pytest_report_to_serializable`` /
+  ``pytest_report_from_serializable`` hooks (rich report <-> simple dict)
+  around ``channel.send`` / ``channel.receive``.
 
-For more examples see :ref:`dumps/loads examples`.
+- **stdlib**: ``dataclasses.asdict(obj)``, ``dt.isoformat()`` /
+  ``datetime.fromisoformat``, or ``json`` with a ``default=`` hook.
 
-.. autofunction:: execnet.dumps(spec)
-.. autofunction:: execnet.loads(spec)
+Channels are the one non-builtin you *can* send: nested channel references
+pass through intact, so callbacks and sub-streams need no encoding.