opcua: rescan while disconnected, heal PLC_COMMS_LOST on connect, and name a plugin freeze-frame's source

docs/api/rest.rst

@@ -1517,13 +1517,19 @@ Query and manage faults.
    - ``freeze_frame``: Data captured at fault confirmation. Entity frames for
      faults that were already confirmed when the gateway started are captured
      at gateway start instead and carry ``"capture_origin": "startup"`` in
-     their ``x-medkit`` block. For a plugin-backed entity that reports its
-     link down, the values are the plugin's last known ones and may predate
-     the confirmation by the length of the outage; such entries carry
-     ``connected`` (the payload's link flag, ``false`` for the loss-of-comms
-     case) and ``source_timestamp`` (the payload's own timestamp, verbatim)
-     in ``x-medkit``, both only when the plugin's payload reports them.
-     ``captured_at`` always dates the capture, not the values.
+     their ``x-medkit`` block. An entity frame also carries ``source`` in
+     ``x-medkit``, naming the path that read the values
+     (``plugin_data_provider`` for the owning plugin's DataProvider,
+     ``plugin_x_plc_data_route`` for its ``x-plc-data`` route). These values
+     are not a ROS message, so ``topic`` and ``message_type`` are empty and
+     ``source`` is the only field saying where the numbers came from. For a
+     plugin-backed entity that reports its link down, the values are the
+     plugin's last known ones and may predate the confirmation by the length of
+     the outage. Such entries carry ``connected`` (the payload's link flag,
+     ``false`` for the loss-of-comms case) and ``source_timestamp`` (the
+     payload's own timestamp, verbatim) in ``x-medkit``, both only when the
+     plugin's payload reports them. ``captured_at`` always dates the capture,
+     not the values.
    - ``rosbag``: Recording file available via bulk-data endpoint
 
    **Response codes:**

docs/config/discovery-options.rst

@@ -85,8 +85,76 @@ filters out ROS 2 internal infrastructure nodes such as ``_ros2cli_*``,
 SOVD entities. The filter applies to both locally discovered Apps and
 peer-discovered Apps (after stripping the peer prefix).
 
+The same switch also excludes the helper nodes the gateway runs inside its own
+process. Three of them exist in every deployment, named after the gateway node:
+
+.. list-table::
+   :header-rows: 1
+   :widths: 40 60
+
+   * - Node
+     - What it is
+   * - ``<gateway>_sub``
+     - The subscription executor that serves ``/data`` reads and cyclic
+       subscriptions.
+   * - ``<gateway>_fault_clients``
+     - The service clients that talk to the fault manager.
+   * - ``<gateway>_lifecycle_state_reader``
+     - The client that reads managed nodes' lifecycle state.
+
+None of these names begins with an underscore, so the convention above does not
+cover them, and without this rule the gateway would list its own plumbing as
+diagnosable Apps. They carry no parameters, services or actions of their own, so
+there is nothing on them to diagnose.
+
+While this setting is ``true`` they are also left out of the linking report on
+``GET /health`` - ``discovery.linking.orphan_count``, and the
+``ros_node_fqns`` of the ``unmanifested_nodes`` warning raised under
+``unmanifested_nodes: error``: a node that can never be an App is not a node to
+declare in a manifest. That report exists only in ``hybrid`` mode, where
+manifest apps are linked to runtime nodes; ``runtime_only`` has no linking block
+at all.
+
+**The gateway's own node stays an App.** Its ROS parameters are what the
+gateway serves as that App's configurations, so
+``/apps/<gateway>/configurations`` is where a client reads and writes them -
+there is no other entity carrying, for instance,
+``aggregation.peer_auth_header``. Two gateways watching one graph also have to
+agree about what is on it, which they cannot do if each hides a different node.
+
+Namespaces: the subscription node is created with the gateway's own namespace,
+while the other two are created from the gateway's node name alone and take the
+process default namespace. A process-wide remap (``-r __ns:=/line_a``) moves all
+four together; a remap naming the gateway alone
+(``-r ros2_medkit_gateway:__ns:=/line_a``) moves the gateway and the
+subscription node and leaves the other two in the default namespace. Both
+spellings are recognised, so the rule holds either way.
+
+.. note::
+
+   A node named ``<gateway>_fault_clients`` or
+   ``<gateway>_lifecycle_state_reader`` in the **root** namespace is treated as
+   plumbing whichever gateway created it. Two gateways that keep the default
+   node name and differ only in namespace produce the same fully qualified name
+   for those two nodes, so the name cannot say whose they are, and each gateway
+   will filter the other's. They carry nothing to diagnose in either process,
+   and the alternative is that a namespaced gateway lists and counts its own
+   plumbing. Give each gateway its own node name (``-r __node:=gateway_a``, as
+   :doc:`../tutorials/multi-instance` does) when several run on one ROS graph
+   and you want each one's nodes distinguishable. The subscription node is not
+   affected: it always follows its gateway's namespace.
+
+A manifest- or plugin-declared App bound to one of these nodes is not served,
+and the gateway logs a warning naming it: bind the App to the node you meant,
+or drop the declaration.
+
 Set to ``false`` if you need to expose all ROS 2 nodes regardless of naming
-convention.
+convention. That re-exposes the three helper nodes as well as the underscore
+ones, and in ``hybrid`` mode the linking report follows: a helper node the
+gateway serves as an App is also an undeclared one, so it is counted in
+``discovery.linking.orphan_count`` and, under ``unmanifested_nodes: error``,
+named in the ``ros_node_fqns`` of the ``unmanifested_nodes`` warning. With
+``true`` it appears in neither.
 
 Function Entities from Namespaces
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -446,6 +514,17 @@ Configuration
        # Default: 100
        plugins.topic_beacon.max_messages_per_second: 100
 
+``beacon_ttl_sec`` takes 0.1 to 2147483647 s, ``beacon_expiry_sec`` 1.0 to 2147483647 s, and
+``max_messages_per_second`` 1 to 10000. A value above its maximum, including ``.inf`` in a
+parameter file, becomes the maximum. NaN, ``-.inf`` and a value below the minimum become the
+minimum. The gateway logs a warning naming the key, the value and what replaced it. Messages
+over the rate limit are dropped without a log.
+
+``max_hints`` takes an integer from 1 to 2147483647. A 64-bit integer outside that range
+becomes the nearer bound, with the same warning. The parameter parser reads an integer that
+does not fit in 64 bits as a double. A double, such as ``1.0e12`` or ``.nan``, is refused with
+a warning and the default 10000 is used.
+
 Beacon Lifecycle
 ^^^^^^^^^^^^^^^^
 
@@ -601,6 +680,36 @@ Configuration
        # Default: 10000
        plugins.parameter_beacon.max_hints: 10000
 
+Every duration takes its minimum - 0.1 s, and 1.0 s for ``beacon_expiry_sec`` - up to
+2147483647 s. Fast DDS keeps the seconds of a wait in a 32-bit signed integer, and a longer
+``param_timeout_sec`` would make the poll thread spin. A value above the maximum, including
+``.inf`` in a parameter file, becomes the maximum. NaN, ``-.inf`` and a value below the
+minimum become the minimum. The gateway logs a warning naming the key, the value and what
+replaced it.
+
+``max_hints`` takes an integer from 1 to 2147483647. A 64-bit integer outside that range
+becomes the nearer bound, with the same warning. The parameter parser reads an integer that
+does not fit in 64 bits as a double. A double, such as ``1.0e12`` or ``.nan``, is refused with
+a warning and the default 10000 is used.
+
+In ``runtime_only`` and ``manifest_only`` mode the plugin reads its poll targets from the
+ROS graph. In ``hybrid`` mode the merge pipeline passes it the discovered Apps, but the
+gateway's refresh then calls it again with no Apps, which clears them. So a poll cycle mostly
+reads the graph there too; only a cycle that starts between the two calls polls the nodes of
+the online Apps that discovery bound to a node. A graph read skips hidden nodes (a name
+starting with ``_``), the gateway's own node and its helper nodes (``<gateway>_sub``,
+``<gateway>_fault_clients`` and ``<gateway>_lifecycle_state_reader``): they carry no beacon.
+
+A parameter request that gets no answer within ``param_timeout_sec`` - waiting for the
+service, listing parameters or getting their values - is given up and removed from its
+client, and the node is skipped for the next 1, 2, 4 and then 8 poll cycles while it keeps
+timing out. A node that answers is polled on every cycle, also when its answer carries no
+values: rclpy answers so when one of the parameters asked for is declared with a type and no
+value, and the plugin then stores no hint for the node. None of this is logged.
+
+Each node keeps one parameter client across poll cycles. A node that is no longer a poll
+target, also when no node is, loses its client and its skip count.
+
 Parameter Naming
 ^^^^^^^^^^^^^^^^
 

docs/config/server.rst

@@ -480,10 +480,12 @@ How long a departed node keeps being listed
 
 Once a node has actually left the ROS graph, ``GET /apps`` stops listing it within
 roughly one refresh: at most ``discovery.refresh_debounce_ms`` plus the 100 ms graph
-poll, or ``refresh_interval_ms``, whichever comes first. Nothing is retained behind
-that - every refresh rebuilds the entity set from a live read of the graph - so this
-is the whole of the gateway's share, and it is the only part of a departure the
-gateway can be held to.
+poll, or ``refresh_interval_ms``, whichever comes first. Every refresh rebuilds the
+entity set from a live read of the graph, so this is the whole of the gateway's share,
+and it is the only part of a departure the gateway can be held to. What the gateway
+remembers between reads - the names it saw running, described below - decides only
+whether an entry the graph still lists after its participant left is exposed; it never
+keeps listing a node the graph no longer lists.
 
 Before that point nothing here is promised, and the difference is not small:
 
@@ -504,6 +506,83 @@ rather than assume a deadline. Anything that measures a departure should establi
 that the process has actually exited first, and only then hold the gateway to the
 figure above.
 
+The graph itself can also keep a node after its process has exited, for as long as the
+gateway runs. The DDS RMWs take a participant's node list from discovery messages on an
+internal topic, read on a listener thread of their own. When that thread takes a message
+only after the participant was removed, the message puts the node back, with an empty
+enclave and none of its endpoints, and nothing removes it again. This page calls such an
+entry a leftover.
+
+An empty enclave on its own does not make a leftover. A node on the far side of a DDS
+router, or a node whose participant was created outside rcl (a micro-ROS agent's, for
+example), also reads an empty enclave, and the router's far side resolves no endpoints
+either. So the gateway leaves an entry out only for a node it has itself seen running.
+Per node name, on every read of the graph:
+
+* an entry with an enclave is listed, and the name counts as seen running;
+* an entry without an enclave is dropped when another entry of the same name has one, so a
+  restarted node is listed once;
+* an entry without an enclave for a name the gateway has seen running is left out when the
+  graph resolves no publisher or subscriber for the node (a service or client is a request
+  and a reply endpoint, so it counts too);
+* an entry without an enclave for a name the gateway has never seen running is listed.
+
+A leftover is left out of ``GET /apps`` and ``GET /apps/{id}``, out of the Functions
+derived from namespaces, and out of the bare-name collision check that would otherwise
+rename a live node sharing its name. The read that starts leaving a node out logs:
+
+.. code-block:: text
+
+   [WARN] [ros2_medkit_gateway]: Node '/ns/name' is not exposed: this gateway saw it running, and the ROS graph still lists it after its participant left, with no endpoints
+
+Later reads that keep leaving it out stay quiet. The warning comes back only after a read
+that did not leave the node out.
+
+Which names the gateway saw running comes from the reads themselves. The names running on
+the previous read are that read's own list, so they cost nothing however many nodes run. A
+name that ran on the previous read and does not run on this one - the graph lists no entry
+of it, or only entries without an enclave - has departed, and the gateway remembers it until
+it runs again or until it is forgotten on a read: the first read that finds no entry of it
+more than 10 seconds after the first read that found none. While an entry of it without an
+enclave is listed, those 10 seconds do not start. They cover the gap between the
+participant's removal and the late message: the time the listener thread takes to process a
+message it already holds, measured under a millisecond on an idle host and up to 0.64 s with
+its process and eight busy threads sharing two cores.
+
+The name is forgotten only on a read. Reads come from graph changes, the refresh backstop
+(``refresh_interval_ms``), the gateway's start and, in ``runtime_only`` mode, a request for an
+App or a Function that a refresh removed from the entity cache while the request ran. A late
+message that arrives before the read that forgets the name stays hidden, however long after
+the 10 seconds it comes: the late message itself changes the graph, and the read that follows
+finds its entry. Only a late message that arrives after that read is listed, like a node the
+gateway never saw running.
+
+The gateway remembers at most 1024 departed names. Past that it forgets first the names the
+graph has listed no entry of for longest, and then the names that departed longest ago; a
+forgotten name whose only entries are leftovers is listed again, without a warning. So once
+1024 leftovers are hidden, a node that departs with no entry of it left is the first name
+forgotten, and a late message for it is listed. Every remembered leftover costs two endpoint
+queries on each read, so the cap also bounds that work.
+
+A node that ran on this host and then appears only behind a DDS router, as an entry without
+an enclave, is hidden like a leftover if it appears before the gateway forgets the name or
+while a leftover of it is still listed.
+
+The startup discovery summary counts peer nodes through the same memory, so in
+``runtime_only`` mode, and in ``hybrid`` mode with the runtime layer enabled, it counts what
+discovery would list at that moment. When discovery does not read the graph itself - in
+``manifest_only`` mode, or in ``hybrid`` mode with ``discovery.runtime.enabled: false`` - the
+summary's read is the first, so no node has been seen running: it counts every entry the
+graph lists, except an entry without an enclave next to an entry of the same name with one.
+
+The ``param_beacon`` plugin reads the graph itself only when discovery gives it no poll
+targets, and then through a memory of its own: a node its reads saw running is not polled
+once only a leftover of it is listed. It logs nothing about it.
+
+A node restarted while a leftover of its previous instance is still listed can show no
+services, and so no operations, because the graph answers per-node queries from whichever
+of the two participants sorts first.
+
 Thread Pools
 ------------
 

docs/tutorials/snapshots.rst

@@ -239,6 +239,26 @@ with:
    ros2 run ros2_medkit_gateway gateway_node --ros-args \
      -p entity_freeze_frame.enabled:=false
 
+A plugin entity's values are not a ROS message, so ``topic`` and
+``message_type`` are empty on these frames. ``x-medkit.source`` names the
+capture path instead, so a consumer can still tell where the values came
+from:
+
+.. list-table::
+   :header-rows: 1
+   :widths: 30 70
+
+   * - ``x-medkit.source``
+     - Meaning
+   * - ``plugin_data_provider``
+     - Read through the owning plugin's ``DataProvider::list_data``.
+   * - ``plugin_x_plc_data_route``
+     - Read by dispatching the owning plugin's own ``x-plc-data`` route
+       in-process (plugins that export no ``DataProvider``).
+
+The field is absent on freeze-frames captured by the fault manager from a ROS
+topic. Those carry a real ``topic`` and ``message_type`` instead.
+
 Example plugin-entity freeze-frame in the fault response:
 
 .. code-block:: json
@@ -250,6 +270,7 @@ Example plugin-entity freeze-frame in the fault response:
      "x-medkit": {
        "topic": "",
        "message_type": "",
+       "source": "plugin_x_plc_data_route",
        "full_data": {"tank_level": 87.5, "pump_running": true},
        "captured_at": "2026-07-14T12:00:00.000Z"
      }

src/ros2_medkit_discovery_plugins/ros2_medkit_param_beacon/CMakeLists.txt

@@ -85,11 +85,7 @@ if(BUILD_TESTING)
   include(ROS2MedkitTestDomain)
 
   # Include plugin source in test (plugin is MODULE/dlopen, test instantiates directly)
-  # TIMEOUT 180: the 12 test cases each spin a node for ~200-400 ms. Under
-  # plain gcc that's ~4 s total, well under the 30 s ctest default, but TSan
-  # instrumentation inflates that 5-10x and occasionally trips the 30 s cap
-  # (CI saw timeouts at test 4/12 around 27 s). 180 s covers the worst-case
-  # sanitizer slowdown without masking real hangs.
+  # TIMEOUT 180: the cases spin real nodes, and TSan slows them 5-10x past the 30 s default.
   medkit_add_gmock(test_param_beacon_plugin
     test/test_param_beacon_plugin.cpp
     src/param_beacon_plugin.cpp

src/ros2_medkit_discovery_plugins/ros2_medkit_param_beacon/README.md

@@ -11,8 +11,35 @@ through standard ROS 2 parameters.
 3. Entity metadata is mapped into the SOVD hierarchy via `BeaconEntityMapper`
 4. Results are exposed at the `x-medkit-param-beacon` vendor extension endpoint
 
-In non-hybrid discovery mode, the plugin discovers poll targets automatically from the
-ROS 2 graph. In hybrid mode, targets come from the manifest.
+In `runtime_only` and `manifest_only` discovery the plugin reads its poll targets from the
+ROS 2 graph. In `hybrid` mode the merge pipeline passes it the discovered Apps, but the
+gateway's refresh then calls it again with no Apps, which clears them, so a poll cycle mostly
+reads the graph there too. A graph read skips hidden nodes (a name starting with `_`), the
+gateway's own node and its helper nodes (`<gateway>_sub`, `<gateway>_fault_clients`,
+`<gateway>_lifecycle_state_reader`): they carry no beacon. The plugin remembers what its own
+reads saw: a node that ran on one of its reads is not polled once the graph lists only a
+leftover of it, an entry with an empty enclave and no endpoints, which a node whose
+participant has left can leave behind (see "How long a departed node keeps being listed" in
+the gateway's `docs/config/server.rst`).
+
+Each node keeps one parameter client across poll cycles. A node that is no longer a target,
+also when no node is, loses its client. A parameter request that gets no answer within
+`param_timeout_sec` - waiting for the service, listing parameters or getting their values -
+is given up and removed from its client, and the node is skipped for the next 1, 2, 4 and
+then 8 poll cycles while it keeps timing out, so a node whose parameter services never answer
+does not stall polling. A node that answers is polled on every cycle, also when its answer
+carries no values: rclpy answers so when one of the parameters asked for is declared with a
+type and no value, and the plugin then stores no hint. None of this is logged.
+
+Every duration below takes its minimum (0.1 s, `beacon_expiry_sec` 1.0 s) up to 2147483647 s,
+the longest wait Fast DDS keeps without the poll thread spinning. A value above the maximum,
+including `.inf` in a parameter file, becomes the maximum; NaN, `-.inf` and a value below the
+minimum become the minimum. Each replacement is logged as a warning.
+
+`max_hints` takes an integer from 1 to 2147483647. A 64-bit integer outside that range becomes
+the nearer bound. The parameter parser reads an integer that does not fit in 64 bits as a
+double. A double, such as `1.0e12` or `.nan`, is refused and the default 10000 is used. Both
+are logged as warnings.
 
 ## Configuration