feat: add columns for anti bias intervention to trials table and qc plot

docs/trials_table_mapping.md

@@ -62,6 +62,8 @@ Columns are grouped by the raw source they map from.
 | Trials column | Mapping |
 | --- | --- |
 | `auto_waterL` / `auto_waterR` | From `is_auto_reward_right`. `1` on the auto-responded side; `0` on the other side, when there was no auto-response (`None`), or when the trial is missing. |
+| `anti_bias_left_water` / `anti_bias_right_water` | Boolean. `True` when the anti-bias algorithm delivered a water intervention to that side — i.e. `trial.metadata.extra.is_bias_water_intervention` is `True` **and** `is_auto_reward_right` points to that side (`False` → left, `True` → right). The anti-bias water uses the same auto-response channel as ordinary autowater, so the `is_bias_water_intervention` flag is what distinguishes it. `False` otherwise. |
+| `anti_bias_lickspout_movement` | Signed horizontal displacement (mm, positive is rightward) the anti-bias algorithm moved the lickspouts on this trial: `trial.lickspout_offset_delta` when `trial.metadata.extra.is_bias_stage_intervention` is `True`, else `0.0`. |
 | `bait_left` / `bait_right` | Boolean. `bait_right` is `True` if `p_reward_right == 1` and `is_auto_reward_right` is `None` or `False`. `bait_left` is `True` if `p_reward_left == 1` and `is_auto_reward_right` is `None` or `True`. |
 | `response_duration` | `response_deadline_duration`. |
 | `reward_consumption_duration` | `Trial -> reward_consumption_duration`. |
@@ -182,6 +184,7 @@ These were mapped during exploration but are no longer in scope:
 | 2026-06-17 | `auto_waterL` / `auto_waterR` now encode no auto-response (`is_auto_reward_right` is `None`) and missing trials as `0` instead of `NULL`. The columns are non-nullable (`int`, default `0`). |
 | 2026-06-20 | Added `reward_size_left` / `reward_size_right` (reward volume in uL) from `task_parameters.reward_size`, and `side_bias` from the per-trial `TrialMetrics` event (`bias` field). |
 | 2026-06-20 | `reward_probabilityL` / `reward_probabilityR` now read the block probability from `trial.metadata.p_reward_left` / `p_reward_right` instead of the top-level per-trial `trial.p_reward_left` / `p_reward_right`. |
+| 2026-07-27 | Added `anti_bias_left_water` / `anti_bias_right_water` (boolean anti-bias water interventions per side) and `anti_bias_lickspout_movement` (mm the anti-bias algorithm shifted the lickspouts) from `TrialOutcome`'s `trial.metadata.extra` (`is_bias_water_intervention` / `is_bias_stage_intervention`), `is_auto_reward_right`, and `lickspout_offset_delta`. These are also overlaid on the QC `side_bias.png` figure. |
 | 2026-07-22 | `lickspout_position_x` / `y1` / `y2` / `z` now derive from the `HarpManipulator` `AccumulatedSteps` stream (microsteps → mm via the `InputSchemas.Rig` manipulator calibration, `full_step_to_mm / microstep_resolution`), sampled per trial via the closest sample in the `[start_time, stop_time)` window and re-referenced to the session-start position (displacement relative to session start, mm), replacing the static `InitialManipulatorPosition` software event. `Motor{i}` maps to `Axis(i + 1)` (X, Y1, Y2, Z). The rig and `AccumulatedSteps` streams are required when there are trials (`build` raises if either is missing). Column descriptions corrected from `um` to `mm`. |
 | 2026-07-24 | `reward_size_left` / `reward_size_right` moved from session-level `task_parameters.reward_size` to per-trial `Trial.reward_size` (fields `.left` / `.right`). The columns are now nullable — `None` when the trial is missing. A missing `TaskLogic` stream no longer raises; session distribution columns are simply null. `min_reward_each_block` moved from `CoupledTrialGenerator` to `CoupledWarmupTrialGenerator`. |
 | 2026-08-06 | **Breaking:** the trial `start_time` / `stop_time` columns are removed and replaced by one start/stop pair per task period: `quiescent_start_time` / `quiescent_stop_time`, `response_start_time` / `response_stop_time`, `reward_consumption_start_time` / `reward_consumption_stop_time`, and `ITI_start_time` / `ITI_stop_time`, read from the `ResponsePeriod` and `RewardConsumptionPeriod` streams in addition to `QuiescentPeriod` and `ItiPeriod`. Each period event marks its period's start, so each stop is the next period's start; `ITI_stop_time` is the next trial's `QuiescentPeriod` timestamp (`NaN` on the last trial). The two new streams are also checked for positional alignment with `TrialOutcome`. NWB's required native `start_time` / `stop_time` are now derived when writing (`quiescent_start_time` → `ITI_stop_time`, falling back to `ITI_start_time`), so the NWB trials table changes in two ways: the old `start_time` / `stop_time` columns are gone, and the native trial extent now ends at the *end* of the ITI rather than at its start. |

src/dynamic_foraging_processing/processing/_trial_table.py

@@ -13,6 +13,9 @@
 from aind_behavior_dynamic_foraging.rig import AindDynamicForagingRig
 from aind_behavior_dynamic_foraging.task_logic import AindDynamicForagingTaskLogic
 from aind_behavior_dynamic_foraging.task_logic.trial_generators import TrialGeneratorSpec
+from aind_behavior_dynamic_foraging.task_logic.trial_generators.block_based_trial_generator import (
+    BlockBasedTrialMetadata,
+)
 from aind_behavior_dynamic_foraging.task_logic.trial_models import (
     Trial,
     TrialMetrics,
@@ -428,6 +431,97 @@ def _auto_water(trial: Trial, *, is_right: bool) -> int:
             return 0
         return int(trial.is_auto_reward_right is is_right)
 
+    @staticmethod
+    def _bias_metadata(trial: Trial) -> BlockBasedTrialMetadata:
+        """Return the block-based extra metadata carrying the anti-bias flags.
+
+        The anti-bias flags (``is_bias_water_intervention``,
+        ``is_bias_stage_intervention``) live on ``trial.metadata.extra``. That
+        field is schema-typed ``Any``, so it deserializes off the stream as a
+        plain ``dict`` rather than a model; a ``BlockBasedTrialMetadata``
+        instance is also accepted. When metadata or extra is missing (e.g. an
+        older session, or a non-block-based generator), the model's all-``False``
+        default is returned so the anti-bias columns are simply inert.
+
+        Parameters
+        ----------
+        trial : Trial
+            The per-trial task-logic model.
+
+        Returns
+        -------
+        BlockBasedTrialMetadata
+            The parsed extra metadata, or an all-``False`` default when absent
+            or unrecognized.
+        """
+        metadata = trial.metadata
+        extra = metadata.extra if metadata is not None else None
+        if isinstance(extra, BlockBasedTrialMetadata):
+            return extra
+        if isinstance(extra, dict):
+            return BlockBasedTrialMetadata.model_validate(extra)
+        return BlockBasedTrialMetadata()
+
+    @staticmethod
+    def _anti_bias_water(
+        trial: Trial, bias_metadata: BlockBasedTrialMetadata, *, is_right: bool
+    ) -> bool:
+        """Return whether the anti-bias algorithm watered the requested side.
+
+        The anti-bias algorithm delivers its water intervention through the same
+        auto-response channel as ordinary autowater (``is_auto_reward_right``:
+        ``True`` right, ``False`` left), so the two are distinguished only by the
+        ``is_bias_water_intervention`` flag. This is ``True`` only when the trial
+        was a bias-water intervention *and* the auto-response was to the
+        requested side.
+
+        Parameters
+        ----------
+        trial : Trial
+            The per-trial task-logic model.
+        bias_metadata : BlockBasedTrialMetadata
+            The trial's extra metadata (see ``_bias_metadata``).
+        is_right : bool
+            ``True`` for the right port, ``False`` for the left port.
+
+        Returns
+        -------
+        bool
+            Whether an anti-bias water intervention targeted the requested side.
+        """
+        if not bias_metadata.is_bias_water_intervention:
+            return False
+        return trial.is_auto_reward_right is is_right
+
+    @staticmethod
+    def _anti_bias_lickspout_movement(
+        trial: Trial, bias_metadata: BlockBasedTrialMetadata
+    ) -> float:
+        """Return the anti-bias lickspout displacement (mm) for this trial.
+
+        The anti-bias algorithm's other intervention shifts the lickspouts
+        horizontally; the per-trial displacement is ``trial.lickspout_offset_delta``
+        (positive is rightward). Reported only when the trial is flagged as a
+        bias-stage intervention, so a stray offset from another source is not
+        attributed to the anti-bias algorithm; ``0.0`` otherwise.
+
+        Parameters
+        ----------
+        trial : Trial
+            The per-trial task-logic model.
+        bias_metadata : BlockBasedTrialMetadata
+            The trial's extra metadata (see ``_bias_metadata``).
+
+        Returns
+        -------
+        float
+            The signed displacement (mm), or ``0.0`` when there was no
+            lickspout intervention.
+        """
+        if not bias_metadata.is_bias_stage_intervention:
+            return 0.0
+        return trial.lickspout_offset_delta
+
     @staticmethod
     def _block_reward_probability(trial: Trial, *, is_right: bool) -> t.Optional[float]:
         """Return the block reward probability for a side from the trial metadata.
@@ -753,6 +847,7 @@ def _build_row(
         trial = outcome.trial
         is_right_choice = outcome.is_right_choice
         is_rewarded = bool(outcome.is_rewarded)
+        bias_metadata = self._bias_metadata(trial)
         start = periods["quiescent_start_time"]
         stop = periods["ITI_start_time"]
 
@@ -778,6 +873,9 @@ def _build_row(
             delay_duration=trial.quiescence_period_duration,
             auto_waterL=self._auto_water(trial, is_right=False),
             auto_waterR=self._auto_water(trial, is_right=True),
+            anti_bias_left_water=self._anti_bias_water(trial, bias_metadata, is_right=False),
+            anti_bias_right_water=self._anti_bias_water(trial, bias_metadata, is_right=True),
+            anti_bias_lickspout_movement=self._anti_bias_lickspout_movement(trial, bias_metadata),
             **session,
             **lickspout,
         )

src/dynamic_foraging_processing/processing/models/trial_config.py

@@ -183,6 +183,26 @@ class TrialConfig(BaseModel):
     auto_waterL: int = Field(default=0, description="Autowater given at Left")
     auto_waterR: int = Field(default=0, description="Autowater given at Right")
 
+    # --- anti_bias (interventions the anti-bias algorithm applies) ---
+    anti_bias_left_water: bool = Field(
+        default=False,
+        description=(
+            "Whether the anti-bias algorithm delivered a water intervention to the left lickport on this trial."
+        ),
+    )
+    anti_bias_right_water: bool = Field(
+        default=False,
+        description=(
+            "Whether the anti-bias algorithm delivered a water intervention to the right lickport on this trial."
+        ),
+    )
+    anti_bias_lickspout_movement: float = Field(
+        default=0.0,
+        description=(
+            "Horizontal distance (mm) the lickspouts were moved by the anti-bias algorithm on this trial (positive is rightward); 0 when no lickspout intervention occurred."
+        ),
+    )
+
     # --- lickspout_position (mapping's `lickspout_positions` -> these four components) ---
     lickspout_position_x: Optional[float] = Field(
         default=None,

src/dynamic_foraging_processing/qc/processed/plots.py

@@ -22,6 +22,11 @@
     lick_latency_by_side,
 )
 
+#: Vertical offset (in side-bias units) of the anti-bias lickspout-move markers
+#: from the zero-bias line: rightward moves sit this far above it, leftward moves
+#: this far below, so the pair reads as arrows straddling y=0.
+_MOVE_MARKER_OFFSET = 0.06
+
 
 def plot_lick_intervals(
     left_lick_times: np.ndarray, right_lick_times: np.ndarray, results_folder: str
@@ -138,8 +143,31 @@ def plot_lick_latency(
     return LICK_LATENCY_PLOT
 
 
-def _add_bias_plot(ax: plt.Axes, side_bias: np.ndarray) -> None:
-    """Draw the per-trial side-bias trace from the trial-table column."""
+def _legend_outside(ax: plt.Axes) -> None:
+    """Place the axes legend just outside the right edge of the panel.
+
+    The trial traces span the full width of these panels, so an in-axes legend
+    sits on top of the data. Anchoring it outside keeps the trace readable; the
+    figure is saved with ``bbox_inches="tight"``, so the legend is not clipped.
+    """
+    ax.legend(loc="upper left", bbox_to_anchor=(1.01, 1.0), borderaxespad=0.0, fontsize="x-small")
+
+
+def _add_bias_plot(
+    ax: plt.Axes,
+    side_bias: np.ndarray,
+    anti_bias_left_water: t.Optional[np.ndarray] = None,
+    anti_bias_right_water: t.Optional[np.ndarray] = None,
+    anti_bias_lickspout_movement: t.Optional[np.ndarray] = None,
+) -> None:
+    """Draw the per-trial side-bias trace with anti-bias interventions overlaid.
+
+    The anti-bias algorithm pushes against a developing side bias, so its two
+    interventions are drawn on top of the bias trace they respond to: water
+    interventions as short ticks at the top (right port) and bottom (left
+    port), and lickspout movements as triangles straddling the zero-bias line,
+    pointing (and coloured) in the direction the spout was moved.
+    """
     ax.set_xlabel("Trial #")
     ax.set_ylabel("Side Bias")
     ax.axhline(+0.7, color="r", linestyle="--")
@@ -153,15 +181,83 @@ def _add_bias_plot(ax: plt.Axes, side_bias: np.ndarray) -> None:
     if len(bias):
         ax.set_xlim([0, len(bias)])
 
+    plotted = False
+    if anti_bias_right_water is not None:
+        right = np.where(np.asarray(anti_bias_right_water, dtype=bool))[0]
+        ax.vlines(right, 0.9, 1.0, color="darkred", linewidth=1, label="Anti-bias water (R)")
+        plotted = True
+    if anti_bias_left_water is not None:
+        left = np.where(np.asarray(anti_bias_left_water, dtype=bool))[0]
+        ax.vlines(left, -1.0, -0.9, color="darkblue", linewidth=1, label="Anti-bias water (L)")
+        plotted = True
+    if anti_bias_lickspout_movement is not None:
+        move = np.asarray(anti_bias_lickspout_movement, dtype=float)
+        # Direction-coded markers straddling the zero-bias line: a rightward move
+        # is a red up-triangle above it, a leftward move a blue down-triangle
+        # below it, matching the red/blue sides used elsewhere in the figure.
+        for indices, offset, marker, color, side in (
+            (np.where(move > 0)[0], _MOVE_MARKER_OFFSET, "^", "red", "R"),
+            (np.where(move < 0)[0], -_MOVE_MARKER_OFFSET, "v", "blue", "L"),
+        ):
+            ax.plot(
+                indices,
+                np.full(len(indices), offset),
+                marker=marker,
+                color=color,
+                linestyle="none",
+                markersize=6,
+                label=f"Anti-bias lickspout move ({side})",
+            )
+        plotted = True
+    if plotted:
+        _legend_outside(ax)
+
+
+def _moved_trials(positions: t.Sequence[t.Optional[np.ndarray]]) -> np.ndarray:
+    """Return the trial indices where any lickspout axis changed position.
+
+    A move is detected as a change from the previous trial on any of the
+    supplied axes, so a trial is flagged once no matter how many axes shifted.
+    NaN samples (missing position readings) are ignored rather than counted as
+    a change.
+
+    Parameters
+    ----------
+    positions : sequence of numpy.ndarray or None
+        Per-trial position arrays (one per axis); ``None`` and empty arrays are
+        skipped.
+
+    Returns
+    -------
+    numpy.ndarray
+        Sorted, unique trial indices at which the lickspouts moved.
+    """
+    moved: t.Set[int] = set()
+    for position in positions:
+        if position is None or len(position) < 2:
+            continue
+        values = np.asarray(position, dtype=float)
+        delta = np.diff(values)
+        changed = np.where(~np.isnan(delta) & (delta != 0))[0] + 1
+        moved.update(int(index) for index in changed)
+    return np.array(sorted(moved), dtype=int)
+
 
 def _add_lickspout_position_plot(
     ax: plt.Axes,
     lickspout_x: t.Optional[np.ndarray],
     lickspout_y1: t.Optional[np.ndarray],
     lickspout_y2: t.Optional[np.ndarray],
     lickspout_z: t.Optional[np.ndarray],
+    anti_bias_lickspout_movement: t.Optional[np.ndarray] = None,
 ) -> None:
-    """Draw lickspout x/y/z positions relative to session start (mm)."""
+    """Draw lickspout x/y/z positions relative to session start (mm).
+
+    Trials where the spouts moved are marked with ticks along the bottom of the
+    panel, split by who moved them: the anti-bias algorithm (same flag as the
+    markers on the side-bias panel) versus the experimenter, which is any other
+    change in position.
+    """
     ax.set_xlabel("Trial #")
     ax.set_ylabel("Lickspout Position \n relative to session start (mm)")
     positions = [
@@ -179,8 +275,39 @@ def _add_lickspout_position_plot(
         # trial-table builder), so plot them directly.
         ax.plot(values, color, label=label)
         plotted = True
+
+    # Ticks sit in axes-fraction coordinates on y so they stay pinned to the
+    # bottom of the panel whatever the position data's range is.
+    transform = ax.get_xaxis_transform()
+    automatic = np.array([], dtype=int)
+    if anti_bias_lickspout_movement is not None:
+        move = np.asarray(anti_bias_lickspout_movement, dtype=float)
+        automatic = np.where(move != 0)[0]
+        ax.vlines(
+            automatic,
+            0.0,
+            0.08,
+            transform=transform,
+            color="g",
+            linewidth=1,
+            label="Automatic move",
+        )
+        plotted = True
+    moved = _moved_trials([lickspout_x, lickspout_y1, lickspout_y2, lickspout_z])
+    manual = np.setdiff1d(moved, automatic)
+    if len(manual):
+        ax.vlines(
+            manual,
+            0.0,
+            0.08,
+            transform=transform,
+            color="k",
+            linewidth=1,
+            label="Manual move",
+        )
+        plotted = True
     if plotted:
-        ax.legend()
+        _legend_outside(ax)
 
 
 def _time_to_trial_index(go_cue_times: np.ndarray, times: np.ndarray) -> t.List[int]:
@@ -305,6 +432,9 @@ def plot_side_bias(
     autowater_right: t.Optional[np.ndarray] = None,
     manual_left_times: t.Optional[np.ndarray] = None,
     manual_right_times: t.Optional[np.ndarray] = None,
+    anti_bias_left_water: t.Optional[np.ndarray] = None,
+    anti_bias_right_water: t.Optional[np.ndarray] = None,
+    anti_bias_lickspout_movement: t.Optional[np.ndarray] = None,
 ) -> str:
     """Save the four-panel side-bias figure.
 
@@ -331,6 +461,14 @@ def plot_side_bias(
         Per-trial autowater indicator arrays.
     manual_left_times, manual_right_times : numpy.ndarray, optional
         Manual-water delivery timestamps (s).
+    anti_bias_left_water, anti_bias_right_water : numpy.ndarray, optional
+        Boolean per-trial arrays flagging anti-bias water interventions on each
+        side; overlaid on the side-bias trace.
+    anti_bias_lickspout_movement : numpy.ndarray, optional
+        Per-trial signed lickspout displacement (mm) applied by the anti-bias
+        algorithm; nonzero trials are marked on the side-bias trace and ticked
+        as automatic moves on the lickspout-position panel, where any other
+        change in position is ticked as a manual move.
 
     Returns
     -------
@@ -343,8 +481,21 @@ def plot_side_bias(
         axis.spines["top"].set_visible(False)
         axis.spines["right"].set_visible(False)
 
-    _add_bias_plot(ax[0], side_bias)
-    _add_lickspout_position_plot(ax[1], lickspout_x, lickspout_y1, lickspout_y2, lickspout_z)
+    _add_bias_plot(
+        ax[0],
+        side_bias,
+        anti_bias_left_water=anti_bias_left_water,
+        anti_bias_right_water=anti_bias_right_water,
+        anti_bias_lickspout_movement=anti_bias_lickspout_movement,
+    )
+    _add_lickspout_position_plot(
+        ax[1],
+        lickspout_x,
+        lickspout_y1,
+        lickspout_y2,
+        lickspout_z,
+        anti_bias_lickspout_movement=anti_bias_lickspout_movement,
+    )
     _add_behavior_plot(
         ax[2],
         animal_response,