fix: incorrect annotation of rewards

docs/trials_table_mapping.md

@@ -61,8 +61,9 @@ Columns are grouped by the raw source they map from.
 
 | Trials column | Mapping |
 | --- | --- |
-| `auto_waterL` / `auto_waterR` | From `is_auto_reward_right`. `1` on the auto-responded side; `0` on the other side, when there was no auto-response (`None`), or when the trial is missing. |
-| `anti_bias_left_water` / `anti_bias_right_water` | Boolean. `True` when the anti-bias algorithm delivered a water intervention to that side — i.e. `trial.metadata.extra.is_bias_water_intervention` is `True` **and** `is_auto_reward_right` points to that side (`False` → left, `True` → right). The anti-bias water uses the same auto-response channel as ordinary autowater, so the `is_bias_water_intervention` flag is what distinguishes it. `False` otherwise. |
+| `auto_waterL` / `auto_waterR` | **Rewarded** autowater only: `1` when `is_auto_reward_right` points to that side **and** `is_rewarded` is `True`. `0` on the other side, when there was no auto-response (`None`), when the trial delivered autowater but did not pay out, or when the trial is missing. `is_auto_reward_right` is the delivery channel — it says free water was triggered and to which side, not what kind — and the anti-bias water intervention is autowater delivered through it, so anti-bias water counts here too. Note this is narrower than the legacy `dynamic-foraging-task` column of the same name, which was ungated ("Autowater given at Left", straight from `B_AutoWaterTrial`). |
+| `auto_water_offeredL` / `auto_water_offeredR` | Ungated autowater: `1` whenever `is_auto_reward_right` points to that side, whether or not the trial paid out. This is the legacy `dynamic-foraging-task` meaning of `auto_waterL` / `auto_waterR` (`B_AutoWaterTrial`, which that GUI also passes to foraging efficiency as `autowater_offered`), kept so the trial table records every autowater delivery. `auto_waterL` / `auto_waterR` are the reward-keyed subset. |
+| `anti_bias_left_water` / `anti_bias_right_water` | Boolean. `True` when the anti-bias algorithm delivered a water intervention to that side — i.e. `trial.metadata.extra.is_bias_water_intervention` is `True` **and** `is_auto_reward_right` points to that side (`False` → left, `True` → right). The anti-bias water uses the same auto-response channel as ordinary autowater, so the `is_bias_water_intervention` flag is what distinguishes it. `False` otherwise. Deliberately **not** gated on `is_rewarded`: these columns record what the anti-bias algorithm did, and the intervention fires at the go cue regardless of how the animal's own choice resolves, so they can be `True` where `auto_waterL` / `auto_waterR` is `0`. |
 | `anti_bias_lickspout_movement` | Signed horizontal displacement (mm, positive is rightward) the anti-bias algorithm moved the lickspouts on this trial: `trial.lickspout_offset_delta` when `trial.metadata.extra.is_bias_stage_intervention` is `True`, else `0.0`. |
 | `bait_left` / `bait_right` | Boolean. `bait_right` is `True` if `p_reward_right == 1` and `is_auto_reward_right` is `None` or `False`. `bait_left` is `True` if `p_reward_left == 1` and `is_auto_reward_right` is `None` or `True`. |
 | `response_duration` | `response_deadline_duration`. |
@@ -191,3 +192,7 @@ These were mapped during exploration but are no longer in scope:
 | 2026-08-06 | Confirmed and documented that the legacy `delay_*` columns describe the acquisition software's **quiescence period**: `delay_start_time` is the `QuiescentPeriod` timestamp (always equal to the new `quiescent_start_time`) and `delay_duration` / `delay_beta` / `delay_min` / `delay_max` summarize `quiescence_period_duration`. `delay_duration` is the *configured* duration — each lick restarts the quiescent period, so the realized `quiescent_stop_time - quiescent_start_time` can be longer. Column descriptions updated accordingly. |
 | 2026-08-12 | `rewarded_historyL` / `rewarded_historyR` now record **earned** reward only: an auto-reward trial (`is_auto_reward_right` set to either side) is `False` on *both* sides, since `TrialOutcome.is_rewarded` is `True` for autowater too and that water is already reported by `auto_waterL` / `auto_waterR`. This matches the `earned` / `automatic` split used for the NWB reward-delivery annotations. |
 | 2026-08-12 | `min_reward_each_block` is now `0` rather than `NULL` when the trial generator exposes no `min_block_reward` — no per-block minimum is a floor of zero, not an unknown. The column is non-nullable (`float`, default `0`). |
+| 2026-08-17 | Reward-delivery annotations now match each delivery to its trial by the `Response` software-event timestamp rather than the `TrialOutcome` timestamp. `TrialOutcome` fires at the *end* of a trial (after the reward-consumption and ITI periods), so a delivery could land nearer the *previous* trial's outcome and inherit its `is_auto_reward_right`, flipping `earned` and `auto`. The valve opens within milliseconds of the response, so the response anchors the delivery to its own trial. |
+| 2026-08-17 | `auto_waterL` / `auto_waterR` are now gated on `is_rewarded`: autowater on a trial that did not pay out is `0`. Free water is triggered immediately at the go cue and the trial then "continues normally", so `is_rewarded` reports the outcome of the animal's own choice — a separate event from the autowater. Gating keeps these columns equal to the `auto` count in the reward-delivery series, which drops the same deliveries, and makes the retained delivery total equal the metadata mapper's `sum(is_rewarded)`. This is narrower than the legacy ungated column of the same name. |
+| 2026-08-17 | `anti_bias_left_water` / `anti_bias_right_water` are explicitly **not** gated on `is_rewarded`, unlike `auto_water*`: they record what the anti-bias algorithm did, and the intervention fires regardless of the trial's outcome. They are therefore no longer a subset of `auto_water*` — an intervention on a trial that did not pay out appears in the anti-bias column and not in the autowater column. |
+| 2026-08-17 | Added `auto_water_offeredL` / `auto_water_offeredR`: autowater ungated by `is_rewarded`, i.e. the legacy `dynamic-foraging-task` meaning of `auto_waterL` / `auto_waterR` (`B_AutoWaterTrial`). With `auto_water*` now reward-keyed to match the reward-delivery series, these keep every autowater delivery recorded in the trial table, so nothing is lost: `auto_water_offered* - auto_water*` is the autowater delivered on trials that did not pay out. |

src/dynamic_foraging_processing/nwb/acquisition/acquisition_builder.py

@@ -11,7 +11,7 @@
 )
 from dynamic_foraging_processing.nwb.utils import clean_for_nwb
 from dynamic_foraging_processing.raw_data_loader import RawDataLoader
-from dynamic_foraging_processing.utils.rewards import get_annotated_rewards
+from dynamic_foraging_processing.utils.rewards import get_reward_deliveries
 
 
 class LickSource(t.NamedTuple):
@@ -49,8 +49,8 @@ def __init__(self, loader: RawDataLoader):
         """
         self.loader = loader
 
-    def get_reward_delivery(self) -> pd.DataFrame:
-        """Get the reward delivery stream from the dataset.
+    def get_valve_writes(self) -> pd.DataFrame:
+        """Get the raw valve command stream.
 
         Returns
         -------
@@ -76,6 +76,26 @@ def get_trial_outcomes(self) -> pd.DataFrame:
             self.loader.dataset.at("Behavior").at("SoftwareEvents").at("TrialOutcome").load().data
         )
 
+    def get_response_times(self) -> np.ndarray:
+        """Get the per-trial ``Response`` software-event timestamps.
+
+        The event fires when the animal's choice is registered, within
+        milliseconds of the valve opening, so it anchors a reward delivery to
+        its trial. Only the event timestamp is used; the payload's ``Item1``
+        field nominally carries a response time but is unreliable (it can lag
+        the event by thousands of seconds), so it is ignored.
+
+        Returns
+        -------
+        numpy.ndarray
+            The ``Response`` event timestamps, positionally aligned with the
+            ``TrialOutcome`` stream.
+        """
+        responses = (
+            self.loader.dataset.at("Behavior").at("SoftwareEvents").at("Response").load().data
+        )
+        return responses.index.to_numpy()
+
     def get_manual_water_times(self) -> pd.DataFrame:
         """Get the manual-water software-event stream.
 
@@ -172,6 +192,7 @@ def _reward_delivery_series(
         writes: pd.DataFrame,
         trial_outcomes: pd.DataFrame,
         manual_water: pd.DataFrame,
+        response_times: np.ndarray,
         *,
         port_column: str,
         is_right: bool,
@@ -181,8 +202,10 @@ def _reward_delivery_series(
         """Build one lick port's reward-delivery series with reward annotations.
 
         Only valve-open events (``port_column`` is truthy) are reward
-        deliveries; the ``data`` field annotates each as earned, manual, or
-        auto via :func:`get_annotated_rewards`.
+        deliveries; the ``data`` field annotates each as earned, manual, or auto
+        via :func:`get_reward_deliveries`, which also drops deliveries on trials
+        that did not pay out, so the series reports reward rather than every
+        valve opening.
 
         Parameters
         ----------
@@ -193,6 +216,9 @@ def _reward_delivery_series(
         manual_water : pandas.DataFrame
             The ``GiveManualWaterRight`` stream; the ``data`` column selects the
             side (``True`` right, ``False`` left).
+        response_times : numpy.ndarray
+            ``Response`` event timestamps, one per trial, used to match each
+            delivery to its trial.
         port_column : str
             Supply-port column for this side (``"SupplyPort0"`` left,
             ``"SupplyPort1"`` right).
@@ -211,10 +237,11 @@ def _reward_delivery_series(
         open_writes = writes[writes[port_column].fillna(False).astype(bool)]
         delivery_times = open_writes.index.to_numpy()
         manual_water_times = manual_water.index[manual_water["data"] == is_right].to_numpy()
-        annotations = get_annotated_rewards(
+        delivery_times, annotations = get_reward_deliveries(
             delivery_times,
             trial_outcomes,
             manual_water_times,
+            response_times,
         )
         return AcquisitionSeries(
             name=name,
@@ -248,9 +275,10 @@ def build_acquisition(
         list of AcquisitionSeries or AcquisitionTable
             Acquisition entries to write to the NWB acquisition module.
         """
-        rewards = self.get_reward_delivery()
+        rewards = self.get_valve_writes()
         trial_outcomes = self.get_trial_outcomes()
         manual_water = self.get_manual_water_times()
+        response_times = self.get_response_times()
 
         acquisition_streams = self.loader.get_all_raw_data()
         acqusition_streams_descriptions = self.loader.raw_data_stream_descriptions
@@ -274,6 +302,7 @@ def build_acquisition(
                 rewards,
                 trial_outcomes,
                 manual_water,
+                response_times,
                 port_column="SupplyPort0",
                 is_right=False,
                 name="left_reward_delivery_time",
@@ -285,6 +314,7 @@ def build_acquisition(
                 rewards,
                 trial_outcomes,
                 manual_water,
+                response_times,
                 port_column="SupplyPort1",
                 is_right=True,
                 name="right_reward_delivery_time",

src/dynamic_foraging_processing/processing/_trial_table.py

@@ -353,7 +353,7 @@ def _rewarded_history(
         (``trial.is_auto_reward_right is not None``) is ``False`` on *both*
         sides here — its water is reported by ``auto_waterL``/``auto_waterR``
         instead. This matches the ``earned``/``automatic`` split in
-        :func:`~dynamic_foraging_processing.utils.rewards.get_annotated_rewards`.
+        :func:`~dynamic_foraging_processing.utils.rewards.get_reward_deliveries`.
 
         A trial with no reward or an ignored trial (no choice) likewise counts
         as not rewarded on either side (``False``).
@@ -442,12 +442,61 @@ def _is_baited(trial: Trial, *, is_right: bool) -> bool:
         return trial.p_reward_left == 1 and auto in (None, True)
 
     @staticmethod
-    def _auto_water(trial: Trial, *, is_right: bool) -> int:
-        """Encode autowater for a side from ``is_auto_reward_right``.
+    def _auto_water(trial: Trial, outcome: TrialOutcome, *, is_right: bool) -> int:
+        """Return whether autowater was rewarded on the requested side.
 
-        Returns ``1`` if the auto response was to the requested side, else ``0``.
-        No auto-response (``is_auto_reward_right`` is ``None``) counts as no
-        autowater (``0``). ``is_right`` is ``True`` for right.
+        ``is_auto_reward_right`` triggers an immediate reward to one side
+        (``True`` right, ``False`` left, ``None`` no autowater). The anti-bias
+        water intervention is itself autowater delivered through this channel, so
+        every auto-triggered reward counts here; ``anti_bias_left_water`` /
+        ``anti_bias_right_water`` mark the subset the anti-bias algorithm drove.
+
+        Autowater on a trial that did not pay out (``is_rewarded`` is ``False``)
+        is ``0``, so this column matches the reward-keyed reward-delivery series:
+        the water fires at the go cue whether or not the animal's own choice
+        later pays out, and the series reports only water that was reward.
+
+        Parameters
+        ----------
+        trial : Trial
+            The per-trial task-logic model.
+        outcome : TrialOutcome
+            The trial's outcome, read for ``is_rewarded``.
+        is_right : bool
+            ``True`` for the right port, ``False`` for the left port.
+
+        Returns
+        -------
+        int
+            ``1`` when rewarded autowater targeted the requested side, else ``0``.
+        """
+        if trial.is_auto_reward_right is None or not outcome.is_rewarded:
+            return 0
+        return int(trial.is_auto_reward_right is is_right)
+
+    @staticmethod
+    def _auto_water_offered(trial: Trial, *, is_right: bool) -> int:
+        """Return whether autowater was *offered* to the requested side.
+
+        The ungated counterpart of :meth:`_auto_water`: ``1`` whenever the trial
+        auto-triggered a reward to this side, whether or not the trial went on to
+        pay out. This is the legacy ``dynamic-foraging-task`` meaning of
+        ``auto_waterL``/``auto_waterR`` (its ``B_AutoWaterTrial``, which that GUI
+        also passes to foraging efficiency as ``autowater_offered``), kept so the
+        trial table still records every autowater delivery while ``auto_water*``
+        stays reward-keyed to match the reward-delivery series.
+
+        Parameters
+        ----------
+        trial : Trial
+            The per-trial task-logic model.
+        is_right : bool
+            ``True`` for the right port, ``False`` for the left port.
+
+        Returns
+        -------
+        int
+            ``1`` when autowater was offered to the requested side, else ``0``.
         """
         if trial.is_auto_reward_right is None:
             return 0
@@ -461,9 +510,9 @@ def _bias_metadata(trial: Trial) -> BlockBasedTrialMetadata:
         ``is_bias_stage_intervention``) live on ``trial.metadata.extra``. That
         field is schema-typed ``Any``, so it deserializes off the stream as a
         plain ``dict`` rather than a model; a ``BlockBasedTrialMetadata``
-        instance is also accepted. When metadata or extra is missing (e.g. an
-        older session, or a non-block-based generator), the model's all-``False``
-        default is returned so the anti-bias columns are simply inert.
+        instance is also accepted. When metadata or extra is missing (e.g. a
+        non-block-based generator), the model's all-``False`` default is returned
+        so the anti-bias columns are simply inert.
 
         Parameters
         ----------
@@ -497,6 +546,13 @@ def _anti_bias_water(
         was a bias-water intervention *and* the auto-response was to the
         requested side.
 
+        This records what the *algorithm* did, so it is not gated on
+        ``is_rewarded``: the intervention fires at the go cue regardless of how
+        the animal's own choice later resolves. It is therefore not a subset of
+        ``auto_waterL``/``auto_waterR``, which count only rewarded autowater --
+        an intervention on a trial that did not pay out appears here and not
+        there.
+
         Parameters
         ----------
         trial : Trial
@@ -897,8 +953,10 @@ def _build_row(
             reward_consumption_duration=trial.reward_consumption_duration,
             ITI_duration=trial.inter_trial_interval_duration,
             delay_duration=trial.quiescence_period_duration,
-            auto_waterL=self._auto_water(trial, is_right=False),
-            auto_waterR=self._auto_water(trial, is_right=True),
+            auto_waterL=self._auto_water(trial, outcome, is_right=False),
+            auto_waterR=self._auto_water(trial, outcome, is_right=True),
+            auto_water_offeredL=self._auto_water_offered(trial, is_right=False),
+            auto_water_offeredR=self._auto_water_offered(trial, is_right=True),
             anti_bias_left_water=self._anti_bias_water(trial, bias_metadata, is_right=False),
             anti_bias_right_water=self._anti_bias_water(trial, bias_metadata, is_right=True),
             anti_bias_lickspout_movement=self._anti_bias_lickspout_movement(trial, bias_metadata),