Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,7 @@ Full guide: `diff_diff.get_llm_guide("practitioner")`.
- [Manipulation Testing](https://diff-diff.readthedocs.io/en/stable/api/regression_discontinuity.html) - Cattaneo, Jansson & Ma (2020) density-discontinuity test (`RDDensityTest`): rddensity 3.0 parity, robust bias-corrected inference, unrestricted/restricted models, mass-point adjustment
- [Parallel Trends Testing](https://diff-diff.readthedocs.io/en/stable/api/diagnostics.html) - simple and Wasserstein-robust parallel trends tests, equivalence testing (TOST)
- [Placebo Tests](https://diff-diff.readthedocs.io/en/stable/api/diagnostics.html) - placebo timing, group, permutation, leave-one-out
- [TWFE Weight Diagnostics](https://diff-diff.readthedocs.io/en/stable/api/twfe_weights.html) - Baker et al. (2025) implicit weights a TWFE regression places on each ATT(g,t), against the ATT^O / ATT^simple targets, with the pre-trend contribution. Ports Callaway's `twfeweights` (MIT)
- [Honest DiD](https://diff-diff.readthedocs.io/en/stable/api/honest_did.html) - Rambachan & Roth (2023) sensitivity analysis: robust CI under PT violations, breakdown values
- [Pre-Trends Power Analysis](https://diff-diff.readthedocs.io/en/stable/api/pretrends.html) - Roth (2022) minimum detectable violation and power curves
- [Power Analysis](https://diff-diff.readthedocs.io/en/stable/api/power.html) - analytical and simulation-based MDE, sample size, power curves for study design
Expand Down
510 changes: 510 additions & 0 deletions benchmarks/R/generate_twfeweights_golden.R

Large diffs are not rendered by default.

6 changes: 5 additions & 1 deletion benchmarks/R/requirements.R
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@ required_packages <- c(
"Synth", # Abadie-Diamond-Hainmueller (2010) synthetic control (SyntheticControl R-parity; ships data(basque))
"qte", # Callaway qte package (Athey-Imbens CiC + QDiD R-parity; ships data(lalonde))
"ptetools", # Callaway ptetools (badcontrols dependency; bad-control DMLDiD lane black-box parity)
"BMisc", # Callaway utility package (twfeweights dependency: weighted_ecdf, orig2t)
"DRDID", # Sant'Anna & Zhao (2020) doubly-robust DiD (twfeweights AIPW dependency)

# Utilities
"jsonlite", # JSON output for Python interop
Expand All @@ -28,7 +30,9 @@ required_packages <- c(

# synthdid must be installed from GitHub
github_packages <- list(
synthdid = "synth-inference/synthdid"
synthdid = "synth-inference/synthdid",
# TWFE weight diagnostics parity goldens (not on CRAN)
twfeweights = "bcallaway11/twfeweights"
)

install_if_missing <- function(pkg) {
Expand Down
586 changes: 586 additions & 0 deletions benchmarks/data/twfeweights_golden.json

Large diffs are not rendered by default.

1,501 changes: 1,501 additions & 0 deletions benchmarks/data/twfeweights_sim_panel.csv

Large diffs are not rendered by default.

1,501 changes: 1,501 additions & 0 deletions benchmarks/data/twfeweights_unbalanced_panel.csv

Large diffs are not rendered by default.

24 changes: 24 additions & 0 deletions changelog.d/20260831-twfe-weight-diagnostics.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
### Added
- **TWFE weight diagnostics** (port of Brantly Callaway's `twfeweights` R
package, MIT): what a two-way fixed effects regression *implicitly* weights
on staggered-adoption data.
- `attgt_weights(results, aggregation="twfe"|"overall"|"simple")` reports the
weight a TWFE regression, ATT^O, or ATT^simple places on each ATT(g,t),
plus post-period negative-weight counts. Returns `ATTGTWeightsResult`.
The CS estimands honour the fit's `anticipation` window (an explicit
`anticipation=` on the frame path); `"twfe"` keeps `t >= g`.
- `decompose_twfe_weights(data, ..., method="fwl")` re-derives the estimate
from its ATT(g,t) building blocks and returns `TWFEDecompositionResult`
with `pre_period_contribution` - the sample contribution of the
pre-treatment cells, which can reflect parallel-trends violations or
sampling variation - and, with `balance_covariates=`, implicit-weight
covariate balance. `plot_twfe_weights()` renders either view (matplotlib
or plotly).
- Validation: rejects NaN / `-inf` cohort labels, covariate-adjusted fits
under `aggregation="twfe"`, unbalanced panels, non-finite outcomes /
covariates, duplicated or non-finite ATT(g,t) cells, an incomplete
group-time grid, and invalid sampling weights. Two structural gaps are
handled as R does instead of raising: a cohort with no estimable post cell
is dropped (`did`'s first-period drop, or running out of comparison
units), and under `control_group="not_yet_treated"` the CS estimands
average over each cohort's available post periods (`aggte`).
17 changes: 17 additions & 0 deletions diff_diff/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -298,6 +298,14 @@
TROPResults,
trop,
)
from diff_diff.twfe_weights import (
attgt_weights,
decompose_twfe_weights,
)
from diff_diff.twfe_weights_results import (
ATTGTWeightsResult,
TWFEDecompositionResult,
)
from diff_diff.two_stage import (
TwoStageBootstrapResults,
TwoStageDiD,
Expand All @@ -323,6 +331,7 @@
plot_sensitivity,
plot_staircase,
plot_synth_weights,
plot_twfe_weights,
)
from diff_diff.wooldridge import WooldridgeDiD
from diff_diff.wooldridge_results import WooldridgeDiDResults
Expand Down Expand Up @@ -459,6 +468,13 @@ def __getattr__(name: str) -> _Any:
"TWFEWeightsResult",
"chaisemartin_dhaultfoeuille",
"twowayfeweights",
# TWFE weight diagnostics (Callaway `twfeweights` port) - distinct from
# the dCDH `twowayfeweights` surface above: these weight ATT(g,t)
# parameters, not (unit, time) cells.
"ATTGTWeightsResult",
"TWFEDecompositionResult",
"attgt_weights",
"decompose_twfe_weights",
# WooldridgeDiD (ETWFE)
"WooldridgeDiD",
"WooldridgeDiDResults",
Expand All @@ -479,6 +495,7 @@ def __getattr__(name: str) -> _Any:
"DurationDiDPretestResults",
# Visualization
"plot_bacon",
"plot_twfe_weights",
"plot_event_study",
"plot_group_effects",
"plot_sensitivity",
Expand Down
1 change: 1 addition & 0 deletions diff_diff/dml_did.py
Original file line number Diff line number Diff line change
Expand Up @@ -3362,6 +3362,7 @@ def fit(
# stays admitted).
is_survey_fit=survey_metadata is not None,
bootstrap_results=bootstrap_results,
covariates=covariates,
)
self.results_ = results
self.is_fitted_ = True
Expand Down
104 changes: 104 additions & 0 deletions diff_diff/guides/llms-full.txt
Original file line number Diff line number Diff line change
Expand Up @@ -1534,6 +1534,60 @@ results.print_summary()
plot_bacon(results)
```

### TWFE Weight Diagnostics

What a TWFE regression implicitly weights on staggered data. Distinct from
`twowayfeweights` (dCDH), which weights (unit, time) cells: these weight
ATT(g,t) parameters. Ported from Brantly Callaway's `twfeweights` R package
(MIT); methodology Baker, Callaway, Cunningham, Goodman-Bacon & Sant'Anna
(2025).

```python
attgt_weights(
results, # CallawaySantAnnaResults, or a (g,t) frame
aggregation="twfe", # "twfe" | "overall" (ATT^O) | "simple"
data=None, unit=None, time=None, first_treat=None, # frame path only
weights=None, # unit-level sampling weights
) -> ATTGTWeightsResult

decompose_twfe_weights(
data, # balanced long panel (it re-estimates)
outcome=, unit=, time=, first_treat=,
method="fwl",
covariates=None,
base_period="first_period", # or "gmin1"
balance_covariates=None, # enables result.covariate_balance()
weights=None,
) -> TWFEDecompositionResult

plot_twfe_weights(result, kind="auto") # "weights" | "balance"
```

`aggregation="twfe"` requires a fit with `base_period="universal"`,
`control_group="never_treated"` AND no covariates (R twfe_weights' three
restrictions); it raises otherwise. ATT^O and ATT^simple weights are
non-negative and sum to one, so comparing `implied_att` across the three
aggregations shows what the TWFE specification costs.

Both entry points fail closed on input R never faced: NaN / `-inf` cohort
labels (never-treated is exactly `0` or `+inf`), duplicated or non-finite
ATT(g,t) cells, an incomplete group-time grid (`"twfe"` needs every cohort x
period cell, the CS estimands every post cell), and sampling weights that are
not finite, non-negative and positive-mass. Two structural gaps mirror R
rather than raising, each with a `UserWarning`: a cohort with no estimable
post cell is dropped from the table and the cohort shares (`did`'s
first-period drop), and under `control_group="not_yet_treated"` the cells CS
marks `zero_treated_control` are treated as structurally absent, so
`"overall"`/`"simple"` average over each cohort's AVAILABLE post periods
(`aggte`). `n_negative_post` / `negative_post_weight_share` report the
pathology (negative weight on POST cells); `n_negative` counts pre cells too,
and is near-half in every staggered design because the TWFE weights sum to
zero over the full grid.

`decompose_twfe_weights` takes the raw panel rather than a fitted result
because it re-estimates. It is tied to `attgt_weights` by an identity:
`attgt_weights(cs, aggregation="twfe").implied_att == decompose_twfe_weights(panel, ...).estimate`.

### StaggeredTripleDifference

DEPRECATED in 3.9, removed in 4.0 (ledger row M-013). Use
Expand Down Expand Up @@ -2017,6 +2071,34 @@ Returned by `BaconDecomposition.fit()` (and the deprecated `bacon_decompose()` w

**Methods:** `summary()`, `print_summary()`, `to_dataframe()`

### ATTGTWeightsResult

Diagnostic result from `attgt_weights`. No inference quintet - the
decomposition is an algebraic identity.

- `weights`: DataFrame with `group`, `time`, `post`, `weight`, `att`
- `implied_att`: `sum(weight * att)` - the TWFE coefficient when
`aggregation="twfe"`
- `n_negative`, `negative_weight_share`: the staggered-TWFE pathology
- `aggregation`, `source`, `control_group`, `base_period`, `n_cells`
- `summary()`, `to_dataframe()`, `to_dict()`

### TWFEDecompositionResult

Diagnostic result from `decompose_twfe_weights`.

- `cells`: DataFrame with `group`, `time`, `post`, `att`, `weight`, `ess`,
`remainder`
- `estimate` == `decomposition` + `remainder`
- `pre_period_contribution`: sample contribution of the PRE-treatment cells
(can reflect differential pre-trends OR sampling variation; diagnostic
evidence, not proof the post-treatment assumption fails)
- `post_only`, `effective_sample_size`, `covariates`, `base_period`
- `covariate_balance(level="summary"|"cell", standardize=True,
post_only=True)`: implicit-weight covariate balance; raises when
`balance_covariates=` was not requested
- `summary()`, `to_dataframe()`, `to_dict()`

### Comparison2x2

Individual 2x2 DiD comparison (used in BaconDecompositionResults).
Expand Down Expand Up @@ -2532,6 +2614,28 @@ plot_bacon(
)
```

### plot_twfe_weights

```python
from diff_diff import plot_twfe_weights

plot_twfe_weights(
results, # ATTGTWeightsResult | TWFEDecompositionResult
kind="auto", # "weights" | "balance" ("auto" picks balance
# when the result carries a balance table)
standardize=True, absolute_value=True, # balance view
annotate=False, ax=None, show=True,
backend="matplotlib", # or "plotly"
)
```

`kind="weights"` scatters weight against ATT(g,t), coloured by pre/post - points
left of the vertical zero line carry negative weight; the title counts POST-only
negatives. `kind="balance"` scatters unweighted against implicitly-weighted
covariate differences; points near the horizontal axis are covariates the
implicit weights balance. With `absolute_value=False` the reference diagonal
spans `[-limit, limit]` so signed differences read correctly.

### plot_power_curve

```python
Expand Down
1 change: 1 addition & 0 deletions diff_diff/guides/llms.txt
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,7 @@ The site is organized into 5 sections, each with a landing page:
- [Manipulation Testing](https://diff-diff.readthedocs.io/en/stable/api/regression_discontinuity.html): Cattaneo, Jansson & Ma (2020) density-discontinuity manipulation test (`RDDensityTest`), parity with R rddensity 3.0 - boundary-adaptive local polynomial density estimation at the cutoff, robust bias-corrected inference, unrestricted/restricted models, jackknife/plugin variances, data-driven bandwidths, mass-point adjustment
- [Parallel Trends Testing](https://diff-diff.readthedocs.io/en/stable/api/diagnostics.html): Simple and Wasserstein-robust parallel trends tests, equivalence testing (TOST)
- [Placebo Tests](https://diff-diff.readthedocs.io/en/stable/api/diagnostics.html): Placebo timing, group, permutation, and leave-one-out diagnostics
- [TWFE Weight Diagnostics](https://diff-diff.readthedocs.io/en/stable/api/twfe_weights.html): Baker et al. (2025) implicit weights on ATT(g,t) - `attgt_weights(results, aggregation='twfe'|'overall'|'simple')` takes a fitted `CallawaySantAnnaResults` (raw ATT(g,t) frame + panel as fallback) and returns the weight each estimand places on each group-time effect, with the negative-weight share; `decompose_twfe_weights(data, outcome=, unit=, time=, first_treat=, method='fwl', covariates=)` re-derives the TWFE estimate from its ATT(g,t) building blocks with `pre_period_contribution` (the pre-treatment cells' sample contribution, which can be pre-trends or sampling noise), and `result.covariate_balance()` reports implicit-weight covariate balance. Plot with `plot_twfe_weights`. R `twfeweights` 0.9.0 output parity
- [Honest DiD](https://diff-diff.readthedocs.io/en/stable/api/honest_did.html): Rambachan & Roth (2023) sensitivity analysis — robust CI under parallel trends violations, breakdown values
- [Pre-Trends Power Analysis](https://diff-diff.readthedocs.io/en/stable/api/pretrends.html): Roth (2022) Section II.A-B no-individually-significant (NIS) box-probability pretest power + minimum detectable violation; `pretest_form='nis'` (default) implements the paper's primary form, `pretest_form='wald'` retained as paper-supported alternative (Propositions 1+3+4 all apply); linear-violation MDV in Roth's γ units when relative-time labels are threaded through `fit()`; full Σ_22 routing on non-bootstrap CallawaySantAnna and SunAbraham adapters and on admitted CS-/StackedDiD-sourced `aggregate('event_study')` containers (StackedDiD persists its ES VCV in every inference mode)
- [Power Analysis](https://diff-diff.readthedocs.io/en/stable/api/power.html): Analytical and simulation-based power analysis — MDE, sample size, power curves for study design
Expand Down
14 changes: 13 additions & 1 deletion diff_diff/staggered.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@

import bisect
import warnings
from typing import TYPE_CHECKING, Any, Dict, List, Optional, Tuple
from typing import TYPE_CHECKING, Any, Dict, List, Optional, Sequence, Tuple

import numpy as np
import pandas as pd
Expand Down Expand Up @@ -3056,6 +3056,7 @@ def fit(
group_time_effects,
is_survey_fit=survey_metadata is not None,
bootstrap_results=bootstrap_results,
covariates=covariates,
)

self.is_fitted_ = True
Expand Down Expand Up @@ -5151,6 +5152,7 @@ def _build_aggregation_kit(
*,
is_survey_fit: bool = False,
bootstrap_results: Optional["CSBootstrapResults"] = None,
covariates: Optional[Sequence[str]] = None,
) -> Optional["AggregationKit"]:
"""Distil the fit-time state post-fit re-aggregation needs.

Expand Down Expand Up @@ -5185,6 +5187,16 @@ def _build_aggregation_kit(
# (or DDD) survey fit does not warn as "CallawaySantAnna" on post-fit
# aggregate(). Legacy kits without the key default at the read site.
bookkeeping["bootstrap_label"] = getattr(estimator, "_BOOTSTRAP_LABEL", "CallawaySantAnna")
# Covariate usage, recorded so downstream diagnostics can refuse designs
# their formulas do not cover (``attgt_weights(aggregation="twfe")``
# mirrors R twfe_weights' ``xformla == ~1`` restriction). Column NAMES
# only - never values - so the data-minimization contract holds.
bookkeeping["covariates"] = tuple(covariates or ())
# Panel balance, recorded so ``attgt_weights`` can reject an unbalanced
# fitted result: its cohort shares and E_t[D] assume the same units in
# every period. Defaults True when the key is absent (a legacy kit, or a
# producer that never computed it).
bookkeeping["is_balanced"] = bool((precomputed or {}).get("is_balanced", True))

# Data minimization: the results object is picklable and users share
# result artifacts, so the kit must not turn it into a carrier for raw
Expand Down
Loading