Estimating Total Effects in Bipartite Experiments with Spillovers and Partial Eligibility

Explainable & Ethical AI
Published: arXiv: 2511.11564v1
Authors

Albert Tan Mohsen Bayati James Nordlund Roman Istomin

Abstract

We study randomized experiments in bipartite systems where only a subset of treatment-side units are eligible for assignment while all units continue to interact, generating interference. We formalize eligibility-constrained bipartite experiments and define estimands aligned with full deployment: the Primary Total Treatment Effect (PTTE) on eligible units and the Secondary Total Treatment Effect (STTE) on ineligible units. Under randomization within the eligible set, we give identification conditions and develop interference-aware ensemble estimators that combine exposure mappings, generalized propensity scores, and flexible machine learning. We further introduce a projection that links treatment- and outcome-level estimands; this mapping is exact under a Linear Additive Edges condition and enables estimation on the (typically much smaller) treatment side with deterministic aggregation to outcomes. In simulations with known ground truth across realistic exposure regimes, the proposed estimators recover PTTE and STTE with low bias and variance and reduce the bias that could arise when interference is ignored. Two field experiments illustrate practical relevance: our method corrects the direction of expected interference bias for a pre-specified metric in both studies and reverses the sign and significance of the primary decision metric in one case.

Paper Summary

Problem
In traditional A/B testing, the Stable Unit Treatment Value Assumption (SUTVA) is often violated, resulting in interference or spillover effects. This means that the outcome of one unit can be affected by the treatment of another unit, especially in networked settings like ride-sharing services. The problem is that only a subset of treatment-side units may be eligible for assignment, while all units continue to interact and generate interference.
Key Innovation
This paper introduces a new framework for estimating total effects in bipartite experiments with spillovers and partial eligibility. The key innovation is the development of two new estimands: the Primary Total Treatment Effect (PTTE) and the Secondary Total Treatment Effect (STTE). The PTTE measures the impact of a treatment on the eligible units, while the STTE measures the impact on the ineligible units. The paper also proposes flexible estimators that leverage generalized propensity scores and machine learning to estimate these effects.
Practical Impact
The practical impact of this research is significant. In settings with interaction across unit types, effect definitions and estimators that target the total impact at rollout can lead to different conclusions than conventional A/B analyses. By accounting for interference, the proposed methods can yield effect estimates that differ materially from analyses that ignore spillovers. This is illustrated in two case studies, where the proposed method corrects the direction of expected interference bias and reverses the sign and significance of the primary decision metric in one case.
Analogy / Intuitive Explanation
Think of a ride-sharing service like a social network, where drivers and riders interact and affect each other's outcomes. Imagine that only a subset of drivers are eligible for a new routing policy, while all drivers continue to interact and generate interference. The Primary Total Treatment Effect (PTTE) measures the impact of the new policy on the eligible drivers, while the Secondary Total Treatment Effect (STTE) measures the impact on the ineligible drivers. By accounting for this interference, the proposed method can provide a more accurate estimate of the total effect of the new policy on the entire system.
Paper Information
Categories:
stat.ME cs.LG stat.ML
Published Date:

arXiv ID:

2511.11564v1

Quick Actions