Speaker: Neil Xu

Abstract: Suppose that there are K parameters of potential interest, and that one can construct a valid (1-\delta)-CI for each of them separately. A data analyst uses an arbitrary data-dependent criterion to select some subset S of them for reporting, or highlighting. The confidence intervals for the selected parameters are no longer valid, due to the selection bias, so the question is how one must adjust these in order to account for selection. We focus on the popular notion of false coverage rate (FCR), which is the expected ratio of the number of selected intervals that miscover, to the number of selected intervals |S|. The main established method is the ``BY procedure'' from a seminal work by Benjamini and Yekutieli (JASA, 2005), that was inspired by the Benjamini-Hochberg (BH) procedure. Unfortunately, the BY procedure involves restrictions on the dependence between CIs and the selection criterion. We propose a natural and much simpler method---both in implementation, and in proof---which is valid under any dependence structure between the original CIs, and any (unknown) selection criterion, but which only applies to a special, yet broad, class of CIs. Our procedure reports (1-\delta|S|/K)-CIs for the selected parameters, and we prove that it controls the FCR at \delta for confidence intervals that implicitly invert e-values; examples include those constructed via self-normalized supermartingale methods, or via universal inference, or via Chernoff-style bounds on the moment generating function, among others. We call it the *e-BY procedure*, since it is inspired by the aforementioned BY procedure, and strongly borrows intuition from the recent e-BH procedure of Wang and Ramdas (JRSSB, 2022). This work has strong implications for multiple testing in sequential settings, since it applies at stopping times, to continuously-monitored confidence sequences along with multi-armed bandit sampling. https://arxiv.org/abs/2203.12572