Multiple Hypothesis Testing and Testing One Hypothesis Multiple Times: a unified (re)view


Ms Sara Algeri (Imperial College London)


In statistics, the problem of testing one hypothesis multiple times can be formulated in terms of hypothesis testing when a nuisance parameter is present only under the alternative, also known as “look-elsewhere effect”. Each possible value of the nuisance parameter specifies a different alternative hypothesis and a unique global p-value is provided to summarize the statistical evidence in support (or against) the null hypothesis. From a physics perspective, this scenario occurs quite often in the searches for new signals over an energy or mass spectrum, and in both nested and non-nested frameworks. An alternative way to search for new emissions is to refer to the classical and widely known multiple hypothesis testing approach. Separate tests of hypothesis are conducted at different locations producing an ensemble of local p-values, the smallest is reported as evidence for the new resonance, once adequately adjusted to control the false detection rate (type I error rate). The aim of this talk is to review both approaches, and propose a graphical tool to identify recurrent scenarios where a simple multiple hypothesis testing procedure can be used to provide valid inference with respect to stringent significance requirements, without encountering the usual problem of over-conservativeness.

Primary author

Ms Sara Algeri (Imperial College London)


Prof. David van Dyk (Imperial College London) Prof. Jan Conrad (The Oskar Klein Centre for Cosmoparticle Physics)

Presentation Materials

Your browser is out of date!

Update your browser to view this website correctly. Update my browser now