Rita Zapata-Vazquez, Anthony O’Hagan and Leo Soares Bastos
School of Mathematics and Statistics, University of Sheffield, UK, Facultad de Medicina, Universidad Autonoma de Yucatan, Mexico and Scientific Computational Program, Oswaldo Cruz Foundation, Brazil
Publication details: Journal of Appplied Statistics 41, 1919-1933, 2014.
Eliciting expert knowledge about several uncertain quantities is a complex task when those quantities exhibit associations. A well-known example of such a problem is eliciting knowledge about a set of uncertain proportions which must sum to 1. The usual approach is to assume that the expert's knowledge can be adequately represented by a Dirichlet distribution, since this is by far the simplest multivariate distribution that is appropriate for such a set of proportions. It is also the most convenient, particularly when the expert's prior knowledge is to be combined with a multinomial sample since then the Dirichlet is the conjugate prior family.
Several methods have been described in the literature for eliciting beliefs in the form of a Dirichlet distribution, which typically involve eliciting from the expert enough judgements to identify uniquely the Dirichlet hyperparameters.
We describe here a new method which employs the device of over-fitting, i.e. eliciting more than the minimal number of judgements, in order to (a) produce a more carefully considered Dirichlet distribution and (b) ensure that the Dirichlet distribution is indeed a reasonable fit to the expert's knowledge. The method has been implemented in a software extension of the Sheffield Elicitation Framework (SHELF) to facilitate the multivariate elicitation process.
Keywords: elicitation; Dirichlet distribution; SHELF; over-fitting.