Tony O'Hagan - Consulting
Consultancy in elicitation
My expertise
I have been a leading proponent in the Bayesian statistics community of the use of
elicitation to formulate expert knowledge about uncertain quantities. I recently led
a major research project, funded by the National Health Service, into methods for
elicitation. One of the outputs of that research was the most comprehensive survey
of the field to date, published by Wiley in 2006. I am co-developer of the
SHELF package of elicitation tools,
and have pioneered methods to quantify the imprecision in elicited distributions.
I can offer various options for training in elicitation. I have an established
one-day training course in the theory and methods of elicitation,
and can also arrange mock elicitation sessions to further develop facilitation skills.
My clients
I have conducted expert elicitations for HMIP (now part of the Health Protection Agency),
Risk Management Solutions Ltd, AstraZeneca and Pfizer. Methods that I developed have been
coded into software which has been widely used in the water and rail industries. I have
advised a number of users of those methods. The one-day course has been given to Evira (Finland),
Pfizer (UK), Novartis (Switzerland), Procter and Gamble (USA), and also as an open-entry
course at Warwick University.
Typical challenges
Expert judgement is a crucial part of decision-making under uncertainty. Eliciting the
knowledge of experts in the form of probability distributions for uncertain parameters
of a decision model is a complex task. If done badly, the resulting probability distributions
do not represent the expert's knowledge accurately, may not bear any resemblance to reality and
can be dominated by arbitrary choices that the expert might make entirely differently if asked
another way. Getting accurate, reproducible and defensible judgements requires
careful management of the elicitation by a skilled facilitator. Expertise and understanding
of group psychology are particularly important when working with several experts
(which is often needed to capture the full range of expert opinion). I can offer a
comprehensive service as an experienced facilitator in elicitations across a range of areas
including nuclear waste disposal, environmental hazards, project costing and
pharmaceutical development.
Risk assessment and management is an important field that has special challenges. Experts are
often called upon to assess risks smaller than can be informed by real data. Risk typically
depends on the occurrence of extreme events, and particularly on the concurrence of several extreme
values. From the perspective of elicitation, this poses particular difficulties in
assessing the tails of probability distributions and correlations between variables. Few
people working in elicitation have given real thought to these issues, whereas I can offer
unique methods to address them.
I am happy to consult on these or other problems in elicitation.
Updated: 24 March 2011
Maintained by: Tony O'Hagan
