Dr Stephen Jewson, RMS Ltd, London
Objective and Subjective Probabilities in Risk Assessment
Quantitative risk assessment often uses parametric models, and these models are often based on very little data, making the parameter estimates very uncertain. An example from the field of natural hazard risk assessment would be estimating the frequency distribution of intense hurricanes: there have been 71 intense hurricanes hitting the US in the last 114 years. This means that the mean number of hurricanes can only be estimated to within around 25%, and more subtle aspects of the distribution of the number of hurricanes are estimated even less well.
Bayesian statistics provides a useful set of tools for quantifying parameter uncertainty and propagating it into predictions. Part of the Bayesian methodology is to specify a prior. If there is concrete prior information, the prior can be based on that. If not, then one has to choose between an objective prior or a subjective prior, and it’s important to understand the difference, and think carefully about which is preferable given the purpose of the analysis being done. In this blog, we briefly review the issues.
Objective Priors and Objective Probabilities
Objective priors are an attempt to get as close as possible to the scientific ideal of objectivity, which is usually taken to mean that methods used to derive predictions and risk estimates should be as non-arbitrary and non-personal as possible. Perfect objectivity is unobtainable, but a key concept in science is to get as close as possible. In practice that means that modelling decisions (such as which model to use, which numerical scheme to use, what temporal or spatial resolution to use, which prior to use) should be based on clearly stated and defensible criteria. For objective priors, there are a number of different criteria that can be used. The two most common and widely accepted are (a) that results should be unchanged if we repeat the analysis after a simple transformation of parameters or variables, and (b) that probabilities should match observed frequencies as closely as possible. These two together are often enough to determine a prior. Although there is no complete theory that determines the best or unique objective prior for each situation, there are many results and methods available. If one accepts the philosophy of wanting an objective prior, the main shortcoming of objective priors is that in some cases, especially for complex or new models, determining an objective prior may become a research project in itself. Some of the methods for determining objective priors, for instance, involve group theory, solving non-linear PDEs, or both. Other shortcomings have been discussed by statisticians, but they are mostly related to philosophical issues that are of little relevance to mathematical modellers in general. For simple cases like the normal distribution, predictions based on Objective priors agree exactly with traditional predictive intervals.
Subjective Priors and Subjective Probabilities
Conversely subjective priors are used when the goal is to include subjective, or personal, opinions into a calculation. The personal opinions, in the form of a distribution, are combined with the information from the model and the data to produce what are usually known as subjective, or personal, probabilities. For many mathematical models, writing down a subjective prior can be a lot easier than trying to determine an objective prior.
Subjective probabilities also have various shortcomings, such as: (a) there is no particular reason for anyone to agree with anyone else’s subjective probabilities, and this can lead to a situation where everyone has their own view of a probability or a risk. Whether this is good or bad depends on the purpose of the analysis, but it can be difficult when it is desirable to reach agreement, (b) for subject areas where the data is already well known by everyone in the field, it is likely to be impossible to ensure that a subjective prior is truly independent from the data, as it should be for the method to be strictly valid, (c) for large dimensional problems, it is unrealistic to either have or write down a prior opinion (e.g. on thousands of correlations in a large correlation matrix), (d) some mathematical models have parameters which only have meaning in the context of the model. In these cases the idea of holding any kind of prior view does not make sense and (e) it can be very difficult to write down a distribution that really does reflect one’s beliefs: for instance, when subjective priors are transformed from one coordinate system to another, they often end up having properties that the owner of the prior didn’t intend.
An additional problem with subjective priors that arises in the catastrophe modelling industry is that some insurance regulators require risk estimates to follow the principles of scientific objectivity, as described above, as closely as possible, and will not accept risk estimates in which the modeller is free to adjust the prior and change the results without clear justification.
Priors in Catastrophe Modelling
Based on the considersations given above, we at RMS are keen to quantify parameter uncertainty in our catastrophe models using Objective Bayesian methods wherever possible. Figuring out how to do that is leading us on an interesting journey thru the statistical literature, and round the world, and is even leading to some new mathematical results. Our collaboration with Trevor Sweeting has already led to the derivation of a new result related to the question of how best to predict data from the multivariate normal (all credit to Trevor for that), and we are now working with Richard Chandler and Ken Liang (funded by NERC), on the question of how to derive objective probabilities for the hurricane prediction problem mentioned above.
A curious problem for UK science and industry is that although the concept of objective probability was originally developed here (by the geophysicist Harold Jeffreys) there now seems to be relatively little expertise in the UK in this field: the biggest and most productive groups of researchers in this area are in the US and Spain. Perhaps one objective of PURE might be to try and encourage the development of more local expertise in this field. We’d certainly be interested to make contact with anybody in the UK who has either interest or expertise in objective probabilities, and how to apply them to the myriad mathematical modelling challenges that we face in catastrophe risk modelling.