Case Study 9: Credible project Uncertainty and Robustness Estimation toolbox (CURE)

Trevor Page, Lancaster University

Paul Smith, Keith Beven, Lancaster University

Francesca Pianosi, Fanny Sarrazin, Suzanna Almeida, Liz Holcome, Jonty Rougier, Jim Freer and Thorsten Wagener, University of Bristol

Michael Hollaway, Lancaster University

DOWNLOAD

THE CHALLENGE

There is a general trend for increasing inclusion of uncertainty estimation (UE) in environmental and hazard modelling because the effective use of model results in decision making requires a level of confidence to be established.  Another requirement is the assessment of the implicit and explicit choices and assumptions made during the modelling and UE process (Rougier and Beven, 2013). In particular, assumptions made regarding the nature of epistemic uncertainties (uncertainties relating to a lack of knowledge), and how they are taken account of, should be recorded and communicated  to stakeholders such that the meaning of the results and any subsequent analysis is put into context.  Good practice in this respect is still developing and is integral to the CREDIBLE Uncertainty and Robustness Estimation Toolbox (CURE), which aims to represent best practice in applying UE methods as well as best practice in being explicit about modelling choices and assumptions.  In this way CURE will contribute to the ongoing development and testing of UE methods and good practice in their application.

WHAT WAS ACHIEVED

The CURE toolbox consists of computer program functions which quantify uncertainties associated with simulation model results for a given application. It provides a broad range of different UE methods that users can apply that are demonstrated using various environmental model applications. Example applications are provided in the form of workflow scripts (sequences of computer program commands that link to CURE program functions) that demonstrate the different UE analysis methods and ultimately help users define and structure their own workflow scripts for their particular applications.  Apart from performing core calculations associated with individual UE methods, CURE functions compute additional modelling and UE diagnostics and results including visualisations that aid in the communication of uncertainty.  The recording of modelling choices and assumptions, including treatment of epistemic uncertainties, is facilitated by a Graphical User Interface (GUI) where users can information for inclusion in a modelling audit trail log. The audit log is an important component for the communication of the meaning of the uncertainty estimates, as it sets the context for the UE.

HOW WE DID IT

CURE is a fully open source toolbox written in the MATLABTM programming language.  As it is aimed at simulation models, it employs a range of different Monte Carlo methods for forward UE (i.e. the forward propagation of uncertainties through a model: e.g. using prior estimates of input and parameter uncertainty) and conditioned UE (i.e. where UE results are conditioned on observations). The UE methods included span both formal statistical and informal approaches, underpinned by different philosophies, such that users are able to explore various approaches.  Each approach and modelling application will include its own implicit and explicit modelling choices and assumptions recorded via the GUI. The GUI takes the form of a number of simple, sequential dialogue boxes where the user is asked to enter information, as text, in a structured way and which can be iteratively edited during any modifications to analyses. The toolbox structure is such that new methods can be easily added and it will be subject to ongoing development and augmentation with additional workflow examples.

CURE1 CURE2

REFERENCES

Rougier, J and Beven, K J, 2013. Model limitations: the sources and implications of epistemic uncertainty, in Rougier J, Sparks, S and Hill, L, Risk and uncertainty assessment for natural hazards, Cambridge University Press: Cambridge, UK, 40-63

NERC, PURE, CREDIBLE (Consortium on Risk in the Environment: Diagnostics, Integration, Benchmarking, Learning and Elicitation) Project:  NE/J017450/1.