Sankararaman, S., & Mahadevan, S. (2011). Model validation under epistemic uncertainty. Reliability Engineering & System Safety, 96(9), 1232-1241.
One of the cited issues with using Bayesian analysis for validation is given by Roy & Oberkampf (2011) is that it does not deal with epistemic uncertainty in the inputs. This paper, and the work of Sankararaman in general, seeks to address this issue. Note, however, that only poorly understood stochastic data is addressed, not poorly understood deterministic data, as we would find in certain inputs to wave models (like bathymetry, perhaps).
There is some disagreement about how epistemic uncertainty can be handled and I think this quote illuminates some of these issues and some solutions:
“of sparse and/or imprecise data. Some researchers argue against a probabilistic approach to handle interval data because information may be added to the problem. However, if the quantity is stochastic to begin with, it is only appropriate to represent it with a probability distribution. Regarding faithfulness to the available information, the proposed method addresses this concern either (1) by quantifying the uncertainty in the parameters of the probability distribution and using a family of distributions in the parametric approach or (2) by constructing non-parametric probability distributions that are more flexible than parametric distributions and more faithful to the available information.”
These “mixed” uncertain inputs are dealt with in two different ways in this paper. Either, they form families of parametric distributions to describe them and then integrate them together or they form a single non-parametric representation of the uncertainty. I don’t want to go into a great deal of detail due to time constraints but a Guassian Process appears in the non-paremetric case.
Note that, in this case, an emulator does not appear to be used. The paper also deals with uncertain model data in the form of interval data. This has not been considered in Bayesian validation before. It does this by using an error interval from the minimum and maximum errors.
An interesting admission in the conclusions:
“This integration is computationally efficient and meaningful for model validation by integrating the contribution of different types/sources of uncertainty into a single validation metric, which is helpful for decision-making purposes. However, it may not be suitable for sensitivity analysis where the quantification of individual contributions of aleatory and epistemic uncertainties to the model output uncertainty is of interest.”
Basically, there is this issue of differing focus again, from whether you want to determine where the uncertainty originates or simply whether the model is OK for use.