Skip to main content
SpringerLink
Log in

Validation Using Bayesian Methods

  • Chapter
  • First Online:
Computer Simulation Validation

Part of the book series: Simulation Foundations, Methods and Applications ((SFMA))

  • 2754 Accesses

Abstract

Quantitative model validation is playing an increasingly important role in performance and reliability assessment of a complicated system whenever simulation is involved. This chapter discusses model validation from a Bayesian perspective, considering in particular data uncertainty. First, Bayes’ theorem is defined, then the Bayesian risk rule method is introduced. Explicit expressions for the Bayesian interval hypothesis testing approach are presented in both univariate and multivariate cases. The problem of non-normal validation data is addressed by the Box–Cox transformation . A generalized procedure is presented to implement Bayesian validation methods. Classic hypothesis testing method is utilized to conduct a comparison study. The impact of the data normality assumption and of the variation of the threshold on model assessment accuracy is investigated by using both classical and Bayesian approaches. The Bayesian methodology is illustrated with a reliability model of rotor blades, a univariate stochastic damage accumulation model, and a multivariate heat conduction problem.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 189.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 249.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • AIAA. (1998). Guide for the Verification and Validation of Computational Fluid Dynamics Simulations, American Institute of Aeronautics and Astronautics, AIAA-G-077-1998, Reston, Virginia, USA.

    Google Scholar 

  • Andrews, D. F., Gnanadesikan, R., & Warner, J. L. (1971). Transformations of multivariate data. Biometrika, 27(4), 825–840.

    Article  Google Scholar 

  • Annis, C. (2002). Modeling high cycle fatigue with Markov chain Monte Carlo-a new look at an old idea. In: Proceedings of 43rd AIAA/ASME/ASCE/AHS/ASC Structure, Structural Dynamics and Materials Conference (AIAA-2002-1380) (pp. 1351–1361). Denver CO 2002.

    Google Scholar 

  • ASME. (2002). Verification & Validation (V&V) methodology and quantitative reliability at confidence: Basis for an investment strategy, PTC 60, UCRL-ID-150874, Livermore, CA: Lawrence Livermore National Laboratory.

    Google Scholar 

  • ASME. (2006). Guide for verification and validation in computational solid mechanics. ASME V&V 10-2006. New York, NY: American Society of Mechanical Engineers.

    Google Scholar 

  • Babuska, I., & Oden, J. T. (2004). Verification and validation in computational engineering and science: Basic concepts. Computer Methods in Applied Mechanics and Engineering, 193(36–38), 4057–4066.

    Article  MathSciNet  MATH  Google Scholar 

  • Balci, O., & Sargent, R. G. (1981). A methodology for cost-risk analysis in the statistical validation of simulation models. Communications of Association for Computing Machinery (ACM), 24(11), 190–197.

    Article  MathSciNet  Google Scholar 

  • Berger, J. O., & Delampady, M. (1987). Testing precise hypotheses. Statistical Science, 2(3), 317–352.

    Article  MathSciNet  MATH  Google Scholar 

  • Box, G. E. P., & Cox, D. R. (1964). An analysis of transformations. Journal of the Royal Statistical Society, 26(2), 211–252.

    MathSciNet  MATH  Google Scholar 

  • Chen, W., Baghdasaryan, L., Buranathiti, T., & Cao, J. (2004). Model validation via uncertainty propagation and data transformations. AIAA Journal, 42(7), 1406–1415.

    Article  Google Scholar 

  • Chen, W., Xiong, Y., Tsui, K. -L., & Wang, S. (2008). A design-driven validation approach using Bayesian prediction models. Journal of Mechanical Design, 130(2), 021101-1-12.

    Google Scholar 

  • DOD. (1996). Verification, validation, and accreditation (VV&A) recommended practices guide. Alexandria, VA: Department of Defense.

    Google Scholar 

  • DOE. (2000). Accelerated strategic computing initiative (ASCI) program plan. DOE/DP-99-000010592, Washington, DC: Department of Energy.

    Google Scholar 

  • Dowding, K. J., Pilch, M., & Hills, R. G. (2008). Formulation of the thermal problem. Computer Methods in Applied Mechanics and Engineering (special issue), 197(29–32), 2385–2389.

    Article  MATH  Google Scholar 

  • Genz, A. (1992). Numerical computation of multivariate normal probabilities. Journal of Computational and Graphical Statistics, 1(2), 141–149.

    Google Scholar 

  • Hills, R.G., & Trucano, T. G. (2002). Statistical validation of engineering and scientific models: A maximum likelihood based metric. Technical Report Sand. No 2001-1783, Albuquerque, NM: Sandia National Laboratories.

    Google Scholar 

  • Hwang, J. T., Casella, G., Robert, C., Wells, M. T., & Farrell, R. H. (1992). Estimation of accuracy in testing. The Annals of Statistics, 20(1), 490–509.

    Article  MathSciNet  MATH  Google Scholar 

  • Jeffreys, H. (1961). Theory of probability (3rd ed.). London, UK: Oxford University Press.

    MATH  Google Scholar 

  • Jensen, F. V., & Jensen, F. B. (2001). Bayesian networks and decision graphs. New York: Springer.

    Book  MATH  Google Scholar 

  • Jiang, X., & Mahadevan, S. (2007). Bayesian risk-based decision method for model validation under uncertainty. Reliability Engineering and System Safety, 92(6), 707–718.

    Article  Google Scholar 

  • Jiang, X., & Mahadevan, S. (2008a). Bayesian wavelet method for multivariate model assessment of dynamic systems. Journal of Sound and Vibration, 312(4–5), 694–712.

    Article  Google Scholar 

  • Jiang, X., & Mahadevan, S. (2008b). Bayesian validation assessment of multivariate computational models. Journal of Applied Statistics, 35(1), 49–65.

    Article  MathSciNet  MATH  Google Scholar 

  • Jiang, X., & Mahadevan, S. (2009a). Bayesian inference method for model validation and confidence extrapolation. Journal of Applied Statistics, 36(6), 659–677.

    Article  MathSciNet  MATH  Google Scholar 

  • Jiang, X., & Mahadevan, S. (2009b). Bayesian structural equation modelling method for hierarchical model validation. Reliability Engineering and System Safety, 94(4), 796–809.

    Article  Google Scholar 

  • Jiang, X., & Mahadevan, S. (2010). Bayesian nonlinear SEM approach for hierarchical validation of dynamical systems. Mechanical Systems and Signal Processing, 24(4), 957–975.

    Article  Google Scholar 

  • Jiang, X., Yuan, Y., Mahadevan, S., & Liu, X. (2013a). An Investigation of Bayesian inference approach to model validation with non-normal data. Journal of Statistical Computation and Simulation, 83(10), 1829–1851.

    Article  MathSciNet  Google Scholar 

  • Jiang, X., Yuan, Y., & Liu, X. (2013b). Bayesian inference method for stochastic damage accumulation modeling. Reliability Engineering and System Safety, 111(3), 126–138.

    Article  Google Scholar 

  • Kass, R., & Raftery, A. (1995). Bayes factors. Journal of the American Statistical Association, 90(430), 773–795.

    Article  MathSciNet  MATH  Google Scholar 

  • Kennedy, M. C., & O’Hagan, A. (2001). Bayesian calibration of computer experiments. Journal of the Royal Statistical Society Series B (Statistical Methodology), 63(3), 425–464.

    Article  MathSciNet  MATH  Google Scholar 

  • Mahadevan, S., & Rebba, R. (2005). Validation of reliability computational models using Bayes networks. Reliability Engineering and System Safety, 87(2), 223–232.

    Article  Google Scholar 

  • Migon H. S., & Gamerman, D. (1999). Statistical inference-an integrated approach. London: Arnold, a Member of the Holder Headline Group.

    Google Scholar 

  • Nataf, A. (1962). Détermination des distributions de probalités dont les marges sont données. Comptes Rendus de l’Académie des Sciences, 225, 42–43.

    MATH  Google Scholar 

  • NIST/SEMATECH (2005). e-Handbook of statistical methods, National Institute of Standards and Technology. Retrieved from http://www.itl.nist.gov/div898/handbook.

  • Nowak, R., & Scott, C. (2004). The Bayes risk criterion in hypothesis testing. Connexions, Retrieved from http://cnx.rice.edu/content/m11533/1.6/.

  • Oberkampf, W. L., & Barone, M. F. (2006). Measures of agreement between computation and experiment: Validation metrics. Journal of Computational Physics, 217(1), 5–36.

    Article  MATH  Google Scholar 

  • Oberkampf, W. L., & Trucano, T. G. (2002). Verification and validation in computational fluid dynamics. Progress in Aerospace Sciences, 38(3), 209–272.

    Article  Google Scholar 

  • Pericchi, L. R. (1981). A Bayesian approach to transformations to normality. Biometrika, 68(1), 35–43.

    Article  MathSciNet  MATH  Google Scholar 

  • Rebba, R., & Mahadevan, S. (2006a). Model predictive capability assessment under uncertainty. AIAA Journal, 44(10), 2376–2384.

    Article  Google Scholar 

  • Rebba, R., & Mahadevan, S. (2006b). Validation of models with multivariate outputs. Reliability Engineering and System Safety, 91(8), 861–871.

    Article  Google Scholar 

  • Rebba, R., & Mahadevan, S. (2008). Computational methods for model reliability assessment. Reliability Engineering and System Safety, 93(8), 1197–1207.

    Article  Google Scholar 

  • Roache, P. J. (1998). Verification and validation in computational science and engineering in: Science and engineering. Albuquerque, NM: Hermosa Publishers.

    Google Scholar 

  • Rosenblatt, M. (1952). Remarks on multivariate transformation. Annals of Mathematical Statistics, 23(3), 470–472.

    Article  MathSciNet  MATH  Google Scholar 

  • Schlesinger, S. (1979). Terminology for model credibility. Simulation, 32(3), 103–104.

    Article  Google Scholar 

  • Schervish, M. J. (1995). Theory of statistics. New York: Springer.

    Book  MATH  Google Scholar 

  • Schwer, L. E. (2007). Validation metrics for response histories: Perspectives and case studies. Engineering with Computers, 23(4), 295–309.

    Article  Google Scholar 

  • Stephens, M. A. (1974). EDF statistics for goodness of fit and some comparisons. Journal of the American Statistical Association, 69(347), 730–737.

    Article  Google Scholar 

  • Zhang, R., & Mahadevan, S. (2003). Bayesian methodology for reliability model acceptance. Reliability Engineering and System Safety, 80(1), 95–103.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Tongji University, Shanghai, 200092, China

    Xiaomo Jiang & Yong Yuan

  2. Clayton State University, Morrow, GA, 30620, USA

    Xueyu Cheng

Authors
  1. Xiaomo Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xueyu Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yong Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaomo Jiang .

Editor information

Editors and Affiliations

  1. University of Bern, Bern, Switzerland

    Claus Beisbart

  2. Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany

    Nicole J. Saam

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Jiang, X., Cheng, X., Yuan, Y. (2019). Validation Using Bayesian Methods. In: Beisbart, C., Saam, N. (eds) Computer Simulation Validation. Simulation Foundations, Methods and Applications. Springer, Cham. https://doi.org/10.1007/978-3-319-70766-2_20

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-70766-2_20

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-70765-5

  • Online ISBN: 978-3-319-70766-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation