The Design and Analysis of Computer Experiments

Preface to the Second Edition

Preface to the First Edition

Contents

1 Physical Experiments and Computer Experiments

1.1 Introduction

1.2 Examples of Computer Simulator Models

1.3 Some Common Types of Computer Experiments

1.3.1 Homogeneous-Input Simulators

1.3.2 Mixed-Input Simulators

1.3.3 Multiple Outputs

1.4 Organization of the Remainder of the Book

2 Stochastic Process Models for Describing Computer Simulator Output

2.1 Introduction

2.2 Gaussian Process Models for Real-Valued Output

2.2.1 Introduction

2.2.2 Some Correlation Functions for GP Models

2.2.3 Using the Correlation Function to Specify a GP with Given Smoothness Properties

2.3 Increasing the Flexibility of the GP Model

2.3.1 Hierarchical GP Models

2.3.2 Other Nonstationary Models

2.4 Models for Output Having Mixed Qualitative and Quantitative Inputs

2.5 Models for Multivariate and Functional Simulator Output

2.5.1 Introduction

2.5.2 Modeling Multiple Outputs

2.5.3 Other Constructive Models

2.5.4 Models for Simulators Having Functional Output

2.6 Chapter Notes

3 Empirical Best Linear Unbiased Prediction of Computer Simulator Output

3.1 Introduction

3.2 BLUP and Minimum MSPE Predictors

3.2.1 Best Linear Unbiased Predictors

3.2.2 Best MSPE Predictors

3.2.3 Some Properties of y"0362y(xte)

3.3 Empirical Best Linear Unbiased Prediction of Univariate Simulator Output

3.3.1 Introduction

3.3.2 Maximum Likelihood EBLUPs

3.3.3 Restricted Maximum Likelihood EBLUPs

3.3.4 Cross-Validation EBLUPs

3.3.5 Posterior Mode EBLUPs

3.3.6 Examples

3.4 A Simulation Comparison of EBLUPs

3.4.1 Introduction

3.4.2 A Selective Review of Previous Studies

3.4.3 A Complementary Simulation Study of Prediction Accuracy and Prediction Interval Accuracy

3.4.3.1 Performance Measures

3.4.3.2 Function Test Beds

3.4.3.3 Prediction Simulations

3.4.4 Recommendations

3.5 EBLUP Prediction of Multivariate Simulator Output

3.5.1 Optimal Predictors for Multiple Outputs

3.5.2 Examples

3.6 Chapter Notes

3.6.1 Proof That (3.2.7) Is a BLUP

3.6.2 Derivation of Formula 3.2.8

3.6.3 Implementation Issues

3.6.4 Software for Computing EBLUPs

3.6.5 Alternatives to Kriging Metamodels and Other Topics

3.6.5.1 Alternatives to Kriging Metamodels

3.6.5.2 Testing the Covariance Structure

4 Bayesian Inference for Simulator Output

4.1 Introduction

4.2 Inference for Conjugate Bayesian Models

4.2.1 Posterior Inference for Model (4.1.1) When = ?

4.2.1.1 Posterior Inference About ?

4.2.1.2 Predictive Inference at a Single Test Input xte

4.2.2 Posterior Inference for Model (4.1.1) When = (?,?Z )

4.3 Inference for Non-conjugate Bayesian Models

4.3.1 The Hierarchical Bayesian Model and Posterior

4.3.2 Predicting Failure Depths of Sheet Metal Pockets

4.4 Chapter Notes

4.4.1 Outline of the Proofs of Theorems 4.1 and 4.2

4.4.2 Eliciting Priors for Bayesian Regression

4.4.3 Alternative Sampling Algorithms

4.4.4 Software for Computing Bayesian Predictions

5 Space-Filling Designs for Computer Experiments

5.1 Introduction

5.1.1 Some Basic Principles of Experimental Design

5.1.2 Design Strategies for Computer Experiments

5.2 Designs Based on Methods for Selecting Random Samples

5.2.1 Designs Generated by Elementary Methods for Selecting Samples

5.2.2 Designs Generated by Latin Hypercube Sampling

5.2.3 Some Properties of Sampling-Based Designs

5.3 Latin Hypercube Designs with Additional Properties

5.3.1 Latin Hypercube Designs Whose Projections Are Space-Filling

5.3.2 Cascading, Nested, and Sliced LatinHypercube Designs

5.3.3 Orthogonal Latin Hypercube Designs

5.3.4 Symmetric Latin Hypercube Designs

5.4 Designs Based on Measures of Distance

5.5 Distance-Based Designs for Non-rectangular Regions

5.6 Other Space-Filling Designs

5.6.1 Designs Obtained from Quasi-Random Sequences

5.6.2 Uniform Designs

5.7 Chapter Notes

5.7.1 Proof That TL is Unbiased and of the Second Part of Theorem 5.1

5.7.2 The Use of LHDs in a Regression Setting

5.7.3 Other Space-Filling Designs

5.7.4 Software for Constructing Space-Filling Designs

5.7.5 Online Catalogs of Designs

6 Some Criterion-Based Experimental Designs

6.1 Introduction

6.2 Designs Based on Entropy and Mean Squared Prediction Error Criterion

6.2.1 Maximum Entropy Designs

6.2.2 Mean Squared Prediction Error Designs

6.3 Designs Based on Optimization Criteria

6.3.1 Introduction

6.3.2 Heuristic Global Approximation

6.3.3 Mockus Criteria Optimization

6.3.4 Expected Improvement Algorithms for Optimization

6.3.4.1 Schonlau and Jones Expected Improvement Algorithm

6.3.4.2 Picheny Expected Quantile Improvement Algorithm

6.3.4.3 Williams Environmental Variable Mean Optimization

6.3.5 Constrained Global Optimization

6.3.6 Pareto Optimization

6.3.6.1 Basic Pareto Optimization Algorithm

6.4 Other Improvement Criterion-Based Designs

6.4.1 Introduction

6.4.2 Contour Estimation

6.4.3 Percentile Estimation

6.4.3.1 Approach 1: A Confidence Interval-Based Criterion

6.4.3.2 Approach 2: A Hypothesis Testing-Based Criterion

6.4.4 Global Fit

6.5 Chapter Notes

6.5.1 The Hypervolume Indicator for Approximations to Pareto Fronts

6.5.2 Other MSPE-Based Optimal Designs

6.5.3 Software for Constructing Criterion-Based Designs

7 Sensitivity Analysis and Variable Screening

7.1 Introduction

7.2 Classical Approaches to Sensitivity Analysis

7.2.1 Sensitivity Analysis Based on Scatterplots and Correlations

7.2.2 Sensitivity Analysis Based on Regression Modeling

7.3 Sensitivity Analysis Based on Elementary Effects

7.4 Global Sensitivity Analysis

7.4.1 Main Effect and Joint Effect Functions

7.4.2 A Functional ANOVA Decomposition

7.4.3 Global Sensitivity Indices

7.5 Estimating Effect Plots and Global Sensitivity Indices

7.5.1 Estimating Effect Plots

7.5.2 Estimating Global Sensitivity Indices

7.6 Variable Selection

7.7 Chapter Notes

7.7.1 Designing Computer Experiments for SensitivityAnalysis

7.7.2 Orthogonality of Sobol´ Terms

7.7.3 Weight Functions g(x) with NonindependentComponents

7.7.4 Designs for Estimating Elementary Effects

7.7.5 Variable Selection

7.7.6 Global Sensitivity Indices for Functional Output

7.7.7 Software

8 Calibration

8.1 Introduction

8.2 The Kennedy and O'Hagan Calibration Model

8.2.1 Introduction

8.2.2 The KOH Model

8.2.2.1 Alternative Views of Calibration Parameters

8.3 Calibration with Univariate Data

8.3.1 Bayesian Inference for the Calibration Parameter ?

8.3.2 Bayesian Inference for the Mean Response ?(x) of the Physical System

8.3.3 Bayesian Inference for the Bias ?(x) and Calibrated Simulator E[ Ys(x,?) | Y ]

8.4 Calibration with Functional Data

8.4.1 The Simulation Data

8.4.2 The Experimental Data

8.4.3 Joint Statistical Models and Log Likelihood Functions

8.4.3.1 Joint Statistical Model That Allows Simulator Discrepancy

8.4.3.2 Joint Statistical Model Assuming No Simulator Discrepancy

8.5 Bayesian Analysis

8.5.1 Prior and Posterior Distributions

8.5.2 Prediction

8.5.2.1 Emulation of the Simulation Output Using Only Simulator Data

8.5.2.2 Emulation of the Calibrated Simulator Output Modeling the Simulator Bias

8.5.2.3 Emulation of the Calibrated Simulation Output Assuming No Simulator Bias

8.6 Chapter Notes

8.6.1 Special Cases of Functional Emulation and Prediction

8.6.2 Some Other Perspectives on Emulation and Calibration

8.6.3 Software for Calibration and Validation

A List of Notation

A.1 Abbreviations

A.2 Symbols

B Mathematical Facts

B.1 The Multivariate Normal Distribution

B.2 The Gamma Distribution

B.3 The Beta Distribution

B.4 The Non-central Student t Distribution

B.5 Some Results from Matrix Algebra

C An Overview of Selected Optimization Algorithms

C.1 Newton/Quasi-Newton Algorithms

C.2 Direct Search Algorithms

C.2.1 Nelder–Mead Simplex Algorithm

C.2.2 Generalized Pattern Search and Surrogate Management Framework Algorithms

C.2.3 DIRECT Algorithm

C.3 Genetic/Evolutionary Algorithms

C.3.1 Simulated Annealing

C.3.2 Particle Swarm Optimization

D An Introduction to Markov Chain Monte Carlo Algorithms

E A Primer on Constructing Quasi-Monte Carlo Sequences

References

Author Index

Subject Index