Assessing climate model projections: State of the art and philosophical reflections

Abstract

The present paper draws on climate science and the philosophy of science in order to evaluate climate-model-based approaches to assessing climate projections. We analyze the difficulties that arise in such assessment and outline criteria of adequacy for approaches to it. In addition, we offer a critical overview of the approaches used in the IPCC working group one fourth report, including the confidence building, Bayesian and likelihood approaches. Finally, we consider approaches that do not feature in the IPCC reports, including three approaches drawn from the philosophy of science. We find that all available approaches face substantial challenges, with IPCC approaches having as a primary source of difficulty their goal of providing probabilistic assessments.

Introduction

The climate system is the system of processes that underlie the behavior of atmospheric, oceanic and cryospheric phenomena such as atmospheric temperature, precipitation, sea-ice extent and ocean salinity. Climate models are designed to simulate the seasonal and longer term behavior of the climate system. They are mathematical, computer implemented representations that comprise two kinds of elements. They comprise basic physical theory—e.g., conservation principles such as conservation of momentum and heat—that is used explicitly to describe the evolution of some physical quantities—e.g., temperature, wind velocity and properties of water vapor. Climate models also comprise parameterizations. Parameterizations are substitutes for explicit representations of physical processes, substitutes that are used where lack of knowledge and/or limitations in computational resources make explicit representation impossible. Individual cloud formation, for example, typically occurs on a scale that is much smaller than global climate model (GCM) resolution and thus cannot be explicitly resolved. Instead, parameterizations capturing assumed relationships between model grid-average quantities and cloud properties are used.

The basic theory of a climate model can be formulated using equations for the time derivatives of the model's state vector variables, x_i, i=1,…,n, as is schematically represented by

$$\frac{\partial x_i}{\partial t}=F_i(x_1\dots x_n,y_1,\dots ,y_n,t)+G_i(t)$$

In Eq. (1), t denotes time, the G_i represent external forcing factors and how these function together to change the state vector quantities, and the F_i represent the many physical, chemical and biological factors in the climate system and how these function together to change the state vector quantities. External forcing factors—e.g., greenhouse gas concentrations, solar irradiance strength, anthropogenic aerosol concentrations and volcanic aerosol optical depth—are factors that might affect the climate system but that are, or are treated as being, external to this system.

The x_i represent those quantities the evolution of which is explicitly described by basic theory, that is the evolution of which is captured by partial time derivatives. The y_i represent quantities that are not explicitly described by basic theory. So these variables must be treated as functions of the x_i, i.e., the y_i must be parameterized. In this case, the parameterizations are schematically represented in Eq. (2).

$$y_i=H_i(x_1,\dots ,x_n)$$

Given initial conditions x_i(t₀) at time t=t₀ and boundary conditions, the climate model calculates values of the state vector at a later time t=t₁ in accordance with Eq. (1).

Climate models play an essential role in identifying the causes of climate change (Fig. 1) and in generating projections. Projections are conditional predictions of climatic quantities. Each projection tells us how one or more such quantities would evolve were external forcing to be at certain levels in the future. Some approaches to assessing projections derive projections, and assess their quality, at least partly independently of climate models. They might, for example, use observations to decide how to extend simulations of present climate into the future (Stott et al., 2006) or derive projections from, and assess them on the basis of, observations (Bentley, 2010, Siddall et al., 2010). We focus on climate-model-based assessment. Such assessment is of the projections of one or more climate models and is assessment in which how good models are in some respect or another is used to determine projection quality. A climate model projection (CMP) quality is a qualitative or quantitative measure, such as a probability, that is indicative of what we should suppose about CMP accuracy.

It is well recognized within the climate science community that climate-model-based assessment of projection quality needs to take into account the effects of climate model limitations on projection accuracy (Randall et al., (2007), Smith, 2006, Stainforth et al., 2007). Following Smith (2006) and Stainforth, Allen et al. (2007), we distinguish between the following main types of climate model limitations:

• External forcing inaccuracy—inaccuracy in a model's representation of external forcing, that is in the G_i in Eq. (1).

• Initial condition inaccuracy—inaccuracy in the data used to initialize climate model simulations, that is in the x_i(t₀).

• Model imperfection—limitations in a model's representation of the climate system or in our knowledge of how to construct this representation, including:

  • Model parameterization limitations—limitations in our knowledge of what the optimal or the appropriate parameter values and parameterization schemes for a model are. This amounts, in the special case where parameterizations are captured by Eq. (2), to limitations in our knowledge of which functions H_i one should include from among available alternatives.

  • Structural inadequacy—inaccuracy in how a model represents the climate system which cannot be compensated for by resetting model parameters or replacing model parameterizations with other available parameterization schemes. Structural inaccuracy in Eq. (1) is manifested in an insufficient number of variables x_i and y_i as well as in the need for new functions of these variables.

Parameterization limitations are illustrated by the enduring uncertainty about climate sensitivity and associated model parameters and parameterization schemes. A relatively recent review of climate sensitivity estimates underscores the limited ability to determine its upper bound as well as the persistent difficulty in narrowing its likely range beyond 2–4.5 °C (Knutti & Hegerl, 2008). The 21 GCMs used by Working Group One of the IPCC fourth report (WG1 AR4) illustrate structural inadequacy. These sophisticated models are the models of the World Climate Research Programme's Coupled Model Intercomparison Project phase 3 (CMIP3) (Meehl, Covey et al., 2007). Some important sub-grid and larger than grid phenomena that are relevant to the evolution of the climate system are not accurately represented by these models, some are only represented by a few of the models and some are not represented at all. Parameterization of cloud formation, for example, is such that even the best available parameterizations suffer from substantial limitations (Randall, Khairoutdinov, Arakawa, & Grabowski, 2003). None of the models represent the carbon cycle, only some represent the indirect aerosol effect and only two represent stratospheric chemistry (CMIP3, 2007). The models also omit many of the important effects of land use change (Mahmood et al., 2010, Pielke, 2005). Many of their limitations, e.g., the limited ability to represent surface heat fluxes as well as sea ice distribution and seasonal changes, are the result of a combination of structural inadequacy and parameterization limitations (Randall et al., 2007, p. 616). CMIP3 simulations illustrate initial condition inaccuracy. Due to constraints of computational power and to limited observations, these simulations start from selected points of control integrations rather than from actual observations of historical climate (Hurrell et al., 2009).

The most ambitious assessments of projection quality, and these are primarily climate-model-based assessments, are those of WG1. The first three WG1 reports rely primarily on the climate-model-based approach that we will call the confidence building approach. This is an informal approach that aims to establish confidence in models, and thereby in their projections, by appealing to models' physical basis and success at representing observed and past climate. In the first two reports, however, no uniform view about what confidence in models teaches about CMP quality is adopted (IPCC, 1990, IPCC, (1996)). The summary for policymakers in the WG1 contribution to the IPCC first assessment report, for example, qualifies projections using diverse phrases such as ‘we predict that’, ‘confidence is low that’ and ‘it is likely that’ (IPCC, 1990). A more systematic view is found in WG1's contribution to the third IPCC assessment report (WG1 TAR). It made use of a guidance note to authors which recommends that main results be qualified by degrees of confidence that are calibrated to probability ranges (Moss & Schneider, 2000). The summary for policymakers provided by WG1 TAR does assign projections such degrees of confidence. It expresses degrees of confidence as degrees of likelihood and takes, e.g., ‘very likely’ to mean having a chance between 90% and 99%, and ‘likely’ to mean having a chance between 66% and 90%. The chapter on projections of future climate change, however, defines degrees of confidence in terms of agreement between models. A very likely projection, for example, is defined (roughly) as one that is physically plausible and is agreed upon by all models used (IPCC, 2001).

WG1 AR4's assessment of projection quality has two stages. First, confidence in models is established as in previous reports. This is mostly achieved in Chapter 8—which describes, among other things, successful simulations of natural variability (Randall et al., 2007)—and in chapter 9—which focuses on identifying the causes of climate change, but also characterizes model successes at simulating 20th century climate change (Hegerl et al., 2007). The second stage is carried out in Chapter 10—which provides WG1 AR4's global projections (Meehl, Stocker et al., 2007)—and Chapter 11—which focuses on regional projections (Christensen et al., 2007). In these chapters, expert judgment is used to assign qualities to projections given established confidence in models and the results of formal, probabilistic projection assessment (Meehl, Stocker et al., 2007). WG1 AR4 is the first WG1 report that makes extensive use of formal assessment, though it recognizes that such approaches are in their infancy (Christensen et al., (2007), Randall et al., (2007)). Both climate-model-based and partly climate-model-independent formal approaches are used.

Although WG1 AR4 assesses models using degrees of confidence, it does not assess projections in these terms. Nor does it equate projection likelihoods with degrees of agreement among models. It does, however, implement the advice to provide probabilistically calibrated likelihoods of projections (IPCC, 2005). For example, unlike WG1 TAR, WG1 AR4 provides explicit likelihood estimates for projected ranges of global mean surface temperature (GMST) changes. It estimates that the increase in GMST by the end of the century is likely to fall within −40% to +60% of the average GCM warming simulated for each emission scenario and provides broader uncertainty margins than the GCM ensemble in particular because GCMs do not capture uncertainty in the carbon cycle (Fig. 2).

The sophistication of WG1 AR4's assessments was enabled by the increasing ability to use multi-GCM and perturbed physics GCM ensembles. Thus, while WG1's first two reports relied on simple climate models to produce long term GMST projections, WG1 TAR and WG1 AR4 relied primarily on state-of-the-art GCM ensembles to assess these and other projections. WG1 AR4 nevertheless still relied on simpler models, including intermediate complexity and energy balance models (Randall et al., 2007).

In this paper, we provide a critical discussion of the (climate-model-based) approaches to assessing projection quality relied on in WG1 AR4 and more recent work by climate scientists. In doing so, we build on the substantial climate science literature, including WG1 AR4 itself. We, however, extend this literature using the perspective of the philosophy of science. Our discussion does focus more than climate scientists themselves tend to on precisely why assessing projection quality is difficult, on what is required of an adequate approach to such assessment and on the limitations of existing approaches. We, nevertheless, also address some of the practical concerns of climate scientists. We outline three views of how to assess scientific claims that are drawn from the philosophy of science and consider how they might further assist in assessing projection quality. Important issues that space does not allow us to address are the special difficulties that assessment of regional projection quality raises. An issue that deserves more attention than we have given it is that of how uncertainty about data complicates assessing projection quality.

We begin (Section 2) by considering what kinds of qualities should be assigned to projections, especially whether probabilistic qualities should be assigned. We then (Section 3) discuss why assessing projection quality is difficult and outline criteria for adequate approaches to doing so. Using these criteria, we proceed to discuss (4 The confidence building approach, 5 The subjective Bayesian approach, 6 The likelihood approach, 7 Putting it all together) the approaches that were used in WG1 AR4, namely the confidence building, the subjective Bayesian and the likelihood approaches. Finally (Section 8), we discuss approaches that are not used, or are not prominent in, WG1 AR4, including the possibilist and three philosophy-of-science-based approaches.