Causal inference in AI education: A primer

1.1 Forward

Before embarking on this journey in causality, it is important to contextualize this work with respect to its intended audience, target domains of application, and source experiences from which anecdotal student experiences are shared.

Intended audience. This work represents an invitation to instructors to concert topics in AI and causality, and is thus appropriate for the following readers:

1. Instructors with a background in causality who are looking to incorporate more AI/ML examples and assignments into their courses, either by extending existing lessons in AI/ML with causal topics or by incorporating AI/ML examples into courses primarily on causal inference.

2. Instructors teaching AI/ML courses who are looking for entry points/motivations to introduce causality but who may be unfamiliar with causal formalisms or procedures.

For readers of category 1, we include brief refreshers on the formal lessons of causality alongside intuitive examples that are classroom-ready supplements delete both for the foundational concepts and those marketed specifically for AI/ML applications. For readers of category 2, please note that this work is not intended as an in-depth primer for all causal topics (the textbooks referenced in the introduction are better suited for this), but instead, contains examples and problems that we hope compel the integration of causality in many avenues of traditional AI/ML.

Source experiences and target domains. Given these specifications, we include several example syllabi and suggested entry points for causal topics in traditional AI curricula within Appendix A. Of these, two are from the authors’ deployments at (a) the high-school level in a course entirely on causality and (b) at the undergraduate level in a course on causal reinforcement learning (mingling the two topics in depth). In the shared experiences in teaching these courses that are mentioned throughout this work, note that the authors enjoyed classes high in student engagement and enjoyment, but for which only anecdotal evidence is available, making no objective claims that must be instead studied empirically. With a wider adoption of causal topics in AI curricula, we invite future study to examine its potential benefits at a more robust population level.

2.1 Motivating causal inference

Causal inference is thus the umbrella label for tools used to compute queries (generally, those at ℒ 2 and ℒ 3 ) from an existing causal model. There are many different types of causal queries at these higher tiers of inference, and before formalizing any of them, students will appreciate some intuition surrounding why they are interesting from a data-scientific perspective.

Note how answers to each of the questions in Example 2.1 have implications for medical policy, e.g., it may be helpful within certain age ranges but harmful in others, it may only be helpful due to its influence on blood pressure (in which case, other prescriptions may more directly help to prevent heart disease), and so on. However, from observational data alone (i.e., outside of the laboratory randomized clinical trial), associations between these variables can complicate the answers to causal questions. SCMs provide formal characterizations and procedures for answering each of these causal questions by applying a structured explanation of the system of causes and effects, which serves as a lens through which one can view the data’s associations. Consider an example characterization of this system as diagrammed in Figure 2.

Figure 2

Possible causal graph explaining the relationship between variables in Example 2.1.

With the extra explanatory power of an SCM layered atop the data it fits, we can intuitively define several different types of causal questions based on its structure, some of the recipes for which will be formalized later.

The aforementioned represents only a sample of the many causal effects of interest for practitioners wishing to translate their data into actionable policy (e.g., see refs. [32,33,34]). However brief, this list of example causal queries serves as a simple motivation for causal inference, and for intuiting the SCMs that can be used to address them. That said, introductory lessons in causality are dominated by such examples in medicine, econometrics, and others; a major deliverable of this work is to shift the same lessons motivated by these in the empirical sciences to settings that AI scientists will find applicable.

2.2 Motivating causal discovery

Given the capabilities of SCMs to answer interesting causal queries like those in Example 2.1, students will likely be curious to learn about the sources of these models. Thus, as an oft close companion to causal inference, causal discovery techniques focus on the construction or learning of SCMs from data. Causal discovery supports the assembly of DAGs, or parts of DAGs, largely by examining independence relations among variables (potentially conditioned on other variables), to offer a mechanism to uncover their causal relationships. In this sense, data alone are sometimes enough for causal inference, but when they are not, a partial DAG (also known as a pattern or equivalence class) can inform practitioners of what else is required to disambiguate. Children instinctively comprehend this and employ playful manipulation to better grasp their environment when information from their senses is insufficient [7]; adults and scientists also perform experiments to confirm their causal hypotheses. Causal discovery with DAGs may provide a systematic way for machines to better understand causal situations beyond the traditional ML task of prediction [23].

All DAGs, regardless of complexity, can be constructed from paths of the three basic structures depicted in Figure 1. The chain in Figure 1(a) consists of X causing Z , followed by Z causing Y . The fork in Figure 1(b) consists of Z having a causal influence on both X and Y . In this case, even though X has no causal effect on Y , knowing the value of X does help predict the value of Y , quintessential correlation without causation. In both the chain and the fork, X is independent of Y if and only if conditioning on Z ( X ⊥ ⊥ Y ∣ Z ): P ( Y = y ∣ Z = z , X = x ) = P ( Y = y ∣ Z = z ) and P ( Y = y ∣ X = x ) ≠ P ( Y = y ) .^[2]

Colliders, as illustrated in Figure 1(c), behave the opposite to chains and forks in regards to independence. Specifically, X ⊥ ⊥ Y without conditioning on Z : P ( Y = y ∣ X = x ) = P ( Y = y ) . However, X and Y notably become dependent when conditioning on Z or any of its descendants: P ( Y = y ∣ Z = z , X = x ) ≠ P ( Y = y ∣ Z = z ) . By holding the common effect Z to a particular value, any change to X would be compensated by a change to Y .

Anecdotally, students have appreciated causal stories to explain these rules of dependence in a causal graph, which may also serve as mnemonics. For each of the following examples, a fruitful exercise can be to have students provide a graphical explanation for the story, which then motivates the rules of independence expected of any graphs with the same patterns.

The graphical nature of these types of exercises can engender high engagement among students compared to typical probability syntax alone. Causal intuition and probabilistic understanding in this puzzle-like context are thus concerted and enhanced. Building upon these intuitions, we can establish independence or isolate effects in more complex graphs by blocking paths from one node to another through a structural criterion called d-separation (directional separation) [35]; d-separation is already taught alongside traditional AI coverage of Bayesian Networks and succinctly stated as follows.

With causal models being core to causal inference, d-separation provides us with an important testing mechanism. Because a DAG demonstrates which variables are independent of each other given a subset of the remaining variables to condition on, probabilities can be estimated from data to confirm these conditional independencies. The fitness of a causal model can therefore be validated (to a degree of confidence), and debugging simplified from global fitness tests to d-separation’s ability to pinpoint error localities. Unfortunately, it is not possible to test every causal relationship between nodes in a DAG, meaning that causal discovery does not always yield the complete DAG, nor are these validity measures a guarantee that a recovered graph represents the true reality [21].

Still, certain structural hints provide hope of recovering causal localities. For instance, a v-structure is defined as a pair of nonadjacent nodes, such as X and Y in Figure 1(c), with a common effect ( Z in the same figure). These v-structures are often embedded throughout larger causal graphs. An example of a testable implication is to check that Z is not included in the set of nodes that render X ⊥ ⊥ Y .

A simple approach to causal discovery is to find every possible DAG compatible with a set of variables and their independence relationships in a dataset. In general, better approaches require further assumptions, but this is an active area of research [36,37,38]. The set of compatible DAGs is called an equivalence class, which, for some causal queries, can be sufficient for identifying causal effects even with partial structures. If further experimentation is necessary, an equivalence class can help target those variables on which experiments need to be performed to discover the true structure [39,40].

The inductive causation (IC) algorithm^[3] [9, p. 204] is a simple approach to causal discovery:

1. For each pair of variables a and b in V , search for a set S a b such that ( a ⊥ ⊥ b ∣ S a b ) holds in P ˆ (stable distribution of V ). Construct an undirected graph G such that vertices a and b are connected with an edge if and only if no set S a b can be found.

2. For each pair of nonadjacent variables a and b with a common neighbor c , check if c ∈ S a b . If it is not, then add arrowheads a → c ← b .

3. In the partially directed graph that results, orient as many of the undirected edges as possible subject to two conditions: (i) any alternative orientation would yield a new v -structure and (ii) any alternative orientation would yield a directed cycle.

The whole process of constructing a causal model can be challenging for students not familiar with modeling. The following example demonstrates a simple workflow in which students can engage.

Figure 3

Equivalence class of graphs constructed from probability distribution in Table 1.

This work introduces a companion causal inference learning system^[5] to help students practice and absorb concepts in causal discovery. As depicted in Figure 4, a teacher simply writes the structural functions and data generating processes of the exogenous variables, and students are presented with the resulting probability distribution and nodes of the equivalence class to connect. Causal discovery exercises such as these provide engaging exploration into the etiology of data generation missing in many statistically focused curricula.

Figure 4

Causal discovery exercise editor.

2.3 Assumptions

This background on chains, forks, colliders, and d -separation affords us basic building blocks for powerful causal inference tools. As important caveats to be discussed with students, the power of a causal model depends on having a correct representation of the system. There are some criteria for assessing whether a model is a fair representation of the underlying data or data generating process. However, some assumptions are sometimes necessary to be asserted. The first that we have assumed is infinite data, leaving the statistical analysis of quantifying uncertainty with finite samples to be dealt with separately. The second is a property known as stability/faithfulness: we assume independencies remain invariant when P ( U ) changes. This means that the conditional independencies in the underlying probability distribution are reflected in the DAG. As a basic violation of this, imagine a child who only eats vegetables ( Y ) if their parents convince them ( X ). The child’s parents are always trying to convince them, P ( X = 1 ) = 1 . Therefore, we might declare X independent of Y , since P ( Y = y ∣ X = x ) = P ( Y = y ) . However, that equality only holds under parameterizations of P ( U ) in which P ( X = 1 ) = 1 . If the parents sometimes relent, then P ( Y = y ∣ X = x ) ≠ P ( Y = y ) and X and Y would be dependent. Stated formally:

The remainder of this work focuses on the potential of causal inference to both elucidate traditional topics in AI and ML and to inspire new avenues for students to explore. Using the preliminaries outlined in this section, students will be equipped to understand the challenges and opportunities at each tier of the PCH.

2.4 Instructor reflections

Intuiting the motivations for causal inference is a challenge to instill in students who have dealt little with real data and the many complex questions that data may or may not be equipped to answer alone. Leading any introductory causal lesson with the intuitions presented in Example 2.1 and Definition 2.3 can spark the important questions that motivated the PCH; questions as simple as “Does obesity shorten life, or is it the soda?” [33] are enough to elicit lively classroom discussion just to introduce the distinctions of types of causal effects and how these are difficult to disentangle from mere associations without the aid of a model.

From these observations, anecdotally, students to treat exercises involving the design and interpretation of compounded conditional independence graphs as puzzles rather than monotonous calculations that may lack translation to purpose. This has elicited classroom enjoyment, which feeds into participatory graph modifications in the spirit of causal discovery. The discussions and debates that ensue develops intuition through active engagement, which can be especially important at the high-school level for engendering intuition before formalism.

For assignments, instructors may find it useful to generate mock datasets (like that in Example 2.5) to help students to understand crucial lessons in causal discovery, d-separation, and challenges like observational equivalence and unobserved confounding. Various software packages exist for this endeavor, though Tetrad and Causal Fusion have been popular choices that students can pick up without large amounts of tutorial.^[6] It is likewise important to instill that causal discovery is a difficult exercise that is far from a magic-wand to be waved over a dataset to produce a trustworthy model; as an ongoing field of inquiry, even in ideal situations, extracting the causal graph can be difficult and many times rests on extra-data sources of information to implement properly. Given these challenges, there are a myriad of reasons and scenarios in which to pursue their solution, many of which we highlight in the coming sections.

3.1 Simpson’s paradox

Table 2 shows that P ( Y = 1 ∣ X = 1 ) = 0.81 > P ( Y = 1 ∣ X = 0 ) = 0.75 , which may lead AdBot to conclude that Ad 1 is always more effective. However, the same data also show within age-specific strata that P ( Y = 1 ∣ X = 1 , Z = 0 ) = 0.85 < P ( Y = 1 ∣ X = 0 , Z = 0 ) = 0.9 and P ( Y = 1 ∣ X = 1 , Z = 1 ) = 0.65 < P ( Y = 1 ∣ X = 0 , Z = 1 ) = 0.7 , indicating that Ad 0 is better. AdBot thus faces a dilemma: if the age of a viewer is not known, which ad is the best choice? This conflict is known as Simpson’s paradox, which long haunted practitioners using only ℒ 1 tools without causal considerations. Its solution, and those to many other problems, can be found in the next tier.

3.2 Linear regression

Linear regression is a common topic in introductory statistics and ML courses. This is due, in part, to linear regression’s interpretability, limited overfitting, and simplicity. As shown later, a linear model’s coefficients explain the impact each variable has on the outcome. This provides intuition behind how causal structure affects learned parameters. Linear regression also provides a base from which to launch more complex ML models and algorithms, and topics like parameters, degrees of freedom, and nonlinearity can be added incrementally. The simplicity of linear regression makes for an ideal starting point for introducing causality to ML. Although this simplicity will seldom yield highly predictive algorithms with real-world data, linear regression can clearly illustrate the value of causal constructs through coding exercises. Student discussion can be fostered through debate about linearity assumptions among exercises and examples.

Other work has provided examples for inferring causal effects from associational multivariate linear regression [41], but which we adapt herein as useful exercises for ML students to start examining problems from different tiers of the causal hierarchy. A first exercise corresponds to the chain DAG shown in Figure 1(a).

The next step is to train an ML model that lacks nonlinear activation functions. The weights of the model can then be analyzed:

The weight on X has a negligible^[8] impact on the result. This also makes intuitive sense as the model was trained on both X and Z , while Y only “listens to” Z (i.e., since Y is a function of Z , f y ( z , u y ) ). Looking only at the weights, it would seem that training intensity is irrelevant to athletic performance. If an analyst wanted to predict the performance of someone with increased training intensity, using this model they would observe no difference in performance. On the other hand, if the model had been trained only on X :

Here, X clearly plays a major role in predicting performance. This time, making a prediction using this model with increased training intensity will yield increased athletic performance.

Which feature vector do we use for our ML model? This decision is not clear because predicting athletic performance when changing only training intensity is an intervention. Thus, this is a causal question requiring tools from ℒ 2 covered in the following section.

The DAG of Figure 5(a) corresponds to this scenario. Similar to Example 3.2, an ML model can be trained on features X and Z or just on X . First, a feature vector consisting of both X and Z produces the following weights and bias:

Figure 5

Potential models explaining Simpson’s paradox. (a) Observed confounder Z between X and Y . (b) M-graph with unobserved confounders U 1 and U 2 between X , Z and Z , Y , respectively.

The weight on X indicates a positive impact on the outcome. Predicting the level of competitiveness of someone with increased preparation time would yield an increased level of competitiveness. This makes sense as the example data were generated, where Y was calculated with a positive multiple of X (1 to be precise). Next, a singleton feature vector of X produces the following weights and bias:

This time, the weight on X is negative, indicating a negative impact on the outcome. It would seem that increasing preparation in this model decreases competitiveness.

These two models have very different weights on X . Which model is correct? The answer depends on the quantity of interest. A causal question, such as, “What is the effect of preparation on competitiveness?” requires an analysis in ℒ 2 .

Since Z listens to both X and Y , the associated collider DAG is in Figure 1(c). An ML model trained with a feature vector consisting of both X and Z produces the following weights and bias:

The weight on X indicates an inverse relationship between athletic performance and negotiating skill. Are better athletes worse negotiators? Using a singleton feature vector of X paints a different picture:

This time, the weight on X is negligible. We know, from the code that generated the example data, that negotiating skill and athletic performance are uncorrelated. So, this appears to be a better model for understanding the causal effect of negotiating skill on athletic performance (a null causal effect). In addition to the two previous examples, this is another example where ℒ 2 tools are necessary to know which variables to include in the feature vector.

Examining the weights is helpful to foster an intuition for why feature selection is critical in understanding causal relationships and queries. As students investigate more expressive, nonlinear, models (for which libraries like PyTorch provide a number of tools), weights become less interpretable despite what may be an increase in accuracy. Still, these causal intuitions to feature selection, their relationship to SCMs, and how they may bias queries remain.

3.3 Instructor reflections

Within the courses in which these causal concepts have been tested, students have exhibited surprise when first exposed to Simpson’s paradox. This revelation is their first hint that the story behind the data is crucial for thorough and valid interpretations of the results. This is a prime opportunity for active learning. By using DAGs as a discussion source [42], students review and debate both the diagrams and the need to be careful about which features to train their ML models on and how to utilize their results.

For many students, learning the mathematics of probability and statistics may feel mechanical, thus missing the forest (the ability to use these as tools to inform decisions, automated or otherwise) for the trees (the rote computation) [43,44]. Examples like those introduced in 3.1–3.4 break the mold of this script and ask students to make a defendable choice with the data and assumptions at-hand because such acts are causal questions often unanswerable by the data alone.

The causal “solutions” to these problems have intuitive, graphical criteria that students tend to find more appealing than reasoning over the symbolic or numerical parameters of each system alone. What follows is an overview of these approaches that can both enhance student understanding of traditional tools in ℒ 1 , and understanding their limits: both when and how to seek solutions to questions at higher tiers of the causal hierarchy.

4.1 Resolving Simpson’s paradox

As such, practitioners are often left with causal questions but only observational data, like in Example 3.1. Herein, we witness an instance of Simpson’s paradox, when a better outcome is predicted for one treatment versus another, but the reverse is true when calculating treatment effects for each subgroup.

Resolving Simpson’s paradox demands that we understand the underlying data-generating causal system, which in general may cause confusion through only the associational lens. Examining Figure 5, these two observationally equivalent causal models of the data in Example 3.1 tell two different interventional stories. In (a), Z is a confounder whose influence in the observational data must be controlled to isolate the causal effect of X → Y . In (b), Z is only spuriously correlated with { X , Y } , and so controlling for Z in this setting will actually enable confounding bias (by the rules of d -separation, since U 1 → Z ← U 2 forms a collider). Practically, this means that if (a) is our explanation of the observed data, then AdBot should consult the age-specific clickthrough rates and display Ad 0; if (b) is our explanation, then we consult the aggregate data and display Ad 1. In this specific scenario, model (a) is the more defendable since there cannot be latent confounders that affect someone’s age as in model (b).

Generalizing the intuitions earlier, the foundational tool from the interventional tier is known as do-calculus [47], which allows analysts to take both observational data and a causal model, and answer interventional queries.

To compare quantities at associational ( ℒ 1 ) and interventional ( ℒ 2 ) tiers, the probability of event Y happening given that variable X was observed to be x is denoted by P ( Y ∣ X = x ) . The probability of event Y happening given that variable X was intervened upon and made to be x is denoted by P ( Y ∣ do ( X = x ) ) . For instance, in Figure 5(a), the effect of intervention d o ( X = x ) would be to sever the edge Z → X .

Formally, to compute the ACE (Def. 2.3) of an ad on clickthroughs in Example 3.1, and assuming our setting conforms to the model in Figure 5(a), we must compute X ’s influence on Y in homogeneous conditions of Z , weighted by the likelihood of each condition Z = z . This adjustment is accomplished through the graphical recipe specified by the Backdoor criterion:

By employing the backdoor criterion, we control for the spurious correlative pathway X ← Z → Y to isolate the desired causal pathway X → Y in estimation of P ( Y ∣ d o ( X ) ) . Numerically applied to the AdBot Example 3.1 (with backdoor admissible covariate Z = { 0 , 1 } ), and assuming the model in Figure 5(a):

From this adjustment, we confirm that displaying Ad X = 0 has the highest ACE on clickthrough rates. In summary, we arrive at this conclusion through the following steps, which are beneficial to highlight for students applying this recipe in general:

1. Example 3.1 demanded that we compute an ACE of ad choice X on clickthrough rates Y in cases that the viewer’s age Z is unknown; this is an ℒ 2 query of the format P ( Y = 1 ∣ d o ( X ) ) whose computation can suffer from Simpson’s Paradox given that the inclusion or exclusion of Z as a control delivers different answers of the optimal ad choice.

2. To resolve this “paradox” and compute the ACE requires assumptions about the causal structure to determine which, if any, spurious pathways demanded control. We encoded these assumptions in the SCM with graphical structure from Figure 5(a).

3. Given this structure, we applied the backdoor adjustment criteria to find P ( Y = 1 ∣ d o ( X = x ) ) ∀ X = x controlling for backdoor admissible variable Z and concluded that the highest likelihood action X = 0 was the best for maximizing clickthroughs.

4.2 Causal recipes for feature selection

The power of do-calculus means ML algorithms can utilize causal effects without having to perform experiments or be trained on experimental data.^[9] This has implications for ML feature selection: bias may be introduced if the causal structure is not consulted. For example, a collider might be conditioned on without conditioning on noncolliders along the path from action X to outcome Y . Consider the M-graph of Figure 5(b): variables U 1 and U 2 cannot be included in the feature vector of an ML model because they are unobserved, and if Z is included in the feature vector, this model will produce correlative ( ℒ 1 ), but not causal ( ℒ 2 + ), predictions.

Notably, if the requested query is indeed correlative, the criteria for feature selection are different than if it were causal, and the addition of features that provide information about the outcome can aid in accuracy without causal considerations. However, queries at tiers above the first must be careful with controlled covariates lest they inadvertently bias the outcome. Reflecting on the pharmacology Example 2.1, we can conceive of queries at different tiers:

1. What is the incidence of heart disease among those who take aspirin?

2. What is the ACE of aspirin on incidence of heart disease?

In the ℒ 2 query, and assuming the causal graph in Figure 2, we would intuitively wish to include Z (age) as a feature to block the backdoor path X ← Z → Y , and avoid including M (blood pressure) as a feature lest we intercept part of aspirin’s effect on heart disease mediated through blood pressure. These intuitions are formalized in the backdoor criterion.

Concretely, revisiting the three linear regression examples of Section 3.2, Example 3.2 poses a decision to use a feature vector consisting of ⟨ X , Z ⟩ or just ⟨ X ⟩ . Since the data generating process makes Z a function of X , and Y a function of Z , the DAG of Figure 1(a) corresponds to this model. The DAG makes it clear that by including Z in the feature vector, we are conditioning a mediator, thus blocking X ’s influence on Y and preventing the correct calculation of the causal effect of X on Y . This can be seen from the fact that Y ⊥ ⊥ X ∣ Z ; therefore, E ( Y ∣ d o ( X ) , Z ) = E ( Y ∣ Z ) .

Since there are no backdoor paths from X to Y , the causal effect can be predicted by not including Z in the feature vector. Students are then left to debate the linearity assumption. Does every additional level of training intensity, within a reasonable range, yield the same increase in athletic performance? This application of ℒ 2 tools to get the causal effect of interest by including only X in the feature vector does not depend on linearity. So, the linearity discussions can aid intuition and lead to the generalization of dropping the linearity assumption.

Example 3.3 showcases the same feature vector decision, ⟨ X , Z ⟩ or ⟨ X ⟩ . This time the corresponding DAG is Figure 5(a), which was used to explain Simpson’s paradox. The backdoor path X ← Z → Y must be blocked to have a model that predicts the causal effect of X , preparation, on Y , competitiveness. Blocking this backdoor between X and Y is accomplished by including Z in the feature vector.

Example 3.4 is a collider scenario depicted in the DAG of Figure 1(c). Here, attention must be paid to including the collider Z in the feature vector. By including Z , predictions will be far more accurate (in fact, excluding Z will make predictions simply the mean of Y ). However, doing so opens a spurious pathway between X and Y , making the causal effect of X on Y naïvely appear to be nonzero, but the DAG makes it clear that the causal effect should be null. Therefore, we must exclude Z from the feature vector if the ML model is to determine the causal effect of X , athletic performance, on Y , negotiating skill.

Students can extend the insights gained from the above (which are useful in eliciting insights distinguishing ℒ 1 and ℒ 2 in simple settings) in more complex models like the following that demands a synthesis of these modular lessons.^[10]

Figure 6

Feature selection playground on a causal diagram with treatment X , outcome Y , and other covariates.

In Figure 6, conventional wisdom allows for the inclusion of all covariates Z = { R , T , W , V } to maximize precise prediction of Y for the ℒ 1 quantity P ( Y ∣ X , R , W , T , V ) , but the causal quantity requires more selectivity. To control for all noncausal pathways requires that Z = { R } alone, because (1) controlling for T opens the backdoor path from X ↔ T ↔ Y , (2) controlling for W blocks the chain from X → W → Y , and (3) controlling for V opens a spurious pathway at collider X → V ← Y . Thus, Z = { R } serves as a backdoor admissible set to allow for estimation of P ( Y ∣ d o ( X ) ) via adjustment as in Example 3.1.

4.3 Transportability and data fusion

Although much of traditional ML education focuses on the ability or suitability of models to fit a particular dataset, there are several adjacent discussions that are commonly omitted, including the qualitative differences between observational ( ℒ 1 ) and experimental ( ℒ 2 ) data, how these datasets can often be “fused” to support certain inference tasks, and how to take data collected at some tier in one environment/population and transport it to another. This transportability problem [49,50] has long been studied in the empirical sciences under the heading of external validity [51,52] and has received attention from the AI and ML communities under a variety of related tasks like transfer learning [53,54] and model generalization [55,56,57]. Many modern techniques have focused on the ability to take a model trained in one environment and then to adapt it to a new setting that may differ in key respects. This capability is particularly palatable to fields that train agents in simulation settings to be later deployed in the real world, often because it is too risky, expensive, or otherwise impractical to perform the bulk of training in reality [58,59,60]. In general, when the training domain differs from the deployment domain (even slightly), predictions are biased, sometimes with significant model degradation. This often occurs when data from the deployment environment is limited, otherwise the ML model could have been trained on deployment data. To illustrate the utility of causal tools for this task, we provide a simple example in the domain of recommender systems that motivates distinctions in environments with heterogeneous data.

The training and deployment causal diagrams of Example 4.2 are depicted in Figure 7. Notably, because we conducted an experiment (i.e., performed an intervention) in environment π (Figure 7(b), represented by the interventional subgraph G x ), the intervention d o ( X ) severs any of the would-be inbound edges to X in the observational setting that we see in the target environment π ∗ (Figure 7(b), representing the unintervened graph G ). Graphically, the challenge in the target environment becomes clear: we wish to estimate P ∗ ( Y = 1 ∣ d o ( X = x ) ) , but this causal effect is not identifiable because it is impossible to control for all backdoor paths between X and Y due to the presence of unobserved confounders indicated by the bidirected arcs. Yet, by assumption, the only difference between the two environments is the difference in age distributions such that P ( Z ) ≠ P ∗ ( Z ) , so insights from the experiment conducted in π (in which the direct effect of X → Y has been isolated) may yet transport into π ∗ . To encode these assumptions of where structural differences occur between environments and thus to determine if and how to transport, we can make use of another graphical tool known as a selection diagram.

Figure 7

Causal and selection diagrams for data collected in different environments but with same causal graph G . (a) Target/deployment environment π ∗ , causal graph G , eliciting P ∗ ( X , Y , Z ) . (b) Source/training environment π , sub model G x , eliciting P ( Z , Y ∣ d o ( X ) ) . (c) Selection diagram D constructed from shared graph G .

Importantly, selection diagrams encode both the differences in causal mechanisms between environments (via the presence of a selection node) and the similarities, with the assumption that any absence of a selection node represents the same local causal mechanisms between environments at that variable. In Example 4.2, the selection diagram requires only a single addition to G (Figure 7(a)): a selection node S representing the difference in age distributions at Z . Notationally, this also allows us to represent distributions in terms of the S variable, such that S = s ∗ indicates that the population under consideration is the target π ∗ . Similarly, we can re-write distributions that are sensitive to selection like P ∗ ( Z ) = P ( Z ∣ S = s ∗ ) and our target query from Example 4.2, P ∗ ( Y = 1 ∣ d o ( X ) ) = P ( Y = 1 ∣ d o ( X ) , S = s ∗ ) . Doing so provides us a starting point for adjustment, similar to the backdoor adjustment from Example 3.1, wherein (using the rules of do-calculus) if we are able to find a sequence of rules to transform the target causal effect into an expression where the d o -operator is independent from the selection variables, transportability is possible [62]. In the present DietBot Example, the goal is thus to phrase P ( Y = 1 ∣ d o ( X ) , S = s ∗ ) in terms of our available data, P ( Z , Y ∣ d o ( X ) ) and P ∗ ( X , Y , Z ) . Such a derivation is as follows:

Equation (1) follows from the law of total probability, (2) from the product rule, (3) from d-separation (because Y ⊥ ⊥ S ∣ Z , d o ( X = x ) ), (4) from do-calculus (because, examining G x , Z ⊥ ⊥ d o ( X = x ) ),^[11] and (5) is simply a notational equivalence for the distribution of Z belonging to π ∗ .

While many theoretical lessons may end at the derivation of the transport formula concluding in equation (5), including the numerical walkthrough using the parameters of Table 3 serves as an effective dramatization for why transportability has important implications for heterogeneous data and policy formation. Consider the scenario wherein the agent designer did not perform a transport adjustment between source and target domains, using only the model that would have been fit during training. In this risky setting, the agent would maximize the source environment’s P ( Y = 1 ∣ d o ( X = x ) ) :

As mentioned earlier, P ( Y = 1 ∣ d o ( X = 0 ) ) > P ( Y = 1 ∣ d o ( X = 1 ) ) , meaning that the optimal choice in the training environment is X = 0 . However, by properly applying the transport formula, we find that the opposite is true in the deployment environment:

The DietBot example provides a host of important lessons at ℒ 2 of the causal hierarchy, juxtaposing different causal inferences that would be obtained in different environments, demonstrating the utility of graphical models and do-calculus, and the dangers of unobserved confounding. Although these theoretical premises are typically taught in the study of causality in the empirical sciences, its practical utility in AI and ML can be driven home by casting transportability in terms of “training and deployment” environments, and by showing the surprise of opposite inferences that would be drawn with and without adjustment. As learning data scientists, students also obtain insights into the risks and opportunities of heterogeneous data, and how their fusion can overcome an otherwise difficult task of training and deployment environment differences. Plainly, in practice, adjustment formulae would not necessarily be computed by hand like in the above, but the experience of the demonstration is valuable for students; a fuller treatment of automated tools used in transportability can be found in refs. [10,50,62].

4.4 Instructor reflections

Within previous offerings of these lessons, the student’s surprise experienced in associational exercises and questions of Section 3 continues with the interventional exercises for feature selection, transportability, and data fusion. More than just discussions arising from the revelations ℒ 2 brings, high-school students have shown a keen interest in immediately using ℒ 2 tools to explain everyday experiences and then learning how to encode those using a formal vocabulary.

Instructors of introductory courses in AI have expressed frustrations discussing probabilistic models like Bayesian networks as ad hoc or supporting topics that lack an impactful conclusion. However, examining these graphical models through the causal lens yields a fruitful experience for students to move beyond the probability calculus and the mantra that “correlation does not equal causation.” Though this mantra is indeed true in general, there is a lesson to be learned in its dual: causation does bestow some structure to observed correlations, and this structure can be harnessed in support of many tasks that lead beyond the data alone.

By using the intuitions of d-separation as the structure of independence relationships in Bayesian networks, this strict graphical explanation of the data serves as an effective stepping stone into causal Bayesian networks and SCMs; by completing this transition, instructors can more fully develop students’ understanding of how probability leads to policy. This insight is clearly illustrated by the use of graphical models in which observations and interventions can disagree (as in Example 3.1, P ( Y ∣ X ) ≠ P ( Y ∣ d o ( X ) ) ), how the environments and circumstances of data collection powerfully matter (as in Example 4.2, argmax x P ( Y = 1 ∣ d o ( X = x ) ) ≠ argmax x P ∗ ( Y = 1 ∣ d o ( X = x ) ) ), and in causal discovery exercises for which an equivalence class of observationally equivalent models may explain some dataset, only some of which may follow a defendable causal explanation.

Along this path, students may struggle to understand the notion of latent variables and unobserved confounding unless the following are explained in unison: (1) the graphical depiction provides a causal explanation for where latent, outside influences may be present, and (2) how these influences outside of the model yield differences in causal ℒ 2 and noncausal ℒ 1 inferences that the data can provide.

5.1 Structural counterfactuals

Despite the expressive and creative potential of counterfactuals, the common student’s initial exposure to them risks being overly formal and notationally heavy, often beginning with the following definition:

Ostensibly, a counterfactual appears similar to the definition of an intervention. However, although the do-operator expresses a population-level intervention across all possible situations u ∈ U ∀ u , a counterfactual computes an intervention for a particular unit/individual/situation U = u . This new syntax allows us to write queries of the format P ( Y X = x = y ∣ X = x ′ ) , which computes the likelihood that the query Y attains value y in the world where X = x (the hypothetical antecedent), given that X = x ′ was observed in reality. The clash between the observed evidence X = x ′ and hypothetical antecedent X = x motivates the need for the new subscript syntax and demonstrates how the previous tiers of the hierarchy cannot express such a query.

These expressions are often a source of syntactic and semantic confusion for beginners; an anecdotally better strategy is to instead begin with a discrete, largely deterministic, simple motivating example with a plain-English counterfactual query, and then to work backward to the formalisms.

Figure 8

SCM M and its associated graph G pertaining to Example 5.1.

To address this counterfactual query, intuitions best begin with the causal graph, whose observational state is depicted in Figure 8. Second, it is instructive to show how the previous layers’ notations break down with the query of interest, as we cannot make sense of the contrasting evidence and hypothesis using the do-operator alone (i.e., the expression P ( R ∣ d o ( X = 0 ) , X = 1 ) is syntactically invalid, having set X to two separate values in the same world). Instead, the query of interest focuses upon the recovery state in the world, where X = 0 , though in reality (a separate world state), X = 1 was observed. This can be expressed via the counterfactual query P ( R X = 0 ∣ X = 1 ) , which can be teased in the lesson either before or after the computational mechanics that follow.

Before performing this computational, it is useful for students to visualize its steps. Intuitively, we expect that some information about our observed evidence X = 1 may change our beliefs about the counterfactual query R X = 0 ; this information thus flows between the observed ( ℒ 1 , associational) and hypothetical ( ℒ 2 , interventional) worlds through the only source of variance in the system: the exogenous variables (in Example 5.1, A ). Depicting this bridge can be accomplished through a technique known as the twin network model [63].

The twin network of the SCM in Example 5.1 is depicted in Figure 9. This model is not only an intuitive depiction of the means of computing the query at-hand but also serves the practical purposes of being a model through which standard evidence propagation techniques can be used to update beliefs from evidence to antecedent, and through which the standard rules of d-separation can be used to determine independence relations between variables in counterfactual queries. It is also useful to examine some axioms of counterfactual notation at this point, noting the equivalence of certain ℒ 3 expressions with previous tiers, like P ( R X = x ) = P ( R ∣ d o ( X = x ) ) (the “potential outcomes” subscripted format for writing the ℒ 2 intervention) and P ( R X = x ∣ X = x ) = P ( R ∣ X = x ) (the consistency axiom in which antecedent and observed evidence are the same, making it an observational quantity from ℒ 1 ).

Figure 9

Twin network M ∗ for the SCM in Example 5.1 and counterfactual query P ( R X = 0 ∣ X = 1 ) .

Returning to our example, the actual computation of P ( R X = 0 ∣ X = 1 ) follows a three-step process motivated by the twin-network representation.

Step 1: Abduction. The abduction step updates beliefs about the shared exogenous variable distributions based on observed evidence, meaning we effectively replace P ( u ) ← P ( u ∣ e ) . In the current, largely deterministic example, this amounts to propagating the evidence that X = 1 through the rest of M ∈ M ∗ , which can be trivially shown to indicate that all variables attain a value of 1 with certainty. However, for the abduction step, we need only update beliefs about the exogenous variable, A :

Step 2: Action. With P ( u ) ← P ( u ∣ e ) (viz., P ( A = 1 ∣ X = 1 ) = 1 ), we can effectively discard/ignore the observational model M and shift to the hypothetical twin M x , forcing X = x per the counterfactual antecedent, which in our example, means severing all inbound edges to X in M x and forcing its value to X = 0 . Let M ∗ be the modified model following steps 1 and 2.

Step 3: Prediction. Finally, we perform standard belief propagation within the modified M ∗ to solve for our query variable, R x , and find that the patient would indeed still have recovered (i.e., P ( R X = 0 = 1 ) = 1 ) because MediBot would still have also administered the other effective treatment, W x = 1 .

This simple example not only demonstrates the mechanics and potential of structural counterfactuals, but also serves as a launchpad for more intricate and challenging applications. Worthwhile follow-on exercises include the addition of noisy exogenous variables to the system in Example 5.1 (e.g., nondeterministic patient recovery), and analogies to linear SCMs in which the three-step process is repeated through application of conditional expectation. Moreover, the example leads into questions of necessity and sufficiency [64] of the medical treatments, which can segue into other, more applied and data-driven, counterfactuals.

5.2 Counterfactuals for metacognitive agents

In some more adventurous explorations in AI oriented at crafting self-improving and reflective artificial agents, counterfactuals in ℒ 3 may prove to be a useful tool for metacognitive agents [65,66]. Related to the transportability problem with DietBot in Example 4.2, agents may find the need to evolve their policies learned earlier in their lifespan or in environments that change over time to optimize their performance. This need complements a growing area of reinforcement learning that incorporates causal concepts, especially with respect to meta-learning [13,67,68]. To demonstrate such a scenario, we reconsider MediBot in a setting wherein its current policy’s decisions are confounded, damaging its performance and requiring it to perform some measure of metacognition to improve that is analogous to the human experience of regret.

The data in Table 4 demonstrates the tell-tale sign of unobserved confounding wherein the observed and experimental treatment effects differ ( P ( Y ∣ x ) ≠ P ( Y ∣ d o ( x ) ) ∃ x ∈ X ), implicating an uncontrolled latent factor that explains the difference. Surprisingly, despite the unknown identity of the confounder, a better treatment policy than MediBot’s current one does indeed exist in this context, and is derived from a counterfactual quantity known as the effect of treatment on the treated (ETT) [70]. The ETT traditionally computes the difference between the effect of an alternate treatment X = x than the one actually given to an individual X = x ′ , the counterfactual component of which can be expressed in this context as P ( Y X = x = 1 ∣ X = x ′ ) , x ≠ x ′ .

With only the partially specified model, and the observational and experimental recovery rates, it is possible to compute the ETT for binary treatments (assuming, in this setting, that the patient requested treatments are observed in equal proportion, P ( X = 0 ) = P ( X = 1 ) = 0.5 ), as in the following derivation that is true for any treatment X = x and its alternative X = x ′ .

This algebraic trick (using only the law of total probability) allows us to derive ℒ 3 quantities of interest from a combination of ℒ 1 and ℒ 2 data (though only for binary treatment) and tells an important tale about the system: MediBot is presently in a state of inevitable regret [71] in which the likelihood of recovery for those given treatment under its policy ( P ( Y = 1 ∣ X = x ′ ) = 0.5 ∀ x ′ ∈ X ) is 40% less than had those same patients been treated differently ( P ( Y X = x = 1 ∣ X = x ′ ) = 0.9 ∀ x ≠ x ′ ∈ X ). The “inevitable” part of this regret is also instructive for distinguishing ℒ 1 (what happens in reality/nature) from ℒ 3 (what could have happened differently) quantities because it seems that no matter what decision MediBot makes in reality, there is always a better one that it could have made instead!

Ostensibly, this computation yields only bleak retrospect, but also leads to a surprising remedy for online agents. Two insights contribute to the solution, known as intent-specific decision-making [13]: (1) the formation of the confounded agent’s observational/naturally decided action (i.e., its intent) can be separated from the ultimately chosen one, and (2) this intended action choice serves as a back door admissible proxy for the state of the UC (see Figure 10(b) with agent intent I ).

Figure 10

SCM associated with Example 5.2 with treatment X , recovery Y , UC U , and intent I . (a) Observational model. (b) The same system, but with intent explicitly modeled.

When intent is explicitly modeled in a confounded decision-making scenario (Figure 10(b)), the ETT (previously, a retrospective ℒ 3 quantity) can be measured empirically before a decision is made by using do-calculus conversions to a ℒ 2 quantity through a process known as intent-specific decision-making.