Solution concepts for games with ambiguous payoffs

Abstract

I consider games with ambiguous payoffs played by non-Expected Utility decision makers. Three equilibrium solutions are studied. Nash equilibrium in which equilibrium mixed strategies must be best responses, Crawford equilibrium in beliefs and pure equilibrium in beliefs in which equilibrium strategies are mixtures of best responses, with the latter restricting best responses to pure actions. I study the interactions between ambiguity preferences on one side and equilibrium properties on the other. I show how the equilibrium concepts differ, computing necessary and sufficient conditions for existence and equivalence. I also show how these solution concepts fare against two fundamental principles of Nash equilibrium in standard games: the principle of indifference and the reduction principle. Given both are central to the computation of Nash equilibrium in games with Expected Utility players, their failure indicates how relaxing the Expected Utility hypothesis may disrupt standard game theoretic results such as the characterization of equilibria in two-player games.

Appendices

Appendix: Proofs

1.1 Proposition 1

Crawford (1990) provides a proof for existence of CEB in a different framework. Kajii and Ui (2005) provides a proof for the existence of PEB with MaxMin Expected utility maximizers. Azrieli and Teper (2011) proves the existence of NE if (AA). These existence results are all standard applications of Kakutani’s fixed point theorem.

For all $i \in N$, and for any $\sigma _{-i} \in \prod _{j \ne i} \Delta (A_j)$, let ${ BR}_i(\sigma _{-i})$ be the set of best responses in pure strategy:

$$\begin{aligned} { BR}_i(\sigma _{-i}) = \underset{a_i \in A_i}{argmax}\ V_i(f_i^{(a_i,\sigma _{-i})}) \end{aligned}$$

Given $A_i$ is finite, ${ BR}_i(\sigma _{-i})$ is non-empty and closed. Now consider the sets of PEB-admissible strategies $B_i (\sigma _{-i}) = \Delta ({ BR}_i(\sigma _{-i})$. It is non-empty, convex and compact. Let $S = \Pi _{i \in N} \Delta (A_i)$ be the set of strategy profiles. Denote $\varphi$ the correspondence from S to $2^S$ which associates $\prod _{i \in N} B_i(\sigma _{-i})$ to all $\sigma$.

S is a non-empty, convex and compact subset of a Euclidean space ${\mathbb {R}}^n$ (where $n = \prod _{i \in N} card(A_i))$. $\varphi (\sigma )$ is non-empty and convex for any value $\sigma$ in S. Finally, $\varphi$ has a closed graph (as all possible $B_i(\sigma _{-i})$ are closed) and is upper hemi-continuous (from the continuity of $V_i$ in $\sigma _{-i}$). Thus, by the Kakutani fixed point theorem, $\varphi$ admits a fixed point $\sigma ^*$, which is a PEB by definition.

To prove existence for CEB, the exact same arguments are made but with ${ BR}_i(\sigma _{-i}) ={argmax}_{\sigma _i \in \Delta (A_i)}\ V_i(f_i^{(\sigma _i,\sigma _{-i})})$ which is closed and non-empty as $\Delta (A_i)$ is closed, non-empty and $V_i$ is continuous. Taking the simplex of these sets ensures convexity regardless of the quasi-concavity of $J_i$. For NE, given the fixed point must apply to ${ BR}_i$ directly, quasi-concavity of the $V_i$’s is needed to ensure its convexity.

Azrieli and Teper (2011) proves there exists games without NE if (AA) is not satisfied constructively. I only give an outline of the proof here. Given non-quasi concavity of at least one $V_i$, there exists $u, u' \in {\mathbb {R}}^{{\mathcal {S}}}$ such that $\forall \lambda \in ]0,1[, V_i(\lambda u + (1-\lambda ) u') < V_i(u) = V_i(u)$. Let $v = \delta u + (1-\delta ) u'$ for $\delta$ small enough. One can then construct a game without equilibrium. Let the ambiguity seeking player i have two actions $a_1, b_1$ and another player j have two actions $b_1$ and $b_2$. All other players have only one action so that one can disregard them. Payoffs are given by $x_j(a_k,b_k,.) = 1_{{\mathcal {S}}}$ and $0_{{\mathcal {S}}}$ otherwise for player j. Payoffs of player i are given by $x_j(a_1,b_1,.) = u_i^{-1}(v), x_j(a_2,b_1,.) = u_i^{-1}(u), x_j(a_1,b_2,.) = u_i^{-1}(u')$ and $x_j(a_2,b_2,.) = u_i^{-1}(v')$. This is similar to a matching pennies game where player j wants to play the same action as i whereas player i wants to mismatch their actions. Additionally, one can choose $\delta$ small enough so that no mixed strategy is a best response of i. The proof that such a $\delta$ exists is involved and present in Azrieli and Teper (2011) so that I do not restate it here. $\square$

1.2 Proposition 2

Given (AA), the set of best responses is convex and as such is the same as the set of mixtures of best responses. As a consequence NE-admissible strategies and CEB-admissible strategies are the same so that NE and CEB are equivalent. If (AA) is not satisfied, then there exists a game for which NE does not exist. Given there exists CEB for this game, the two concepts are not equivalent if (AA) is dropped.

An immediate consequence of strict quasi-convexity is that if a mixed action is a best response, then the pure actions in its support must also be best responses. Thus,

$$\begin{aligned} \underset{a_i \in A_i}{argmax}\ V_i(f_i^{(a_i,\sigma _{-i})}) \subseteq \underset{\sigma _i \in \Delta (A_i)}{argmax}\ V_i(f_i^{(\sigma _i,\sigma _{-i})}) \end{aligned}$$

and

$$\begin{aligned} \underset{\sigma _i \in \Delta (A_i)}{argmax}\ V_i(f_i^{(\sigma _i,\sigma _{-i})}) \subseteq \Delta \left( \underset{a_i \in A_i}{argmax}\ V_i(f_i^{(a_i,\sigma _{-i})})\right) \end{aligned}$$

Taking mixture over these sets on both sides of the inclusion yields the fact that PEB-admissible strategies are included in the set of CEB-admissible strategies (first inclusion) and the reverse (second inclusion). As a result both sets of admissible strategies are the same and the two solutions are equivalent.

That CEB and PEB are not equivalent if (AS) is dropped is a direct consequence of the failure of the principle of indifference for CEB. Indeed, the principle of indifference always holds for PEB from its definition. Furthermore if (AS) is not satisfied, then there exists $i, \alpha \in ]0,1[, u$ and $u'$ such that $V_i(\alpha u + (1-\alpha ) u') = V_i(u) > V_i(u')$. Let a game in which player i gets u for a given action $a_1$ and $u'$ for another action $a_2$, irrespective of the other players, then the mixed strategy $((a_1,\alpha );(a_2,1-\alpha ))$ is a CEB/NE so that the principle of indifference does not hold for CEB/NE when (AS) is dropped. $\square$

1.3 Proposition 3

The principle of indifference always holds for PEB from its definition. Furthermore if (AS) is not satisfied, then there exists $i, \alpha \in ]0,1[, u$ and $u'$ such that $V_i(\alpha u + (1-\alpha ) u') = V_i(u) > V_i(u')$. Let a game in which player i gets u for a given action $a_1$ and $u'$ for another action $a_2$, irrespective of the other players, then the mixed strategy $((a_1,\alpha );(a_2,1-\alpha ))$ is a CEB/NE so that the principle of indifference does not hold for CEB/NE when (AS) is dropped.

Given equivalence of PEB and CEB when (AS) and the fact that the principle holds for PEB, the principle holds for CEB when (AS). And again for NE given any NE is also a CEB. $\square$

1.4 Proposition 4

That all equilibrium notions satisfy the reduction approach for expected utility players is a known result. Whenever one player’s functional is not linear, then one can construct a game in which NE of the reduced game is distinct from that of the original game. From non-linearity, there exists a player i, a value $\alpha \in ]0,1[$ and two values u and $u'$ such that $V_i(\alpha u + (1-\alpha ) u') \ne \alpha V_i(u) + (1-\alpha ) V_i(u')$. This implies that $V_i(\beta u + (1-\beta ') u') = \alpha V_i(u) + (1-\alpha ) V_i(u') \rightarrow \beta \ne \alpha$. Let another player j have single-state “matching pennies” payoffs with two possible actions. Let player i have two actions, a sure action which yields $\alpha V_i(u) + (1-\alpha ) V_i(u')$ and an uncertain action which yields u or $u'$ depending on the action chosen by j. Then $[(\alpha ,1-\alpha );(1/2,1/2)]$ is an equilibrium of the reduced game whereas $[(\beta ,1-\beta );(1/2,1/2)]$ is a PEB of the original game. Thus, (EU) is necessary for the reduction principle to apply to PEB. Given failure of the principle of indifference for NE/CEB when not (AS), the reduction principle cannot apply either. Furthermore, given equivalence of CEB and PEB and the non-existence of NE when (AS) imply it does not apply either when (AS) is assumed (to the exclusion of (EU)).

Note the constructive example in the proof does not yield a very interesting game. One such example is presented in Fig. 12. Thus the game presented in Sect. 3, though it is the failure of the reduction principle which yields the interesting behavior, it is not directly linked to the proof. $\square$

Examples

1.1 $\alpha$-MaxMin expected utility and ambiguity aversion

In the examples used throughout in this paper, I have only considered the two extreme cases of $\alpha$-MaxMin expected utility where either $\alpha = 0$ or $\alpha =1$. In this section, I discuss how the results would apply for the less extreme cases where $\alpha \in ]0,1[$. I will assume for the purpose of clarity that the set of beliefs of the $\alpha$-MaxMin player is the full set of belief $\Delta ({\mathcal {S}})$

Proposition 5

Given full belief set and $card({\mathcal {S}}) = 2, \alpha$-MaxMin preferences are (AA) if and only if $\alpha \ge \frac{1}{2}$ and (AS) if and only if $\alpha \le \frac{1}{2}$.

Given full belief set and $card({\mathcal {S}}) > 2, \alpha$-MaxMin preferences are (AA) if and only if $\alpha =1$ and (AS) if and only if $\alpha =0$.

Proof

The case of $card({\mathcal {S}}) = 2$ results from straightforward computations. Let $f = (1,0)$ and $g=(0,1)$. Then $V(f) = V(g) = 1-\alpha$ and $V(f/2+g/2) = 1/2$. Thus, V is not quasi-concave if $\alpha < 1/2$ nor is it quasi-convex if $\alpha > 1/2$.

Assume now that $\alpha \ge 1/2$. Let f and g be two acts, if these are co-monotonic, then $V(\lambda f+(1-\lambda )g) = \lambda V(f)+(1-\lambda )V(g)$. If f dominates (weakly) g, then $V(\lambda f + (1-\lambda )g) = \alpha V(g) + (1-\alpha ) V(f)$.

Assume now, without loss of generality that $f_1 > g_1$ and $f_2 < g_2$. Let $\lambda _0$ be the value that maximizes $min_p p(\lambda f_1 + (1-\lambda )g_1) + (1-p)(\lambda f_2 + (1-\lambda )g_2)$ over $\lambda \in [0,1]$. Then ${\mathbb {E}}_p\ \lambda _0 f + (1-\lambda _0)g$ is independent on p.

If $\alpha = 1/2$, then $V_{1/2}(\lambda f + (1-\lambda ) g)$ is linear in $\lambda$ and therefore quasi-concave. If $\alpha > 1/2$, then by definition $V_{\alpha }(\lambda f + (1-\lambda ) g) < V_{1/2}(\lambda f + (1-\lambda ) g)$.

Furthermore, by definition of $\lambda _0$, one gets that $V_{\alpha }(\lambda _0 f + (1-\lambda _0) g) = V_{1/2}(\lambda _0 f + (1-\lambda _0) g)$. Thus, either $V_{1/2}(f)$ or $V_{1/2}(g)$ is smaller than $V_{\alpha }(\lambda _0 f + (1-\lambda _0) g)$ which in turn implies that either $V_{\alpha }(f)$ or $V_{\alpha }(g)$ is smaller than $V_{\alpha }(\lambda _0 f + (1-\lambda _0) g)$.

Finally, given $V_{\alpha }(\lambda f + (1-\lambda ) g)$ is linear in $\lambda$ both on $[0;\lambda _0]$ and on $[\lambda _0;1]$, it must be that for all $\lambda$, either $V_{\alpha }(f)$ or $V_{\alpha }(g)$ is smaller than $V_{\alpha }(\lambda f + (1-\lambda ) g)$ which shows that $V_{\alpha }$ is quasi-concave if $\alpha \ge 1/2$. The same proof applies for quasi-convexity and $\alpha \le 1/2$.

If $card({\mathcal {S}}) > 2$ and $\alpha \ne 0,1$.

Let $f = (1,0,\varepsilon ,\varepsilon ,...)$ and $g = (0,1,\varepsilon ,\varepsilon ,...)$ with $0 < \varepsilon <min\big (\alpha ; (1-\alpha ); \frac{(1-\alpha )^2}{\alpha }\big )$. Finally let $\lambda = \alpha$

Then, $V(f) = V(g) = 1-\alpha$ and $V(\lambda f + (1-\lambda ) g) = \alpha \varepsilon + (1-\alpha ) (1-\alpha ) < (1-\alpha )$ if $\alpha \le 1/2$. If $\alpha > 1/2$, then $V(\lambda f + (1-\lambda ) g) = \alpha \varepsilon + (1-\alpha ) \alpha = \alpha (1-\alpha -\varepsilon )$ which is smaller than $(1-\alpha )$ from the choice of $\varepsilon$ made. In both cases, V is not quasi-concave.

Now let $f = (1,0,1-\varepsilon ,1-\varepsilon ,...)$ and $g = (0,1,1-\varepsilon ,1-\varepsilon ,...)$ with $0 < \varepsilon <min\left( \alpha ; (1-\alpha ); \frac{\alpha ^2}{(1-\alpha )}\right)$. Let $\lambda = \alpha$.

Then $V(f) = V(g) = 1-\alpha$ as before and $V(\lambda f + (1-\lambda ) g) = (1-\alpha )(1-\varepsilon )+\alpha (1-\alpha ) > (1-\alpha )$ if $\alpha \ge \frac{1}{2}$. Otherwise, $V(\lambda f + (1-\lambda ) g) = (1-\alpha )(1-\varepsilon )+\alpha ^2$ which is also strictly greater than $1-\alpha$ from the choice of $\varepsilon$. V is therefore not quasi-convex. $\square$

Note that even though this proposition is only proven for when the decision maker holds the full set of beliefs, the proof can be extended to any other set of beliefs. This would just complexify the computations. Note however that for the second part to hold, there must be at least three states that are perceived as ambiguous. Otherwise, the situation would be as in the case of two states.

From this proposition, one can apply all propositions of the paper. For instance, assuming there are three states of the world, all players satisfy the (R) assumption and that one player is $\alpha$-MaxMin Expected Utility decision maker with $\alpha = \frac{1}{2}$ and full set of belief, then:

• there exists one game where NE does not exist;

• CEB and PEB would exist for all one-shot game with simultaneous moves;

• there exists one game where (CEB and PEB), (CEB and NE), or (PEB and NE) are not the same;

• the principle of indifference holds for PEB;

• there exists one game where the principle of indifference does not hold for either CEB and NE;

• the principle of indifference always holds for PEB;

• there exists one game where the reduction principle does not hold for PEB, CEB and NE.

1.2 Example 1

In Example 1, the players are assumed to be ambiguity seeking with the following payoffs:

Figure 6 provides the best response graph for the first example. As we can see, the best response function of the row player presents a hole.

Fig. 6

NE-admissible strategies for Example 1

This can be seen by computing the best response of the row player when the column player randomizes between Left and Right equiprobably:

$$\begin{aligned} U(\sigma _U) = max_p\ \left[ \frac{1}{2}\sigma _U( (p+2(1-p)-p-(1-p) )\right. \\ \left. + \frac{1}{2}(1-\sigma _U)(-p-(1-p) + 2p - (1-p) ) \right] \end{aligned}$$

Which simplifies to:

$$\begin{aligned} U(\sigma _U) = max_p\ \frac{1}{2} \left[ \sigma _U( 1-p ) + (1-\sigma _U) p \right] \end{aligned}$$

Thus, the value of the mixed action $\sigma _U$ is $\frac{1}{2} \sigma _U$ if $\sigma _U > \frac{1}{2}$ and $\frac{1}{2}(1-\sigma _U)$ otherwise. The best response of the row player is therefore either to play Up or Down but any non-degenerate mixed strategy would lead to a lower payoff.

Fig. 7

CEB/PEB-admissible strategies for Example 1

As seen before, Up and Down are both best responses, thus, if one allows mixing of best responses (as opposed to mixed best responses), then the best response graph for either equilibrium in beliefs is given above.

1.3 Example 2

Example 2 is just a decision problem in disguise as both players’ payoffs are independent on the other’s action.

Figure 8 provides the best response graphs for both pure equilibrium in beliefs and Nash equilibrium.

Fig. 8

Red PEB-admissible strategies to Example 2, Orange NE-admissible strategies

1.4 Example 3

I compute here the best response function for the row and column players for Example 3.

Row player:

Given $\sigma _L$, the value of playing $\sigma _U$ to the row player is given by:

$$\begin{aligned} min_p\ 3 \sigma _U [p \sigma _L + (1-p) \sigma _U (1- \sigma _L)] + (1-\sigma _U) \end{aligned}$$

Which simplifies to $3 \sigma _U \sigma _L + (1-\sigma _U)$ if $\sigma _L \le \frac{1}{2}$ and $3 \sigma _U(1- \sigma _L) + (1-\sigma _U)$ otherwise.

Differentiating with regard to $\sigma _U$ yields that U is the best response if $\sigma _L$ is greater than $\frac{1}{3}$ (with the first condition) and smaller than $\frac{2}{3}$ (with the second condition). Furthermore, if $\sigma _L = \frac{1}{3}$ or $\frac{2}{3}$, the row player is indifferent between all mixed strategies. There is therefore no difference between the EB best responses and the NE best responses in that particular case. This is what gives rise to the “U-turn” shape in Fig. 9.

Fig. 9

PEB-admissible strategies to Example 3

I now compute the mixed best response (NE) for the column player.

Given $\sigma _U$, the value of playing $\sigma _L$ to the row player is given by:

$$\begin{aligned} min_p\ p [\sigma _L (1-\sigma _U) + (1-\sigma _L) \sigma _U] + 2 (1-p) [\sigma _L \sigma _U + (1-\sigma _L)(1-\sigma _U)] \end{aligned}$$

Which yields:

$$\begin{aligned}&min_p\ 2[\sigma _L \sigma _U + (1-\sigma _L)(1-\sigma _U)] + p [\sigma _L (1-\sigma _U) + (1-\sigma _L) \sigma _U\\&\quad - 2\sigma _L \sigma _U - 2(1-\sigma _L)(1-\sigma _U)]\\&min_p\ 2[\sigma _L \sigma _U + (1-\sigma _L)(1-\sigma _U)] + p [3 \sigma _L + 3 \sigma _U - 6 \sigma _L \sigma _U -2] \end{aligned}$$

Assume, for now that $\sigma _U$ is between $\frac{1}{3}$ and $\frac{1}{2}$. Then, regardless of $\sigma _L$, we have that $3 \sigma _L + 3 \sigma _U - 6 \sigma _L \sigma _U -2$ is between $-\frac{1}{2}$ and $\sigma _L-1$ and is negative. We therefore have that $p=1$ is the minimizing probability and the value of $\sigma _L$ is given by $\sigma _L (1-\sigma _U) + (1-\sigma _L) \sigma _U$. When differentiated with respect to $\sigma _L$, we get 1 so that $\sigma _L = 1$ is the best response.

Assume now that $\sigma _U \ge \frac{1}{3}$. Whether $p=0$ or $p=1$ is the minimizing probability now depends on $\sigma _L$ as well. Assume $\sigma _L > \frac{2-3\sigma _U}{3-6\sigma _U}$, then the minimizing probability is $p=0$ so that the value of $\sigma _L$ is now given by $2 [\sigma _L \sigma _U + (1-\sigma _L)(1-\sigma _U)]$ which is decreasing in $\sigma _L$. Indeed, he differential gives $4 \sigma _U - 1$ which is negative given $\sigma _U$ is lower than $\frac{1}{3}$.

The opposite conclusion arises if $\sigma _L \le \frac{2-3\sigma _U}{3-6\sigma _U}$ is assumed. Thus, the best response to $\sigma _U$ if $\sigma _U$ is lower than $\frac{1}{3}$ is $\frac{2-3\sigma _U}{3-6\sigma _U}$. This is what yields the curve in Fig. 10.

The same calculations, but with opposite conclusions can be made when $\sigma _U$ is greater than $\frac{1}{2}$.

Fig. 10

NE-admissible strategies to Example 3

Fig. 11

NE-admissible strategies. 5 mixed-strategy NE, 0 pure!

1.5 Example of a game with five mixed strategy NE

The following example is of interest as it breaks the rule that a game generically only have one pure equilibrium, on mixed equilibrium or one mixed and two pure equilibria. In this example, both players are ambiguity averse.

The game being symmetric, I only provide the calculations for the best responses of the row player. The value of playing $\sigma _U$ is:

$$\begin{aligned}&min_p \quad p [\sigma _U \sigma _L - \sigma _U (1-\sigma _L) - (1-\sigma _U) \sigma _L + 2 (1-\sigma _U)(1-\sigma _L)]\\&\qquad + (1-p)[2 \sigma _U \sigma _L - \sigma _U (1-\sigma _L) - (1-\sigma _U) \sigma _L + (1-\sigma _U) (1-\sigma _L)]\\&\qquad \qquad \qquad \qquad min_p\ [...] + p [(1-\sigma _U) (1-\sigma _L) - \sigma _U \sigma _L]\\&\qquad \qquad \qquad \qquad \qquad min_p\ [...] + p [1-\sigma _U -\sigma _L] \end{aligned}$$

Assume $1- \sigma _U - \sigma _L > 0$. Then $p=0$ and the value of $\sigma _U$ becomes:

$$\begin{aligned} 5 \sigma _U \sigma _L - 2 \sigma _U -2 \sigma _L + 1 \end{aligned}$$

Differentiating with respect to $\sigma _U$ yields $5 \sigma _L-2$. Thus, if $\sigma _L \le \frac{2}{5}$, the best response is $\sigma _U = 0$, which satisfies the condition that $1- \sigma _U - \sigma _L > 0$. Likewise, if $\sigma _L \ge \frac{3}{5}$, the best response is $\sigma _U = 1$.

More interesting is the case where $\sigma _L$ is between $\frac{2}{5}$ and $\frac{3}{5}$. Then if $1- \sigma _U - \sigma _L > 0$ the differential with respect to $\sigma _U$ is positive which implies a best response of $1-\sigma _L$. The same result is obtained if $1-\sigma _U - \sigma _L \ge 0$ is assumed. This gives the sloping line in the middle of the best response graph presented in Fig. 11.

Fig. 12

Full Red PEB-admissible strategies, Dotted Orange reduced game best responses, Full black normalized payoffs to “U” in the reduced game, Full grey: normalized payoffs to “U” in the original game, Dotted black and grey payoff to “D”

There are therefore five mixed equilibria $(\sigma _U;\sigma _L)$: $\left( \frac{1}{2}; \frac{1}{2}\right) , \left( \frac{2}{5}; \frac{2}{5}\right) , \left( \frac{2}{5}; \frac{3}{5}\right) , \left( \frac{3}{5}; \frac{2}{5}\right)$ and $\left( \frac{3}{5}; \frac{3}{5}\right)$.

1.6 Another example of the failure of the reduction principle

In this example, the equilibrium in beliefs is compared to that of the reduced equilibrium. The computations are straightforward and not provided here. I only present in Fig. 12 the best response graph with the valuation of the Up and Down actions. Whereas the payoff to Down are not modified when the game is reduced (there is no ambiguity if Down is chosen), this is not the case for Up. Indeed, whereas the two endpoints (when the column player picks Left or Right) are the same, whether the game is reduced or not, the payoffs of Up when the opponent plays a mixed strategy are not linear in the full game. In the reduced game however, these payoffs are forced to be linear. This implies that both these curves will not cross the Down payoff curve for the same mixed strategy of the column player. Thus, the change in strategy does not occur at the same time and the PEB is therefore different than the mixed equilibrium of the reduced game.