Skip to main content
SpringerLink
Log in

Refusing to budge: a confirmatory bias in decision making?

  • Original Article
  • Published:
Mind & Society Aims and scope Submit manuscript

Abstract

Confirmatory bias, defined as the tendency to misinterpret new pieces of evidence as confirming previously held hypotheses, can lead to implacable, even incorrect decision making. It is one of the biases, along with anchoring, framing, and other judgment heuristic errors, that may lead to non-optimal behavior. This paper tests for the existence of confirmatory bias behavior in a uniquely economic setting (tax policy) and in a context relatively lacking in ambiguity. It also tests whether the confirmatory bias phenomenon can be prevalent enough to affect aggregate outcomes, a characteristic important in economic models in particular. The results indicate not only that confirmatory bias exists, but that the confirmatory bias effect may be stronger for evidence relating to losses than for comparable evidence relating to gains.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Notes

  1. Confirmatory bias can also refer, more broadly, to control strategies where people seek (and not just interpret) biased information when forming initial hypotheses, but that separate branch of the literature is not the focus of this study.

  2. It may be useful to think of the problem as loosely akin to a “selectivity bias” or “added-variable bias” in information processing; the idea is that sometimes people misread evidence relevant to a particular hypothesis and either misread this evidence by refusing to admit its contrary nature, thereby disregarding it as useless (selectivity bias), or people misread the evidence by processing it as supportive of previously held views even when it is not (added-variable bias). Either way, such data manipulation leads to biased estimates (both in support of the a priori hypothesis) and poor decision-making behavior.

  3. Overconfidence, a behavioral phenomenon related to confirmatory bias, has been subject to some scrutiny in the economics literature (see, for example, Ganguly et al. 2000; Madsen 1994; Camerer 1997; Camerer and Lovallo 1999; Eales et al. 1990; Dubra 2004; Hvide 2002; Golec and Tamarkin 1995; Barber and Odean 2001), but similar attention has not been paid to the confirmatory bias effect itself.

  4. While it appears that direct tests of the confirmatory bias phenomenon have yet to be published in the economics literature, it is possible to reevaluate previously published work in light of the confirmatory bias effect. For example, in an historical article documenting the pervasive extent of homestead failures in the Upper Great Plains between 1890 and 1925, Libecap and Hansen (2000) come to the conclusion that the widespread homesteading failures, which resulted in significant monetary and nonmonetary losses, were made because: “…the previous wet period and the strong claims of dryfarming experts led homesteaders to discount observations of dry weather and to place more weight on past opinions about the ability of the region to withstand droughts” (p. 28). This conclusion, while reasonable, begs the question as to why such a misreading of the available information took place. A strong case could be made that this is evidence of confirmatory bias behavior. It is quite likely that other papers in a similar vein could also be reinterpreted––and better understood––under the confirmatory bias lens.

  5. In another context, Ausubel (1991) finds that, despite consistent, unambiguous evidence of credit card debt (monthly statements), many credit card holders (47%) insist that they “nearly always” pay their balances in full (with an additional 26% claiming that they “sometimes pay” their balances in full). The fact is that three-quarters of all active credit card accounts at major banks average over $1,000 in balances at any moment in time. This belief by debt holders in their ability to pay off their credit cards every month and not incur continuous, often exorbitant finance charges, flies in the face of their (unambiguous) monthly available evidence. It can also lead them to incur further, perhaps nonoptimal, extra credit card debt.

  6. Keep in mind that the subjects are being asked to evaluate only the new evidence. They are not (initially) being asked to reevaluate their overall opinion of capital punishment. Once the new evidence is assimilated, subjects may very well then use it to update their beliefs regarding capital punishment, but it is incorrect according to the traditional neo-classical model to use prior beliefs to subjectively analyze newly available evidence.

  7. This structure conforms to the “added-variable” type of bias in informational processing mentioned in the introduction.

  8. See Rabin and Schrag (1999) for a fuller exposition of the model, including situations of possible underconfidence.

  9. Other applications have been made to management consulting (Watson et al. 1998), medical practitioners (Arkes et al. 1981) forensic evaluation (Borum et al. 1993) and prediction (Kahneman and Tversky 1973). Cutting closer to home, Cross (1977) has found that 94% of college professors (drawn from a wide distribution of disciplines) say they do above average work. An informal survey of economic professors at the 2004 AEA conference found similar ability overconfidence levels, as well as a professed readiness to bet real money on those beliefs.

  10. Preston and Harris (1965) found significant overconfidence regarding driver ability from respondents surveyed in a hospital just after a bad car accident.

  11. This implies, notably, that learning and experience will not automatically correct an agent’s misperceived beliefs. As Baumann et al. (1991) note, agents must first perceive a need for change in order for learning and experience to effect change, and confirmatory bias implies that actors do not perceive anything wrong with their biasedly confident decision making.

  12. In a similar vein, overconfidence may be used not just to fool ourselves into being happier, more productive individuals, overconfidence may sometimes even be used to fool others. As an example, an overconfidence in one’s defensive abilities may lead to challenges of a much more powerful enemy. Such hubris, rather than leading to defeat, may actually cause the rival to question herself and ultimately back down without a fight, thereby handing the victory to the less able, but overconfident, aggressor. The author thanks Rob Fleck for pointing out this natural selection application of the overconfidence effect.

  13. The data from the two survey periods were combined to form a single comprehensive data set for use in all future data analysis, after first confirming through statistical analysis (an insignificant dummy variable proxying for the two sections) that the data were formally consistent.

  14. In order to reduce survey bias, the order of the answer choices among the questions and among the respondents was mixed; half the subjects received answer choices moving from “greatly helped…” or “greatly weaken…” vertically down the page to “greatly hurt…” or “greatly strengthen…” while the other half of the respondents received surveys with the answer choices moving in the other direction.

  15. If we can assume that this subject pool is representative of the entire Montana State University student body, then this revealed proclivity is not surprising; a majority of MSU students are Montana natives, and Montana historically has been a Republican-leaning state.

  16. The separate sections of the survey were clearly delineated in bold as “Part I,” “Part II,” etc., and presented on separate pages of the survey booklet in order to reduce, as much as possible, potential order effects between the questions. This method was suggested in Dillman (2000).

  17. This is the point at which it is correct to use the new evidence to update prior beliefs, but only, of course, if that new evidence was first analyzed objectively and independently.

  18. Even models that allow for an altruism component of voting behavior (Nelson 1994), never suggest that this component is strong enough to overwhelm the objective of maximization of real income, although, altruism may dampen it.

  19. An alternative interpretation of subjects’ response may be related to ideology’s commitment enhancing role in settings of uncertainty (Downs 1957), but that explanation lacks teeth as there is little uncertainty in this particular setting, which is the point. The relatively unambiguous survey evidence makes it directly clear to subjects how these policy proposals will affect their ultimate tax returns. Put another way, there are two types of uncertainty which may affect actors’ behavior: uncertainty with the datum and uncertainty of preferences. As in most of economics, this paper assumes away uncertainty of preferences. That leaves only uncertainty in the data which the survey evidence was explicitly designed to avoid.

  20. The author thanks an anonymous reviewer for pointing out that the more linear relationship observed in the Survey A graph, as opposed to the Survey B graph, may be due to a ceiling effect.

  21. This is not necessarily true for confirmatory bias stemming from control strategies, such as with Wason’s 2–4–6 and other selection tasks, where strongly held beliefs are not required for biased hypothesis updating.

  22. Note that 84 observations in Survey A and 73 observations in Survey B, or 60 and 54% of the total sample sizes respectively, fit into the two tails of the normal distributions of responses from both surveys.

  23. Another way of making the same point is that {Support, Favor} and {Opposition, Against} correspond to the added-variable interpretation of confirmatory bias behavior, while {Support, Neutral} and {Opposition, Neutral} relate to the selectivity bias interpretation of confirmatory bias behavior.

  24. This author believes that, had the survey evidence been of the more traditional ambiguous kind, the results in the final row of Survey A would have exhibited a stronger Against column response, and a less strong Neutral column response. It is easier to misread conflicting evidence as supportive of your initial hypothesis if it is somewhat ambiguous; if it is not ambiguous, the best you can do while still maintaining your initial beliefs is to disregard the new information.

  25. These results from Survey A and Survey B are both normally distributed as well.

  26. Akerlof (1989), in an insightful paper on voting behavior, considers a similar setting when he models decision-making behavior subject to an “illusion” bias. He finds, as we do, that while “it is usually assumed that individuals vote in their own best interest…this best interest may be neither obvious nor pleasant to contemplate…the desire and ability for self-delusion can lead to poor social decisions.” (p. 10, italics added).

  27. Common Data Set, online at: http://www.commondataset.org/ (The Office of Planning and Analysis, Montana State University-Bozeman 2003).

  28. A somewhat different interpretation, suggested to the author by Ken Troske, is that conservatives (who tend to support Bush tax policies) are more politically ideological than liberals (who tend not to support Bush tax policy). This difference in political ideological degree may be the driving factor behind the nonsymmetric results, rather than that of prospect theory and varying preferences for gains versus losses. From the survey data collected it is impossible to be certain which interpretation is correct, but either way, both interpretations support the assumption that order effects are not driving the results; they merely offer different explanations for the motivation behind the confirmatory bias effect.

  29. An anonymous reviewer also suggested that the nonsymmetric results may be due to a framing effect whereby people “support cuts” more than they “oppose increases” (somewhat similar to the asymmetry of information it takes to confirm versus falsify hypotheses (Trope and Thompson 1997; Cameron and Trope 2004)). While we can not rule out this interpretation entirely either, we believe it unlikely to be the complete explanation for our results, or more than 62% of those initially predisposed against Bush Administration tax policy would have more positively “supported” the tax cuts.

References

  • Akerlof GA (1989) The economics of illusion. Econ Pol 1(1):1–15

    Article  Google Scholar 

  • Arkes HR, Wartmann RL, Saville PD, Harkness AR (1981) Hindsight bias among phyicians weighing the likelihood of diagnoses. J Appl Psychol 66(2):252–254

    Article  Google Scholar 

  • Ausubel LM (1991) The failure of competition in the credit card market. Am Econ Rev 81(1):50–81

    Google Scholar 

  • Babcock L, Loewenstein G (1997) Explaining bargaining impasse: the role of self-serving biases. J Econ Perspect 11(1):109–126

    Google Scholar 

  • Barber BM, Odean T (2001) Boys will be boys: gender, overconfidence, and common stock investment. Q J Econ 116(1):261–292

    Article  Google Scholar 

  • Baumann AO, Deber RB, Thompson GG (1991) Overconfidence among physicians and nurses: the ‘Micro-Certainty, Macro-Uncertainty’ phenomenon. Soc Sci Med 32(2):167–174

    Article  Google Scholar 

  • Borum R, Otto R, Golding S (1993) Improving clinical judgment and decision making in forensic evaluation. J Psychol Law 21(1):35–76

    Google Scholar 

  • Bruner JS, Potter MC (1964) Interference in visual recognition. Science 144(3617):424–425

    Article  Google Scholar 

  • Camerer CF (1997) Progress in behavioral game theory. J Econ Perspect 11(4):167–188

    Google Scholar 

  • Camerer CF, Lovallo D (1999) Overconfidence and excess entry: an experimental approach. Am Econ Rev 89(1):306–318

    Article  Google Scholar 

  • Cameron JA, Trope Y (2004) Stereotype-biased search and processing of information about group members. Soc Cogn 22(6):650–672

    Article  Google Scholar 

  • Cross P (1977) Not can but will college teaching be improved. New Dir High Ed 17:1–15

    Article  Google Scholar 

  • Dardenne B, Leyens JP (1995) Confirmation bias as a social skill. Pers Soc Psychol Bull 21(11):1229–1239

    Article  Google Scholar 

  • Darley JM, Gross PH (1983) A hypothesis-confirming bias in labeling effects. J Pers Soc Psychol 44(1):20–33

    Article  Google Scholar 

  • Dillman DA (2000) Mail and internet surveys: the tailored design method, 2nd edn, Wiley, New York

    Google Scholar 

  • Dougherty TW, Turban DB, Callender JC (1994) Confirming first impressions in the employment interview: a field study of interviewer behavior. J Appl Psychol 79(5):659–665

    Article  Google Scholar 

  • Downs A (1957) An economic theory of democracy. Harper and Row Publishers, New York

    Google Scholar 

  • Dubra J (2004) Optimism and overconfidence in search. Rev Econ Dyn 7:198–218

    Article  Google Scholar 

  • Dunning D, Meyerowitz JA, Holzberg AD (1989) Ambiguity and self-evaluation: the role of idiosyncratic trait definitions in self-serving assessments of ability. J Pers Soc Psychol 57(6):1082–1090

    Article  Google Scholar 

  • Eales JS, Engel BK, Hauser RJ, Thompson SR (1990) Grain price expectations of Illinois farmers and grain merchandisers. Am J Agric Econ 72(3):701–708

    Article  Google Scholar 

  • Ganguly AR, Kagel JH, Moser DV (2000) Do asset market prices reflect traders’ judgment biases? J Risk Uncertain 20(3):219–245

    Article  Google Scholar 

  • Golec J, Tamarkin M (1995) Do betters prefer long shots because they are risk-lovers, or are they just overconfident? J Risk Uncertain 11:51–64

    Article  Google Scholar 

  • Griffin D, Tversky A (1992) The weighing of evidence and the determinants of confidence. Cognit Psychol 24(3):411–435

    Article  Google Scholar 

  • Haverkamp BE (1993) Confirmatory bias in hypothesis testing for client-identified and counselor self-generated hypotheses. J Couns Psychol 40(3):303–315

    Article  Google Scholar 

  • Hvide HK (2002) Pragmatic beliefs and overconfidence. J Econ Behav Organ 48:15–28

    Article  Google Scholar 

  • Kahneman D, Tversky A (1973) On the psychology of prediction. Psychol Rev 80(4):237–251

    Article  Google Scholar 

  • Kahneman D, Slovic P, Tversky A (eds) (1982) Judgment under uncertainty: heuristics and biases. Cambridge University Press, New York

    Google Scholar 

  • Keren G (1987) Facing uncertainty in the game of bridge: a calibration study. Organ Behav Hum Decis Process 39(1):98–114

    Article  Google Scholar 

  • Keren G (1988) On the ability of monitoring non-veridical perceptions and uncertain knowledge: some calibration studies. Acta Psychol 67(2):95–119

    Article  Google Scholar 

  • Klayman J, Ha YW (1987) Confirmation, disconfirmation, and information in hypothesis testing. Psychol Rev 94(2):211–228

    Article  Google Scholar 

  • Libecap GD, Hansen ZK (2000) ‘Rain Follows the Plow’ and dryfarming doctrine: the climate information problem and homestead failure in the Upper Great Plains, 1890–1925. National Bureau of Economic Research Working Paper, Historical Paper 127

  • Lord CG, Ross L, Lepper MR (1979) Biased assimilation and attitude polarization: the effect of prior theories on subsequently considered evidence. J Pers Soc Psychol 37(11):2098–2109

    Article  Google Scholar 

  • Lord CG, Lepper MR, Preston E (1984) Considering the opposite: a corrective strategy for social judgment. J Pers Soc Psychol 47(6):1231–1243

    Article  Google Scholar 

  • Madsen JB (1994) Tests of rationality versus an ‘Over Optimist’ bias. J Econ Psychol 15:587–599

    Article  Google Scholar 

  • Miller AG, McHoskey JW, Bane CM, Dowd TG (1993) The attitude polarization phenomenon: role of response measure, attitude extremity, and behavioral consequences of reported attitude change. J Pers Soc Psychol 64(4):561–574

    Article  Google Scholar 

  • Nelson P (1994) Voting and imitative behavior. Econ Inq 32:92–102

    Article  Google Scholar 

  • Nickerson RS (1998) Confirmation bias: a ubiquitous phenomenon in many guises. Rev Gen Psychol 2(2):175–220

    Article  Google Scholar 

  • Nisbett RE, Ross L (1980) Human inference: strategies and shortcomings of social judgment. Prentice Hall, New Jersey

    Google Scholar 

  • The Office of Planning and Analysis, Montana State University-Bozeman (2003) Common Data Set. The Thomson Corporation and US News and World Report

  • Peltzman S (1980) The growth of government. J Law Econ 23(2):209–287

    Article  Google Scholar 

  • Plous S (1991) Biases in the assimilation of technological breakdowns: do accidents make us safer? J Appl Soc Psychol 21(13):1058–1082

    Article  Google Scholar 

  • Preston CE, Harris S (1965) Psychology of drivers in traffic accidents. J Appl Psychol 49(4):284–288

    Article  Google Scholar 

  • Rabin M (1998) Psychology and economics. J Econ Lit 36(1):11–46

    Google Scholar 

  • Rabin M, Schrag J (1999) First impressions matter: a model of confirmatory bias. Q J Econ 114(1):37–82

    Article  Google Scholar 

  • Snyder M, Swann WB (1978) Behavioral confirmation in social interaction: from social perception to social reality. J Exp Soc Psychol 14:148–162

    Article  Google Scholar 

  • Strohmer DC, Shivy VA (1994) Bias in counselor hypothesis testing: testing the robustness of counselor confirmatory bias. J Couns Dev 73:191–197

    Google Scholar 

  • Svenson O (1981) Are we all less risky and more skillful than our fellow driver? Acta Psychol 47:233–250

    Google Scholar 

  • Swann WB, Snyder M (1980) On translating beliefs into action: theories of ability and their application in an institutional setting. J Pers Soc Psychol 38(6):879–888

    Article  Google Scholar 

  • Taylor SE, Brown JD (1988) Illusion and well-being: a social psychological perspective on mental health. Psychol Bull 103(2):193–210

    Article  Google Scholar 

  • Trope Y, Mackie DM (1987) Sensitivity to alternatives in social hypothesis-testing. J Exp Soc Psychol 23:445–459

    Article  Google Scholar 

  • Trope Y, Thompson EP (1997) Looking for truth in all the wrong places? Asymmetric search of individuating information about stereotyped group members. J Pers Soc Psychol 73(2):229–241

    Article  Google Scholar 

  • Tweney RD, Doherty ME, Worner WJ, Pliske DB, Mynatt CR (1980) Strategies of rule discovery in an inference task. Q J Exp Psychol 32:109–123

    Article  Google Scholar 

  • Wason PC (1960) On the failure to eliminate hypotheses in a conceptual task. Q J Exp Psychol 12:129–140

    Article  Google Scholar 

  • Watson A, Rodgers T, Dudek D (1998) The human nature of management consulting: judgment and expertise. Manage Dec Econ 19(7–8):495–503

    Article  Google Scholar 

Download references

Acknowledgments

Many special thanks to Rob Fleck, Dino Falaschetti, Susan Feigenbaum, Scott Plous, Ron Harstad, David Mandy, Jeff Milyo, Ken Troske, and two anonymous reviewers for their insight, criticism, support, and advice.

Author information

Authors and Affiliations

  1. Department of Economics, University of Missouri—St Louis, 8001 Natural Bridge Road, St Louis, MO, 63121, USA

    Lea-Rachel D. Kosnik

Authors
  1. Lea-Rachel D. Kosnik
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lea-Rachel D. Kosnik.

Appendix 1: Survey questions

Appendix 1: Survey questions

1.1 Part I: Initial predisposition

  1. Q1

    To what extent do you think the tax cuts which Congress passed and George W. Bush signed into law during this past presidential term have affected the US economy?

    • greatly helped the US economy

    • somewhat helped the US economy

    • had no effect on the US economy

    • somewhat hurt the US economy

    • greatly hurt the US economy

  2. Q2

    In the long run do you think tax cuts strengthen or weaken the economy?

    • greatly weaken the economy

    • somewhat weaken the economy

    • have no effect on the economy

    • somewhat strengthen the economy

    • greatly strengthen the economy

  3. Q3

    In recent years, President Bush and Congress have made major cuts in federal income tax rates. What is your opinion regarding this tax agenda?

    • strongly approve

    • somewhat approve

    • have no opinion

    • somewhat disapprove

    • strongly disapprove

  4. Q4

    Just your opinion: What do you think about the size of President Bush’s tax cuts? Should they have been:

    • much smaller than they were

    • somewhat smaller than they were

    • about the same as they were

    • somewhat bigger than they were

    • much bigger than they were

1.2 Part II: Evidence assimilation

1.2.1 Survey A:

President Bush is due to propose to Congress, within the next month, an additional tax cut to be effective for the 2004 tax year. This tax cut takes the form of an extension to the HOPE scholarship Credit. Currently the HOPE Scholarship Credit is an income tax credit for students in the first two years of college (or other eligible post-secondary training), which allows them to credit their tax returns up to 100% of the first $1,000 of tuition and fees and 50% of the second $1,000 (the amounts are indexed for inflation after 2001). The credit is available on a per-student basis for net tuition and fees (less grant aid) paid for college enrollment after 31 December 1997.

The proposal President Bush is presenting to Congress next month extends the benefits and terms of this credit. It will be available for up to four years of college (instead of just the first two), and the amount of tuition and fees available for credit will be raised to 100% of the first $2,000 of tuition and fees and 50% of the second $2,000 (the amounts are still indexed for inflation). Note that other education and learning credits––such as the Lifetime Learning Credit––will not be affected in any way. If you are a current college student, therefore, you will unambiguously benefit from this proposal.

1.2.2 Survey B:

President Bush is due to propose to Congress, within the next month, a tax increase to be effective for the 2004 tax year. This tax increase takes the form of a reduction of the HOPE scholarship credit. Currently the HOPE Scholarship Credit is an income tax credit for students in the first 2 years of college (or other eligible post-secondary training), which allows them to credit their tax returns up to 100% of the first $1,000 of tuition and fees and 50% of the second $1,000 (the amounts are indexed for inflation after 2001). The credit is available on a per-student basis for net tuition and fees (less grant aid) paid for college enrollment after 31 December 1997.

The proposal President Bush is presenting to Congress next month reduces the benefits and terms of this credit. It will only be available for the first year of college (instead of the first two years as before), and the amount of tuition and fees available for credit will be reduced to 50% of the first $1,000 of tuition and fees and 25% of the second $1,000 (the amounts are still indexed for inflation). Note that other education and learning credits––such as the Lifetime Learning Credit––will not be affected in any way. If you are a current college student, therefore, you will unambiguously lose from this proposal.

  1. Q5

    The HOPE Scholarship Credit is an income tax credit for:

    • employee moving expenses

    • college tuition and fees expenses

    • on the job training expenses

    • medical expenses

  2. Q6

    The proposed amendment to the HOPE Scholarship Credit, if passed, would take effect:

    • retroactively back to tax year 1997

    • in tax year 2010

    • in tax year 2007

    • in tax year 2004

  3. Q7

    The proposed amendment to the HOPE Scholarship Credit includes increasing (decreasing) the number of years college students are eligible to claim this credit from two to:

    • one

    • six

    • four

    • three

  4. Q8

    Please indicate your support or opposition for this latest tax proposal:

    • strongly support

    • somewhat support

    • have no opinion

    • somewhat oppose

    • strongly oppose

1.3 Part III: Attitude polarization

  1. Q9

    After reading and thinking about the new proposed tax amendment, which of the following statements comes closest to your point of view?

    • I now have much greater faith in President Bush’s tax agenda

    • I now have greater faith in President Bush’s tax agenda

    • I now have less faith in President Bush’s tax agenda

    • I now have much less faith in President Bush’s tax agenda

    • My opinion has not changed

  2. Q10

    After reading and thinking about the new proposed tax amendment, which of the following statements comes closest to your point of view?

    • I am much less inclined to support President Bush’s tax agenda

    • I am somewhat less inclined to support President Bush’s tax agenda

    • I am somewhat more inclined to support President Bush’s tax agenda

    • I am much more inclined to support President Bush’s tax agenda

    • My opinion has not changed

  3. Q11

    After reading and thinking about the new proposed tax amendment, which of the following statements comes closest to your point of view?

    • I see passage of all of President Bush’s income tax proposals much more likely

    • I see passage of all of President Bush’s income tax proposals somewhat more likely

    • I see passage of all of President Bush’s income tax proposals somewhat less likely

    • I see passage of all of President Bush’s income tax proposals much less likely

    • My opinion has not changed

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kosnik, LR.D. Refusing to budge: a confirmatory bias in decision making?. Mind Soc 7, 193–214 (2008). https://doi.org/10.1007/s11299-007-0043-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11299-007-0043-5

Keywords

Search

Navigation