Skip to main content
SpringerLink
Log in

Rational Task Analysis: A Methodology to Benchmark Bounded Rationality

  • Published:
Minds and Machines Aims and scope Submit manuscript

Just as a scissors cannot cut paper without two blades, a theory of thinking and problem solving cannot predict behavior unless it encompasses both an analysis of the structure of task environments and an analysis of the limits of rational adaptation to task requirements.

(Newell and Simon 1972, p. 55)

Abstract

How can we study bounded rationality? We answer this question by proposing rational task analysis (RTA)—a systematic approach that prevents experimental researchers from drawing premature conclusions regarding the (ir-)rationality of agents. RTA is a methodology and perspective that is anchored in the notion of bounded rationality and aids in the unbiased interpretation of results and the design of more conclusive experimental paradigms. RTA focuses on concrete tasks as the primary interface between agents and environments and requires explicating essential task elements, specifying rational norms, and bracketing the range of possible performance, before contrasting various benchmarks with actual performance. After describing RTA’s core components we illustrate its use in three case studies that examine human memory updating, multitasking behavior, and melioration. We discuss RTA’s characteristic elements and limitations by comparing it to related approaches. We conclude that RTA provides a useful tool to render the study of bounded rationality more transparent and less prone to theoretical confusion.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. Krueger and Funder (2004, Table 1, p. 317) provide a “partial list” of 42 errors of judgment, and http://en.wikipedia.org/wiki/List_of_cognitive_biases (retrieved on Dec. 22, 2014) collects over 180 cognitive biases, many of which can be re-interpreted as smart adaptations (Gigerenzer 2004).

  2. See Scriven (1991, p. 346) for an elaboration of this example.

  3. Behavioral patterns that closely mirror the shape of task environments are reminiscent of Simon’s ant-on-the-beach analogy: “The apparent complexity of our behavior over time is largely a reflection of the complexity of the environment in which we find ourselves” (Simon 1996, p. 53).

  4. This Bayesian agent formalized the learning task as one of inferring a posterior distribution over the relevant history window of environmental states, a function that maps each choice history onto one of a discrete number of states, and the probability of obtaining a reward for choosing either option in each possible state of the environment (see Sims et al. 2013, p. 143 ff., for details).

  5. Similar shifts of perspective are reported in the literature on decision by sampling (Fiedler and Juslin 2006; Stewart et al. 2006). The consequences of presenting risk-related information in different representational formats are explored in studies on the description-experience gap (Hertwig et al. 2004; Hertwig and Erev 2009). Both paradigms provide strong additional arguments for the adoption of a subject-based perspective when conducting research and interpreting experimental results.

  6. RA’s relative neglect of agent-based constraints was also responsible for Simon’s skepticism towards this framework (Simon 1991).

  7. See Todd and Gigerenzer (2001), for a comparison of Simon’s scissors with the alternative metaphors of Shepard’s mirror and Brunswik’s lens.

  8. See the related notions of “achievement” and “correspondence” (Hammond and Stewart 2001).

  9. A volume edited by Hammond and Stewart (2001) provides an overview of Brunswik’s essential contributions.

  10. A review of representative design and its impact on judgment and decision-making research is provided by Dhami et al. (2004).

References

  • Akerlof, G. A., & Shiller, R. J. (2010). Animal spirits: How human psychology drives the economy, and why it matters for global capitalism. Princeton, NJ: Princeton University Press.

    Google Scholar 

  • Anderson, J. R. (1990). The adaptive character of thought. Hillsdale, NJ: Lawrence Erlbaum.

    Google Scholar 

  • Anderson, J. R., Bothell, D., Byrne, M. D., Douglass, S., Lebiere, C., & Qin, Y. (2004). An integrated theory of the mind. Psychological Review, 111(4), 1036–1060.

    Article  Google Scholar 

  • Anderson, J. R., & Milson, R. (1989). Human memory: An adaptive perspective. Psychological Review, 96(4), 703–719.

    Article  Google Scholar 

  • Anderson, J. R., & Schooler, L. J. (1991). Reflections of the environment in memory. Psychological Science, 2(6), 396–408.

    Article  Google Scholar 

  • Ariely, D. (2008). Predictably irrational: The hidden forces that shape our decisions. New York, NY: Harper Collins.

    Google Scholar 

  • Birnbaum, M. H. (1983). Base rates in Bayesian inference: Signal detection analysis of the cab problem. The American Journal of Psychology, 96(1), 85–94.

  • Brunswik, E. (1943). Organismic achievement and environmental probability. Psychological Review, 50(3), 255–272.

    Article  Google Scholar 

  • Brunswik, E. (1955). Representative design and probabilistic theory in a functional psychology. Psychological Review, 62(3), 193–217.

    Article  Google Scholar 

  • Brunswik, E. (1956). Perception and the representative design of psychological experiments. Berkeley, CA: University of California Press.

    Google Scholar 

  • Burns, B. D. (2004). Heuristics as beliefs and as behaviors: The adaptiveness of the “hot hand”. Cognitive Psychology, 48(3), 295–331.

    Article  Google Scholar 

  • Burns, K. (2001). TRACS: A tool for research on adaptive cognitive strategies: The game of confidence and consequence. http://www.tracsgame.com. Accessed 1 May 2015.

  • Burns, K. (2002). On straight TRACS: A baseline bias from mental models. In Proceedings of the 24th Annual Meeting of the Cognitive Science Society (pp. 154–159). Lawrence Erlbaum, Hillsdale, NJ.

  • Dawkins, R. (1999). The extended phenotype: The long reach of the gene, revised edn. Oxford, UK: Oxford University Press.

    Google Scholar 

  • Dhami, M. K., Hertwig, R., & Hoffrage, U. (2004). The role of representative design in an ecological approach to cognition. Psychological Bulletin, 130(6), 959–988.

    Article  Google Scholar 

  • Ferguson, N. (2008). The ascent of money: A financial history of the world. London, UK: Penguin.

    Google Scholar 

  • Fiedler, K., & Juslin, P. (Eds.). (2006). Information sampling and adaptive cognition. New York, NY: Cambridge University Press.

    Google Scholar 

  • Fu, W. T., & Gray, W. D. (2004). Resolving the paradox of the active user: Stable suboptimal performance in interactive tasks. Cognitive Science, 28(6), 901–935.

    Article  Google Scholar 

  • Gigerenzer, G. (1991). How to make cognitive illusions disappear: Beyond heuristics and biases. European Review of Social Psychology, 2(1), 83–115.

    Article  Google Scholar 

  • Gigerenzer, G. (1996). On narrow norms and vague heuristics: A reply to Kahneman and Tversky. Psychological Review, 103, 592–596.

    Article  Google Scholar 

  • Gigerenzer, G. (2004). The irrationality paradox. Behavioral and Brain Sciences, 27(3), 336–338.

    Article  Google Scholar 

  • Gigerenzer, G., & Brighton, H. (2009). Homo heuristicus: Why biased minds make better inferences. Topics in Cognitive Science, 1(1), 107–143.

    Article  Google Scholar 

  • Gigerenzer, G., Hertwig, R., & Pachur, T. (Eds.). (2011). Heuristics: The foundations of adaptive behavior. New York, NY: Oxford University Press.

    Google Scholar 

  • Gigerenzer, G., Todd, P. M., & the ABC Research Group (1999). Simple heuristics that make us smart. New York, NY: Oxford University Press.

  • Gray, W. D., & Boehm-Davis, D. A. (2000). Milliseconds matter: An introduction to microstrategies and to their use in describing and predicting interactive behavior. Journal of Experimental Psychology: Applied, 6(4), 322–335.

    Google Scholar 

  • Gray, W. D., Neth, H., & Schoelles, M. J. (2006). The functional task environment. In A. F. Kramer, D. A. Wiegman, & A. Kirlik (Eds.), Attention: From theory to practice (pp. 100–118). New York, NY: Oxford University Press.

    Chapter  Google Scholar 

  • Hammond, K. R., & Stewart, T. R. (2001). The essential Brunswik: Beginnings, explications, applications. New York, NY: Oxford University Press.

    Google Scholar 

  • Herrnstein, R. J. (1981). Self-control as response strength. In C. M. Bradshaw, E. Szabadi, & C. F. Lowe (Eds.), Recent developments in the quantification of steady-state operant behavior (pp. 3–20). Amsterdam, NL: Elsevier.

    Google Scholar 

  • Herrnstein, R. J. (1982). Melioration as behavioral dynamism. Quantitative Analyses of Behavior, 2, 433–458.

    Google Scholar 

  • Herrnstein, R. J. (1990). Behavior, reinforcement and utility. Psychological Science, 1(4), 217–224.

    Article  Google Scholar 

  • Herrnstein, R. J. (1990). Rational choice theory. American Psychologist, 45(3), 356–367.

    Article  Google Scholar 

  • Herrnstein, R. J. (1991). Experiments on stable suboptimality in individual behavior. The American Economic Review, 81(2), 360–364.

    Google Scholar 

  • Herrnstein, R. J., & Prelec, D. (1992). A theory of addiction. Choice over time, 331–360.

  • Herrnstein, R. J., & Vaughan, W, Jr. (1980). Melioration and behavioral allocation. In J. E. R. Staddon (Ed.), Limits to action: The allocation of individual behavior (pp. 143–176). New York, NY: Academic Press.

    Chapter  Google Scholar 

  • Hertwig, R., Barron, G., Weber, E. U., & Erev, I. (2004). Decisions from experience and the effect of rare events in risky choice. Psychological Science, 15(8), 534–539.

    Article  Google Scholar 

  • Hertwig, R., & Erev, I. (2009). The description-experience gap in risky choice. Trends in Cognitive Sciences, 13(12), 517–523.

    Article  Google Scholar 

  • Hertwig, R., Hoffrage, U., & the ABC Research Group (2013). Simple heuristics in a social world. New York, NY: Oxford University Press.

  • Heyman, G. M., & Dunn, B. (2002). Decision biases and persistent illicit drug use: An experimental study of distributed choice and addiction. Drug and Alcohol Dependence, 67(2), 193–203.

    Article  Google Scholar 

  • Howes, A., Lewis, R. L., & Vera, A. (2009). Rational adaptation under task and processing constraints: Implications for testing theories of cognition and action. Psychological Review, 116(4), 717.

    Article  Google Scholar 

  • Kahneman, D. (2003). Maps of bounded rationality: Psychology for behavioral economics. American Economic Review, 93(5), 1449–1475.

    Article  Google Scholar 

  • Kahneman, D., Slovic, P., & Tversky, A. (1982). Judgment under uncertainty: Heuristics and biases. Cambridge, UK: Cambridge University Press.

    Book  Google Scholar 

  • Kahneman, D., & Tversky, A. (1996). On the reality of cognitive illusions. Psychological Review, 103(3), 582–591.

    Article  Google Scholar 

  • Kieras, D. E., & Meyer, D. E. (2000). The role of cognitive task analysis in the application of predictive models of human performance. In J. M. C. Schraagen, S. E. Chipman, & V. L. Shalin (Eds.), Cognitive task analysis (pp. 237–260). Mahwah, NJ: Lawrence Erlbaum.

    Google Scholar 

  • Knight, F. H. (1921). Risk, uncertainty and profit. Chicago, IL: University of Chicago Press.

    Google Scholar 

  • Koehler, J. J. (1996). The base rate fallacy reconsidered: Descriptive, normative, and methodological challenges. Behavioral and Brain Sciences, 19(01), 1–17.

    Article  Google Scholar 

  • Krueger, J. I., & Funder, D. C. (2004). Towards a balanced social psychology: Causes, consequences, and cures for the problem-seeking approach to social behavior and cognition. Behavioral and Brain Sciences, 27(3), 313–327.

    Google Scholar 

  • Lewis, R. L., Howes, A., & Singh, S. (2014). Computational rationality: Linking mechanism and behavior through bounded utility maximization. Topics in Cognitive Science, 6(2), 279–311.

    Article  Google Scholar 

  • Lopes, L. L. (1991). The rhetoric of irrationality. Theory and Psychology, 1(1), 65–82.

    Article  Google Scholar 

  • Meyer, D. E., & Kieras, D. E. (1997). A computational theory of executive cognitive processes and multiple-task performance. I: Basic mechanisms. Psychological Review, 104(1), 3–65.

    Article  Google Scholar 

  • Neth, H., Carlson, R. A., Gray, W. D., Kirlik, A., Kirsh, D., & Payne, S. J. (2007). Immediate interactive behavior: A symposium on embodied and embedded cognition. In D. S. McNamara & J. G. Trafton (Eds.), Proceedings of the 29th Annual Meeting of the Cognitive Science Society (pp. 33–34). Cognitive Science Society, Austin, TX.

  • Neth, H., & Gigerenzer, G. (2015). Heuristics: Tools for an uncertain world. In R. Scott & S. Kosslyn (Eds.), Emerging trends in the social and behavioral sciences. New York, NY: Wiley Online Library.

  • Neth, H., Khemlani, S. S., & Gray, W. D. (2008). Feedback design for the control of a dynamic multitasking system: Dissociating outcome feedback from control feedback. Human Factors, 50(4), 643–651.

    Article  Google Scholar 

  • Neth, H., Khemlani, S. S., Oppermann, B., & Gray, W. D. (2006). Juggling multiple tasks: A rational analysis of multitasking in a synthetic task environment. In Proceedings of the Human Factors and Ergonomics Society, vol. 50, (pp. 1142–1146). Sage, San Francisco, CA.

  • Neth, H., Sims, C. R., & Gray, W. D. (2005). Melioration despite more information: The role of feedback frequency in stable suboptimal performance. In Proceedings of the Human Factors and Ergonomics Society, vol. 49, (pp. 357–361). Sage, Orlanco, FL.

  • Neth, H., Sims, C. R., & Gray, W. D. (2006). Melioration dominates maximization: Stable suboptimal performance despite global feedback. In R. Sun & N. Miyake (Eds.), Proceedings of the 28th Annual Meeting of the Cognitive Science Society (pp. 627–632). Lawrence Erlbaum, Hillsdale, NJ.

  • Neth, H., Sims, C. R., Veksler, V. D., & Gray, W. D. (2004). You can’t play straight TRACS and win: Memory updates in a dynamic task environment. In K. D. Forbus, D. Gentner & T. Regier (Eds.), Proceedings of the 26th Annual Meeting of the Cognitive Science Society (pp. 1017–1022). Lawrence Erlbaum, Hillsdale, NJ.

  • Newell, A. (1990). Unified theories of cognition. Cambridge, MA: Harvard University Press.

    Google Scholar 

  • Newell, A., & Simon, H. A. (1972). Human problem solving. Englewood Cliffs, NJ: Prentice Hall.

    Google Scholar 

  • Nickerson, R. S. (2000). Null hypothesis significance testing: A review of an old and continuing controversy. Psychological Methods, 5, 241–301.

    Article  Google Scholar 

  • Norman, D. A., & Bobrow, D. G. (1975). On data-limited and resource-limited processes. Cognitive Psychology, 7(1), 44–64.

    Article  Google Scholar 

  • Oaksford, M., & Chater, N. (2007). Bayesian rationality: The probabilistic approach to human reasoning. New York, NY: Oxford University Press.

    Book  Google Scholar 

  • Pleskac, T. J., & Hertwig, R. (2014). Ecologically rational choice and the structure of the environment. Journal of Experimental Psychology: General, 143(5), 2000–2019.

    Article  Google Scholar 

  • Rachlin, H., & Laibson, D. I. (Eds.). (1997). The matching law: Papers on psychology and economics by Richard Herrnstein. New York, NY: Russell Sage Foundation.

    Google Scholar 

  • Ross, L., & Nisbett, R. E. (1991). The person and the situation: Perspectives of social psychology. New York, NY: McGraw-Hill.

    Google Scholar 

  • Samuels, R., Stich, S., & Bishop, M. (2002). Ending the rationality wars: How to make disputes about human rationality disappear. In R. Elio (Ed.), Common sense, reasoning and rationality (pp. 236–268). New York, NY: Oxford University Press.

    Chapter  Google Scholar 

  • Sanborn, A. N., Griffiths, T. L., & Navarro, D. J. (2010). Rational approximations to rational models: Alternative algorithms for category learning. Psychological Review, 117(4), 1144–1167.

    Article  Google Scholar 

  • Scriven, M. (1991). The methodology of evaluation. In A. A. Bellack & H. M. Kliebard (Eds.), Curriculum and evaluation (pp. 334–371). Berkeley, CA: McCutchan.

    Google Scholar 

  • Shakeri, S. (2003). A mathematical modeling framework for scheduling and managing multiple concurrent tasks. Ph.D. thesis, Oregon State University, Corvallis, OR.

  • Shakeri, S., & Funk, K. (2007). A comparison of human and near-optimal task management behavior. Human Factors, 49(3), 400–416.

    Article  Google Scholar 

  • Simon, H. A. (1955). A behavioral model of rational choice. The Quarterly Journal of Economics, 69(1), 99–118.

    Article  Google Scholar 

  • Simon, H. A. (1956). Rational choice and the structure of the environment. Psychological Review, 63(2), 129–138.

    Article  Google Scholar 

  • Simon, H. A. (1990). Invariants of human behavior. Annual Reviews in Psychology, 41(1), 1–20.

    Article  Google Scholar 

  • Simon, H. A. (1991). Cognitive architectures and rational analysis: Comment. In K. VanLehn (Ed.), Architectures for intelligence: The 22nd Carnegie Mellon Symposium on Cognition (pp. 25–39). Lawrence Erlbaum, Hillsdale, NJ.

  • Simon, H. A. (1996). The Sciences of the artificial (3rd ed.). Cambridge, MA: The MIT Press.

    Google Scholar 

  • Sims, C. R., Neth, H., Jacobs, R. A., & Gray, W. D. (2013). Melioration as rational choice: Sequential decision making in uncertain environments. Psychological Review, 120(1), 139–154. doi:10.1037/a0030850.

    Article  Google Scholar 

  • Stewart, N., Chater, N., & Brown, G. D. A. (2006). Decision by sampling. Cognitive Psychology, 53(1), 1–26.

    Article  Google Scholar 

  • Thaler, R. H. (1994). Quasi rational economics. New York, NY: Russell Sage Foundation.

    Google Scholar 

  • Todd, P. M., & Gigerenzer, G. (2001). Shepard’s mirrors or Simon’s scissors? Behavioral and Brain Sciences, 24(04), 704–705.

    Article  Google Scholar 

  • Todd, P. M., Gigerenzer, G., & the ABC Research Group (2012). Ecological rationality: Intelligence in the world. New York, NY: Oxford University Press.

  • Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. Science, 185(4157), 1124–1131.

    Article  Google Scholar 

  • Vaughan, W, Jr. (1981). Melioration, matching, and maximization. Journal of the Experimental Analysis of Behavior, 36(2), 141–149.

    Article  MathSciNet  Google Scholar 

  • Venturino, M. (1997). Interference and information organization in keeping track of continually changing information. Human Factors, 39(4), 532–539.

  • Wilson, J. Q., & Herrnstein, R. J. (1985). Crime and human nature. New York, NY: Simon & Schuster.

    Google Scholar 

  • Yechiam, E., Erev, I., Yehene, V., & Gopher, D. (2003). Melioration and the transition from touch-typing training to everyday use. Human Factors, 45(4), 671–684.

    Article  Google Scholar 

Download references

Acknowledgments

We thank the attendants of the workshop on Finding Foundations for Bounded and Adaptive Rationalityl (taking place on Nov. 22–24, 2013, and organized by Ralph Hertwig, Arthur Paul Pedersen, and Renata Suter) as well as two anonymous reviewers for helpful feedback and suggestions.

Author information

Authors and Affiliations

  1. Center for Adaptive Behavior and Cognition (ABC), Max Planck Institute for Human Development, Berlin, Germany

    Hansjörg Neth

  2. Social Psychology and Decision Sciences, University of Konstanz, Konstanz, Germany

    Hansjörg Neth

  3. Drexel University, Philadelphia, PA, USA

    Chris R. Sims

  4. Rensselaer Polytechnic Institute, Troy, NY, USA

    Wayne D. Gray

Authors
  1. Hansjörg Neth
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chris R. Sims
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wayne D. Gray
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hansjörg Neth.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Neth, H., Sims, C.R. & Gray, W.D. Rational Task Analysis: A Methodology to Benchmark Bounded Rationality. Minds & Machines 26, 125–148 (2016). https://doi.org/10.1007/s11023-015-9368-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11023-015-9368-8

Keywords

Search

Navigation