Abstract
After more than 60 years, Shannon’s research continues to raise fundamental questions, such as the one formulated by R. Luce, which is still unanswered: “Why is information theory not very applicable to psychological problems, despite apparent similarities of concepts?” On this topic, S. Pinker, one of the foremost defenders of the widespread computational theory of mind, has argued that thought is simply a type of computation, and that the gap between human cognition and computational models may be illusory. In this context, in his latest book, titled Thinking Fast and Slow, D. Kahneman provides further theoretical interpretation by differentiating the two assumed systems of the cognitive functioning of the human mind. He calls them intuition (system 1) determined to be an associative (automatic, fast and perceptual) machine, and reasoning (system 2) required to be voluntary and to operate logical-deductively. In this paper, we propose a mathematical approach inspired by Ausubel’s meaningful learning theory for investigating, from the constructivist perspective, information processing in the working memory of cognizers. Specifically, a thought experiment is performed utilizing the mind of a dual-natured creature known as Maxwell’s demon: a tiny “man–machine” solely equipped with the characteristics of system 1, which prevents it from reasoning. The calculation presented here shows that the Ausubelian learning schema, when inserted into the creature’s memory, leads to a Shannon-Hartley-like model that, in turn, converges exactly to the fundamental thermodynamic principle of computation, known as the Landauer limit. This result indicates that when the system 2 is shut down, both an intelligent being, as well as a binary machine, incur the same minimum energy cost per unit of information (knowledge) processed (acquired), which mathematically shows the computational attribute of the system 1, as Kahneman theorized. This finding links information theory to human psychological features and opens the possibility to experimentally test the computational theory of mind by means of Landauer’s energy cost, which can pave a way toward the conception of a multi-bit reasoning machine.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Pozo (2008), quoting Ausubel's theory, among others, also identifies two processes of learning: an automatic (effortless), and a controlled (effortful) that occurs voluntarily. These two ways to acquire knowledge are intrinsically (and clearly) attached to dual process theory of mind.
It is important to highlight that our contextual motivation for using a rate of modification of the cognitive structure (a metric over time that is directly proportional to the product of the information and the subsumers) was essentially inspired by Piaget (2001), because, according to Wadsworth (2003), the Piagetian assimilation process accounts for the growth of the intellectual structure that is provoked by, in his words, “a quantitative change”. For his part, Novak (2010) claims that Piaget’s assimilation process and Ausubel's subsumption process are elementally similar. In passing, Ausubel himself recognized a general similarity between Piaget's formulation of the assimilation process and his own assimilation theory (Ausubel and Novak 1978; Nielson 1980), which led us to infer that the Ausubelian subsumption can be quantitatively treated.
Note that the Ausubelian interactional product, sI, which features the core of the learning schema by assimilation, is an associative (adaptive) representational procedure. Goldammer and Kaehr (1989) argued that adaptive learning can be represented by a Hebbian algorithm that simulates the linkage between the domain and the internal structure. On the other hand, Gerstner and Kistler (2002) claimed that Hebbian learning is a bilinear-type operation that represents a mathematical abstraction of coincidence detection mechanisms. So, taking into account that the internal structure is represented in our thought experiment by a space of subsumers, and that a domain space is just the boundary condition of this structure (i.e. the informational ambience), Ausubel's interactional product, consequently, takes the form of a bilinear isomorphism that combines elements of these two spaces, s and I (Konrad 2007 ).
It is worth remembering that Ausubel's theory holds that new information is linked to relevant pre-existing aspects of cognitive structure with both the newly acquired information and the pre-existing structure being modified in the process—based on this conceptual model, Ausubelian theory describes the cognitive process of subsumption with its underlying principles of progressive differentiation and integrative reconciliation (Nielson 1980). But, for this cognitive process occurs, the linkage between new information and pre-existing structure must be made by means of an Ausubelian interactional product, an Ausubelian matching.
Here, D s,I appears outside the integral, because it is not a direct function of s and I, but depends only on measure of the distance between a given information, I, and a reference subsume, s. This metric space is a number, a normalized length equal to 0 when s and I are maximally similar, and equal to 1 when s and I are maximally dissimilar (Li et al. 2004).
Brookes' equation represents the procedure for the retention of information described by Ausubel's theory of meaningful learning, when information and the subsumer can still be dissociated. This notation, considering the plus sign (+) in Brookes' equation, assumes the form of a superposition that indicates a cognitive change provoked by an external stimulus, leading the one-subsumer cognitive structure from state S to state S + ∆S. The consequence of this superposition concept is the immediate characterization of K(S) as an eigenvalue associated with eigenstate S, and K(S + ∆S) as an eigenvalue associated with eigenstate S + ∆S, which gives us the integration limits used in Eq. (3). In this context, it would be preferable to define the process of cognitive change as Brookes' superposition (or Brookes’ state transition), rather than Brookes' equation.
Cole (2011) claims that the use of information in information science is firmly linked to the information itself, which Brookes (1980) defined in his fundamental equation of information science as that which modifies a user’s knowledge structure. In an earlier paper, Cole (1997) advocated a quantitative treatment for information based on Brookes' viewpoint. Bawden (2011) also suggests a possible quantitative treatment of Brookes' equation.
For convenience, \( \ln \left(\frac{K(S + \Updelta S)}{K(S)}\right) \) was rewritten as \( \ln \left(1 + \frac{\Updelta K}{K(S)}\right) \), because we not know the \( \frac{K(S + \Updelta S)}{K(S)} \) ratio, but the \( \frac{\Updelta K}{K(S)} \) term can be determined, as will be shown below.
Toyabe et al. (2010) have also achieved a k B TI ≥ k B T ln(·) relationship for a quasistatic information heat engine such as the Szilárd engine. This means that a clever feedback protocol, such as Maxwell's demon, converts information into energy up to k B TI.
For that \( k_{B} T \left( {\Updelta E=\frac{\text{heat}}{\text{info}}}\right)\)is a minimum amount of heat per unit of information (Levitin,1998), the\(\frac{\text{ln(.)}}{\text{info}}=1\) condition must then be satisfied.
In Ausubel’s theory, meaningfull learning occurs when the individual creates a connection between information, I, and subsumer, s, resulting in an “interactional product” sI (Moreira 2011). The notion of subsumption of information in the Ausubelian meaningfull learning is that s and I remain dissociated from the product sI during the first stage of assimilation process of information and, over time, I becomes closely linked with s, being anchored in the ideational complex sI.
According to Todd (1999), Brookes also assumed that the unit of knowledge is concept based, a notion derived from Ausubel's assimilation theory of cognitive learning.
During the process of assimilation, the new meaningful gradually loses its identity as it becomes part of the modified anchoring structure. This process is termed obliterative subsumption and is dependent on dissociability strength between s and I. This gradual loss of separable identity ends with the meaning being forgoteen when the idea falls below the “threshold of availability” proposed by Ausubel. (Seel 2012).
Brookes' equation shows that the two cognitive states, K(S) and I, are compressed into one state, K(S + ∆S). Thus, the \( \frac{\Updelta K}{K(S)} \) term represents a relative cognitive change, a squeeze cognitive (a reduction) equal to 50 %. Consequently, the Boltzmann factor obtained from Eq. (9) is \( e^{{ - \frac{\Updelta E}{{k_{B} T}}}} \le 2 \), since \( \frac{K(S + \Updelta S)}{K(S)} \) is equal to \( 1 + \frac{\Updelta S}{K(S)} \). This weighting factor evaluates the relative probability of a determined state occurring in a multi-state system, i.e., it is a “non-normalized probability” that needs be ≫1 for the system to be described for a non-quantum statistics; otherwise, the system exhibits quantum behavior (Carter 2001). So, the Boltzmann factor ≤2 calculated here indicates that the subsumption process requires a quantum statistics to describe the cognitive learning. In other words, our calculation suggests that the final stage of the subsumption process of information—within the Kahneman’s intuition-system—produces a quantum quantity, as Aerts (2011) has advocated. Toyabe et al. (2010) and Sagawa and Ueda (2008) also have achieved this same value for the Boltzmann factor in an feedback control system, however, they raised this outcome by means of physics; here, we were able to achieve this outcome by means of psychology.
As an external stimulus is subsumed into an individual’s knowledge structure—which, from Bennett’s thermodynamics viewpoint, corresponds to the deletion of information from memory devices—the relative cognitive change \( \left( {\frac{\Updelta K}{K(S)}} \right) \) assumes negative values, indicating a cognitive squeeze.
This singular consequence reinforces the idea that Landauer limit is an intrinsic feature of algorithmic rules, rather than a quantity associated to physical part of computer.
For certain, we are not asserting that the human mind, as a whole, operates like an algorithmic machine, but, when system 2 was left out of our calculation for the reason explained above, the outcome pointed precisely to Landauer's principle of thermodynamics, which—considering Frasca's work—mathematically indicates that the adaptive (associative) learning process (notably Ausubel's progressive differentiation) works in the same way as a step-by-step computational procedure.
Recently, a group of researchers was able to experimentally show Landauer's principle in a two-state bit memory (Bérut et al. 2012). It is therefore plausible to presume that Landauer's principle in human mind can also be subjected to experimentation. Landauer's limit is an energy boundary, in the form of heat dissipation—a physical quantity, which can provide a robust evidence of an algorithmic-like activity within the "gears" of a type 1 cognitive system. In addition, Landauer's boundary can also quantitatively underpin latest Kahneman’s qualitative presuppositions for a intuition-system (Morewedge and Kahneman 2010).
References
Aerts, D. (2011) Quantum theory and conceptuality: Matter, stories, semantics and space-time. arXiv:1110.4766v1 [quant-ph].
Ausubel, D. P. (1963). The psychology of meaningful verbal learning: An introduction to school learning. New York: Grune & Stratton.
Ausubel, D. P. (2010). The acquisition and retention of knowledge: A cognitive view. Dordrect, Boston: Kluwer Academic Publishers.
Ausubel, D. P., Novak, J. D., & Hanesian, H. (1978). Educational psychology: A cognitive view (2nd ed.). New York: Holt, Rinehart & Winston.
Bawden, D. (2011). Brookes equation: The basis for a qualitative characterization of information behaviours. Journal of Information Science, 37(1), 101–108.
Bennett, C. H. (1982). The thermodynamics of computation: A review. International Journal of Theoretical Physics, 21, 905–940.
Bennett, C. H. (1987). Demons, engines and the second law. Scientific American, 257:88–96.
Bennett, C. H. (2003). Notes on Landauer’s principle, reversible computation and Maxwell’s demon by Charles H. Bennett. Studies in History and Philosophy of Modern Physics, 34, 501–510.
Bérut, A., Arakelyan, A., Petrosyan, A., Ciliberto, S., Dillenschneider, R., & Lut, E. (2012). Experimental verification of Landauer’s principle linking information and thermodynamics. Nature, 483, 187–189.
Brookes, B. C. (1980). The foundations of information science, Part 1. Philosophical aspects. Journal of Information Science, 2(3/4), 125–134.
Brookes, B. C. (1981). Information technology and the science of information. In: R. N. Oddy et al. (Eds.), Information retrieval research (pp. 1–8). London: Butterworths.
Brown, H. D. (2007). Principles of language learning and teaching. New York: Addison Wesley Longman.
Burgin, M. (2010). Theory of information: Fundamentality, diversity and unification. Singapore: World Scientific Publishing.
Buzzoni, M. (2008). Thought experiment in the natural sciences. Wuerzburg: Koenigshausen & Neumann.
Carter, A. H. (2001). Classical and statistical thermodynamics. New Jersey: Prentice-Hall, Inc.
Castro, A. (2013). On the quantum principles of cognitive learning. Commentary on Pothos & Busemeyer. Behavioral and Brain Sciences (to appears).
Cole, C. (1997). Calculating the information content of an information process for a domain expert using Shannon’s mathematical theory of communication: A preliminary analysis. Information Processing and Management, 33(6), 715–726.
Cole, C. (2011). A theory of information need for information retrieval that connects information to knowledge. Journal of the American Society for Information Science and Technology, 62(7), 1216–1231.
Davenport, D. (2012). Computationalism: Still the only game in town. A reply to Swiatczak’s “Conscious Representations: An intractable problem for the computational theory of mind”. Minds and Machines, 22, 183–190.
Fodor, J. (2000). The mind doesn’t work that way: The scope and limits of computational psychology. Cambridge: MIT Press.
Frasca, M. (2012). Probabilistic Turing machine and Landauer limit. arXiv:1206.0207v2 [math-ph].
Fresco, N. (2012). The explanatory role of computation in cognitive science. Minds and Machines, 22, 353–380.
Gerstner, W., & Kistler, W. M. (2002). Mathematical formulations of Hebbian learning. Biological Cybernetics, 87, 404–415.
Goldammer, E., & Kaehr, R. (1989). Poly-contextural modelling of heterarchies in brain functions. In: R. M. J. Cotterill (Ed.), Models of brain functions (pp. 483–497). Cambridge: Cambridge University Press.
Hosoya, A., Maruyama, K., & Shikano, Y. (2011). Maxwell’s demon and data compression. Physical Review E, 84, 061117.
Jones, D. S., Plank, M. J., & Sleeman, B. D. (2010). Differential equations and mathematical biology. Boca Raton: Taylor & Francis Group.
Kahneman, D. (2002). Maps of bounded rationality: A perspective on intuitive judgment and choice. Nobel Prizes 2002: Presentations, Biographies and Lectures. Almqvist and Wiksell Int.
Kahneman, D. (2011). Thinking, fast and slow. New York: Farrar, Straus and Giroux.
Khemlani, S., & Johnson-Laird, P. N. (2012, January 30). Theories of the syllogism: A meta-analysis. Psychological Bulletin. Advance online publication. doi:10.1037/a0026841.
Kittel, C., & Kroemer, H. (1980). Thermal physics (2nd ed). New York: Freeman & Co.
Konrad, A. (2007). On inquiry: Human concept formation and construction of meaning through library and information science intermediation (Unpublished doctoral dissertation). University of California, Berkeley. Retrieved from http://escholarship.org/uc/item/1s76b6hp.
Ladyman, J. (2009). What does it mean to say that a physical system implements a computation? Theoretical Computer Science, 410, 376–383.
Lambson, B., Carlton, D., & Bokor, J. (2011). Exploring the thermodynamic limits of computation in integrated systems: Magnetic memory, nanomagnetic logic, and the landauer limit. Physical Review Letters, 107, 010604.
Laming, D. (2010). Statistical information and uncertainty: A critique of applications in experimental psychology. Entropy, 12, 720–771.
Landauer, R. (1961). Irreversibility and heat generation in the computing process. IBM Journal of Research and Development, 5, 183–191.
Lenski, W. (2010). Information: A conceptual investigation. Information, 1, 74–118.
Levitin, L. B. (1998). Energy cost of information transmission (along the path to understanding). Physica D, 120, 162–167.
Li, M., Chen, X., Li, X., Ma, B., & Vitányi, P. M. B. (2004). The similarity metric. IEEE Transaction on Information Theory, 50–12, 3250–3264.
Luce, R. D. (2003). Whatever happened to information theory in psychology? Review of General Psychology, 7, 183–188.
Maroney, O. (2009). Information processing and thermodynamic entropy. In E. N. Zalta (Ed.), The stanford encyclopedia of philosophy (Fall 2009 Edition). http://plato.stanford.edu/archives/fall2009/entries/information-entropy/
Moon, B., Hoffman, R. R., & Cañas, A. (2011). Applied concept mapping: Capturing, analyzing, and organizing knowledge. Boca Raton, FL: CRC Press.
Moreira, M. A. (1993). Metacognitive learning and related topics in the proceedings of the third international seminar on misconceptions and educational strategies in science and mathematics. Ithaca, NY: Misconceptions Trust.
Moreira, M. A. (2011). Meaningful learning: From the classical to the critical view. Meaningful Lenarnin Review, 1, 1–15.
Morewedge, C. K., & Kahneman, D. (2010). Associative processes in intuitive judgment. Trends in Cognitive Sciences, 14(10), 435–440.
Nielson, L. A. (1980). Comparative evaluation of the cognitive theories of Piaget and Ausubel (Report No. CS 005811). Sarasota, FL: American Reading Conference. (ERIC Document Reproduction Service No. ED195 987).
Norton, J. D. (2011). Waiting for Landauer. Studies in History and Philosophy of Modern Physics, 42, 184–198.
Novak, J. D. (1977). An alternative to piagetian psychology for science and mathematics education. Science Education, 61, 453–477.
Novak, J. D. (2010). Learning, creating and using knowledge: Concept maps as facilitative tools in Schools and corporations (2nd ed.). New York: Taylor & Francis.
Novak, J. D. (2011). A theory of education: Meaningful learning underlies the constructive integration of thinking, feeling, and acting leading to empowerment for commitment and. Meaningful Learning Review, 1, 1–14.
Novak, J. D., & Gowin, D. B. (2002) Learning how to learn. Cambridge.
Piaget, J. (2001). The psychology of intelligence. London: Routledge Classics.
Pinker, S. (2005). So how does the mind work? Mind and Language, 20, 1–24.
Plenio, M. B., & Vitelli, V. (2001). The physics of forgetting: Landauer’s erasure principle and information theory. Contemporary Physics, 42, 25–60.
Pozo, J.I. (2008) Aprendices y maestros: La Psicologia Cognitiva Del Aprendizaje. Alianza Editorial Sa, 614p.
Sagawa, T., & Ueda, M. (2008). Second law of thermodynamics with discrete quantum feedback control. Physical Review Letters, 100, 080403.
Sagawa, T., & Ueda, M. (2010). Generalized Jarzynski equality under nonequilibrium feedback control. Physical Review Letters, 104, 090602.
Seel, N. M. (2003). Psychologie des Lernens. Lehrbuch für Pädagogen und Psychologen (München Reinhardt).
Seel, N. M. (2012) Assimilation theory of learning. In N. M. Seel (Ed.), Encyclopedia of the sciences of learning. New York: Springer.
Seising, S. (2010). Cybernetics, system(s) theory, information theory and fuzzy sets and systems in the 1950s and 1960s. Information Sciences, 180, 4459–4476.
Smith, R. A. (2006). Culture and the arts in education: Critical essays on shaping human experience. New York: Teachers College Press.
Swiatczak, B. (2011). Conscious representations: An intractable problem for the computational theory of mind. Minds and Machines, 21, 19–32.
Szilárd, L. (1929). Uber die Ausdehnung der phänomenologischen Thermodynamik auf die Schwankungserscheinungen, Zeits. Physik, 32, 753–788.
Todd, R. J. (1999). Back to our beginnings: Information utilization, Bertram Brookes and the fundamental equation of information science. Information Processing and Management, 35, 851–870.
Toyabe, S., Sagawa, T., Ueda, M., Muneyuki, E., & Sano, M. (2010). Experimental demonstration of information-to energy conversion and validation of the generalized Jarzynski equality. Nature Physics, 6, 988–992.
Vigo, R. (2011). Representational information: A new general notion and measure of information. Information Sciences, 181, 4847–4859.
Wadsworth, B. J. (2003). Piaget’s theory of cognitive and affective development (5th ed.). Boston, USA: Allyn & Bacon.
Zurek, W. H. (1990) Maxwell’s demon, Szilard’s engine and quantum measurements. In H. S. Leff & A. F. Rex (Eds.), Maxwell’s Demon. Entropy, information, computing (pp. 249–259). Bristol: Adam Hilger.
Acknowledgments
This work was supported by the Centro Nacional de Pesquisa Tecnológica em Informática para a Agricultura (Empresa Brasileira de Pesquisa Agropecuária—Embrapa).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
de Castro, A. The Thermodynamic Cost of Fast Thought. Minds & Machines 23, 473–487 (2013). https://doi.org/10.1007/s11023-013-9302-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11023-013-9302-x