Skip to main content
SpringerLink
Log in

A Neural Network for Creative Serial Order Cognitive Behavior

  • Published:
Minds and Machines Aims and scope Submit manuscript

Abstract

If artificial neural networks are ever to form the foundation for higher level cognitive behaviors in machines or to realize their full potential as explanatory devices for human cognition, they must show signs of autonomy, multifunction operation, and intersystem integration that are absent in most existing models. This model begins to address these issues by integrating predictive learning, sequence interleaving, and sequence creation components to simulate a spectrum of higher-order cognitive behaviors which have eluded the grasp of simpler systems. Its capabilities are described based on simulations calling for increasing levels of functionality and are used to show how the model can progress from fundamental sequence learning and recall tasks to sophisticated behaviors such as an ability to solve simple mathematical expressions and a creative capacity for the formation and application of inductive rules.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  • Anderson, J. A. (1995). Teaching arithmetic to a neural network. In An introduction to neural networks. Cambridge, MA: MIT Press.

  • Anumolu, V., Bray, N., & Reilly, K. (1997). Neural network models of strategy development in children. Neural Networks, 10, 7–24.

    Article  Google Scholar 

  • Baddeley, A. (1992). Working memory. Science, 255, 556–559.

    Article  Google Scholar 

  • Badre, D., & Wagner, A. D. (2006). Computational and neurobiological mechanisms underlying cognitive flexibility. Proceedings of the National Academy of Sciences of the United States of America, 103(18), 7186–7191.

    Article  Google Scholar 

  • Baram, Y. (1994). Memorizing binary vector sequences by a sparsely encoded network. IEEE Transactions on Neural Networks, 5, 974–981.

    Article  Google Scholar 

  • Bradski, G., Carpenter, G., & Grossberg, S. (1994). STORE working memory networks for storage and recall of arbitrary temporal sequences. Biological Cybernetics, 71, 469–480.

    Article  Google Scholar 

  • Browne, A., & Sun, R. (2001). Connectionist inference models. Neural Networks, 14, 1331–1355.

    Article  Google Scholar 

  • Burgess, N., & Hitch, G. (1996). A connectionist model of STM for serial order. In S. Gathercole (Ed.), Models of short-term memory. East Sussex, UK: Psychology Press.

    Google Scholar 

  • Carpenter, G., & Grossberg, S. (1987). A massively parallel architecture for a self-organizing neural pattern recognition machine. Computer Vision, Graphics, and Image Processing, 37, 54–115.

    Article  Google Scholar 

  • Cho, B., Rosenbloom, P. S., & Dolan, C. P. (1993). Neuro-Soar: A neural-network architecture for goal-oriented behavior. In P. Rosenbloom, J. Laird, & A. Newell (Eds.), The soar papers: Research on integrated intelligence. Cambridge, MA: MIT Press.

    Google Scholar 

  • Clark, A. (1993). Associative engines. Cambridge, MA: MIT Press.

    Google Scholar 

  • Crone, E. A., Wendelken, C., Donohue, S. E., & Bunge, S. A. (2006). Neural evidence for dissociable components of task-switching. Cerebral Cortex, 16, 475–486.

    Article  Google Scholar 

  • Dawson, M., & Schopflocher, D. (1992). Autonomous processing in parallel distributed processing networks. Philosophical Psychology, 5, 199–219.

    Article  Google Scholar 

  • Dayhoff, J. (1990). Neural network architectures: An introduction. New York: Van Nostrand Reinhold.

    Google Scholar 

  • Donaldson, S. (2001). An artificial neural network for multi-level interleaved and creative serial order cognitive behavior. Doctoral dissertation, Department of Computer Science, University of Alabama at Birmingham, Birmingham.

    Google Scholar 

  • Donaldson, S. (2003a). An artificial neural network model for reading comprehension. In H. Arabnia, R. Joshua, & Y. Mun, (Eds.), Proceedings of the internal conference on artificial intelligence (Vol. 1). Las Vegas, NV: CSREA Press.

    Google Scholar 

  • Donaldson, S. (2003b). A neural network for high-level cognitive control of serial order behavior. In D. Ventura, & S. Das (Eds.), Proceedings of the 7th joint conference on information sciences (6th international conference on computational intelligence and natural computing). Research Triangle Park, NC: Association for Intelligent Machinery.

    Google Scholar 

  • Dyer, M. (1991). Connectionism versus symbolism in high-level cognition. In T. Horgan & J. Tienson (Eds.), Connectionism and the philosophy of mind. Dordrecht, Netherlands: Kluwer.

    Google Scholar 

  • Elman, J. (1990). Finding structure in time. Cognitive Science, 14, 179–211.

    Article  Google Scholar 

  • Estes, W. K. (1972). An associative basis for coding and organization in memory. In A. Melton & E. Martin (Eds.), Coding processes in human memory. Washington, D.C.: V.H. Winston and Sons.

    Google Scholar 

  • Gaudiano, P., & Grossberg, S. (1991). Vector associative maps: unsupervised real-time error-based learning and control of movement trajectories. Neural Networks, 4, 147–183.

    Article  Google Scholar 

  • Goebel, R. (1991). Binding, episodic short-term memory, and selective attention, or why are PDP models poor at symbol manipulation? In D. Touretzky, J. Elman, T. Sejnowski, & G. Hinton (Eds.), Connectionist models: Proceedings of the 1990 summer school. San Mateo, CA: Morgan Kaufmann.

    Google Scholar 

  • Grossberg, S. (1969). Some networks that can learn, remember, and reproduce any number of complicated space-time patterns, I. Journal of Mathematics and Mechanics, 19, 53–91.

    MATH  MathSciNet  Google Scholar 

  • Grossberg, S. (1978a). A theory of human memory: self-organization and performance of sensory-motor codes, maps, and plans. In R. Rosen & F. Snell (Eds.), Progress in theoretical biology. New York: Academic Press.

    Google Scholar 

  • Grossberg, S. (1978b). Behavioral contrast in short-term memory: serial binary memory models or parallel continuous memory models? Journal of Mathematical Psychology, 17, 199–219.

    Article  Google Scholar 

  • Grossberg, S. (1987). Competitive learning: From interactive activation to adaptive resonance. Cognitive Science, 11, 23–63.

    Article  Google Scholar 

  • Hinton, G., & Sejnowski, T. (1986). Learning in Boltzmann machines. In D. Rumelhart & J. McClelland (Eds.), Parallel distributed processing: Explorations in the microstructure of cognition, Volume 1: Foundations. Cambridge, MA: MIT Press.

    Google Scholar 

  • Hofstadter, D. (1995). Fluid concepts and creative analogies. New York: Basic.

    Google Scholar 

  • Hopfield, J. (1982). Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Sciences, 79, 2554–2558.

    Article  MathSciNet  Google Scholar 

  • Houghton, G. (1990). The problem of serial order: a neural network model of sequence learning and recall. In R. Dale, C. Mellish, & M. Zock (Eds.), Current research in natural language generation. London: Academic Press.

    Google Scholar 

  • Houghton, G., Hartley, T., & Glasspool, D. (1996). The representation of words and nonwords in short-term memory: Serial order and syllable structure. In S. Gathercole (Ed.), Models of short-term memory. East Sussex, UK: Psychology Press.

    Google Scholar 

  • Kandel, E. (1995). Cellular mechanisms of learning and memory. In E. Kandel, J. Schwartz, & T. Jessell (Eds.), Essentials of neural science and behavior. Norwalk, CT: Appleton & Lange.

    Google Scholar 

  • Kanerva, P. (1988). Sparse distributed memory. Cambridge, MA: MIT Press.

    MATH  Google Scholar 

  • Koechlin, E., & Jubault, T. (2006). Broca’s area and the hierarchical organization of human behavior. Neuron, 50, 963–974.

    Article  Google Scholar 

  • Kolodner, J. (1997). Educational implications from analogy: A view from case based reasoning. American Psychologist, 52, 57–66.

    Article  Google Scholar 

  • Kosko, B. (1988). Adaptive inference in fuzzy knowledge networks. In M. Gupta & T. Yamakawa (Eds.), Fuzzy computing: Theory, hardware and applications. Amsterdam: Elsevier Science.

    Google Scholar 

  • Kuhl, P. (1994). Learning and representation in speech and language. Current Opinion in Neurobiology, 4, 812–822.

    Article  Google Scholar 

  • Laird, J. E., Newell, A., & Rosenbloom, P. S. (1987). SOAR: An architecture for general intelligence. Artificial Intelligence, 33, 1–64.

    Article  Google Scholar 

  • Lashley, K. (1951). The problem of serial order in behavior. In L. Jeffress (Ed.), Cerebral mechanisms in behavior. New York: John Wiley.

    Google Scholar 

  • Lenat, D., Guha, R. V., Pittman, K., Pratt, D., & Shepherd, M. (1990). Cyc: Toward programs with common sense. Communications of the ACM, 33(8), 30–49.

    Article  Google Scholar 

  • MacGregor, J. (1987). Short-term memory capacity: Limitation or optimization? Psychological Review, 94, 107–108.

    Article  Google Scholar 

  • McClelland, J., McNaughton, B., & O’Reilly, R. (1995). Why there are complementary learning systems in the hippocampus and neocortex: Insights from the successes and failures of connectionist models of learning and memory. Psychological Review, 102, 419–457.

    Article  Google Scholar 

  • McClelland, J., & Rumelhart, D. (1988). Explorations in parallel distributed processing. Cambridge, MA: MIT Press.

    Google Scholar 

  • Murdock, B. (1996). Item, associative, and serial-order information in TODAM. In S. Gathercole (Ed.), Models of shortterm memory. East Sussex, UK: Psychology Press.

  • O’Reilly, R. (1996). Biologically plausible error-driven learning using local activation differences: the generalized recirculation algorithm. Neural Computation, 8(5), 895–938.

    MathSciNet  Google Scholar 

  • O’Reilly, R., & Munakata, Y. (2000). Computational explorations in cognitive neuroscience. Cambridge, MA: MIT Press.

    Google Scholar 

  • Page, M., & Norris, D. (1998). The primacy model: a new model of immediate serial recall. Psychological Review, 105, 781.

    Article  Google Scholar 

  • Pinker, S. (1997). How the mind works. New York: W. W. Norton & Company.

    Google Scholar 

  • Pinker, S., & Bloom, P. (1990). Natural language and natural selection. Behavioral and Brain Sciences, 13, 707–784.

    Google Scholar 

  • Pollack, J. (1990). Recursive distributed representations. Artificial Intelligence, 46, 77–105.

    Article  Google Scholar 

  • Riccio, D., Rabinowitz, V., & Axelrod, S. (1994). Memory: When less is more. American Psychologist, 49, 917–926.

    Article  Google Scholar 

  • Rosenblatt, F. (1964). A model for experiential storage in neural networks. In J. T. Tou & R. H. Wilcox (Eds.), Computer and information sciences. Washington, D.C: Spartan.

    Google Scholar 

  • Schacter, D. (1996). Searching for memory. New York: Basic.

    Google Scholar 

  • Schneider, W. (1993). Varieties of working memory as seen in biology and in connectionist/control architectures. Memory and Cognition, 21, 184–192.

    Google Scholar 

  • Schumacher, E. H., Seymour, T. L., Glass, J. M., Fencsik, D. E., Lauber, E. J., Kieras, D. E., & Meyer, D. E. (2001). Virtually perfect time sharing in dual-task performance: uncorking the central cognitive bottleneck. Psychological Science, 12(2), 101–108.

    Article  Google Scholar 

  • Sigman, M., & Dehaene, S. (2006). Dynamics of the central bottleneck: dual-task and task uncertainty. PLOS Biology, 4(7), 1227–1238.

    Article  Google Scholar 

  • Smith, E. E., Geva, A., Jonides, J., Miller, A., Reuter-Lorenz, P., & Koeppe, R. A. (2001). The neural basis of task-switching in working memory: effects of performance and aging. Proceedings of the National Academy of Sciences of the United States of America, 98(4), 2095–2100.

    Article  Google Scholar 

  • Smithers, T. (1997). Autonomy in robots and other agents. Brain and Cognition, 34, 88–106.

    Article  Google Scholar 

  • Smolensky, P. (1988). On the proper treatment of connectionism. Behavioral and Brain Sciences, 11, 1–74.

    Article  Google Scholar 

  • Smolensky, P. (1990). Tensor product variable binding and the representation of symbolic structures in connectionist systems. Artificial Intelligence, 46, 159–216.

    Article  MathSciNet  MATH  Google Scholar 

  • Sohn, M., Ursu, S., Anderson, J. R., Stenger, V. A., & Carter, C. S. (2000). The role of prefrontal cortex and posterior parietal cortex in task switching. Proceedings of the National Academy of Sciences of the United States of America, 97(24), 13448–13453.

    Article  Google Scholar 

  • Stoet, G., & Snyder, L. (2003). Executive control and task-switching in monkeys. Neuropsychologia, 41, 1357–1364.

    Article  Google Scholar 

  • Sun, R. (1994). Integrating rules and connectionism for robust common sense reasoning. New York: John Wiley.

    Google Scholar 

  • Taylor, J. (1995). Self-organization in the time domain. In M. Arbib (Ed.), The handbook of brain theory and neural networks. Cambridge, MA: MIT Press.

    Google Scholar 

  • Thurston, P., & Reilly, K. (1987). Association of successive stimuli in neural-like networks. Proceedings of the 1987 ACM Southeast Regional Conference. New York: ACM Press.

    Google Scholar 

  • Touretzky, D., & Hinton, G. (1988). A distributed connectionist production system. Cognitive Science, 12, 423–466.

    Article  Google Scholar 

  • Wang, D., & Arbib, M. (1993). Timing and chunking in processing temporal order. IEEE Transactions on Systems, Man, and Cybernetics, 23, 993–1009.

    Article  Google Scholar 

  • Wang, D., & Yuwono, B. (1995). Anticipation based temporal pattern generation. IEEE Transactions on Systems, Man, and Cybernetics, 25, 615–628.

    Article  Google Scholar 

  • Wang, L., & Alkon, D. (1993). Temporal processing with a biologically based artificial network. In L. Wang & D. Alkon (Eds.), Artificial neural networks: Oscillations, chaos, and sequence processing. Los Alamitos, CA: IEEE Computer Society Press.

    Google Scholar 

  • Widrow, B., & Hoff, M. (1960): Adaptive switching circuits. In 1960 IRE WESCON Convention Record. New York: IRE.

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics and Computer Science, Samford University, Birmingham, AL, USA

    Steve Donaldson

Authors
  1. Steve Donaldson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Steve Donaldson.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Donaldson, S. A Neural Network for Creative Serial Order Cognitive Behavior. Minds & Machines 18, 53–91 (2008). https://doi.org/10.1007/s11023-007-9085-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11023-007-9085-z

Keywords

Search

Navigation