Variable binding has long been a challenge to connectionists. Attempts to perform variable binding using localist and distributed connectionist representations are discussed, and problems inherent in each type of representation are outlined.
Learners are able to use 2 different types of knowledge to perform a skill. One type is a conscious mental model, and the other is based on memories of instances. The authors conducted 3 experiments that manipulated training conditions designed to affect the availability of 1 or both types of knowledge about an artificial grammar. Participants were tested for both speed and accuracy of their ability to generate letter sequences. Results indicate that model-based training leads to slow accurate responding. Memorybased (...) training leads to fast, less accurate responding and highest achievement when perfect accuracy was not required. Evidence supports participants’ preference for using the memory-based mode when exposed to both types of training. Finally, the accuracy contributed by model-based training declined over a retention interval. (shrink)
Implicit processes are thought to be relatively fast, inaccessible, holistic, and imprecise, while explicit processes are slow, accessible and precise (e.g., Reber, 1989, Sun 2002). This dichotomy is closely related to some other well-known dichotomies including symbolic versus subsymbolic processing (Rumelhart et al., 1986), conceptual versus subconceptual processing (Smolensky, 1988), and conscious versus unconscious processing (Jacoby et al., 1994). This dichotomy has been justified by extensive studies of implicit and explicit learning, implicit and explicit memory, and implicit versus explicit metacognition (...) (Reder, 1996). (shrink)
This paper introduces a hybrid model that combines connectionist, symbolic, and reinforcement learning for tackling reactive sequential decision tasks by a situated agent. Both procedural skills and high-level symbolic representations are acquired through an agent's experience interacting with the world, in a bottom-up direction. It deals with on-line learning, that is, learning continuously from on-going experience in the world, without the use of preconstructed data sets or preconceived concepts. The model is a connectionist one based on a two-level approach proposed (...) earlier. Acknowledgements: This work is supported in part by O ce of Naval Research grant N00014-95-1-0440. (shrink)
The models used in social simulation to date have mostly been very simplistic cognitively, with little attention paid to the details of individual cognition. This work proposes a more cognitively realistic approach to social simulation. It begins with a model created by Gilbert (1997) for capturing the growth of academic science. Gilbert’s model, which was equation-based, is replaced here by an agent-based model, with the cognitive architecture CLARION providing greater cognitive realism. Using this cognitive agent model, results comparable to previous (...) simulations and to human data are obtained. It is found that while diﬀerent cognitive settings may aﬀect the aggregate number of scientiﬁc articles produced, they do not generally lead to diﬀerent distributions of number of articles per author. The paper concludes with a discussion of the correspondence between our model and the constructivist view of academic science. It is argued that using more cognitively realistic models in simulations may lead to novel insights. (shrink)
This paper addresses automatic partitioning in complex reinforcement learning tasks with multiple agents, without a priori domain knowledge regarding task structures. Partitioning a state/input space into multiple regions helps to exploit the di erential characteristics of regions and di erential characteristics of agents, thus facilitating learning and reducing the complexity of agents especially when function approximators are used. We develop a method for optimizing the partitioning of the space through experience without the use of a priori domain knowledge. The method (...) is experimentally tested and compared to a number of other algorithms. As expected, we found that the multi-agent method with automatic partitioning outperformed single-agent learning. (shrink)
�� This work describes a cognitively realistic ap- proach to social simulation. It begins with a model created by Gilbert  for capturing the growth of academic science. Gilbert’s model, which was equation-based, is replaced here by an agent-based (neural network) model, with the (neural net- work based) cognitive architecture CLARION providing greater cognitive realism. Using this agent model, results comparable to previous simulations and to human data are obtained. It is found that while different cognitive settings may affect the (...) aggregate number of scientiﬁc articles produced by the model, they do not generally lead to different distributions of number of articles per author. It is argued that using more cognitively realistic models in simulations may lead to novel insights. (shrink)
During the two days of the workshop, various presentations and discussions brought to light many new ideas, controv ersies, and syntheses. The fo cus was on learning and architecture s that feature hybrid representations and supp ort hybrid learning. It was a general consensus among the workshop participants that hybrid connectionist-symb olic mo dels constitute a promising aven ue toward developing more robust, more p owerful, and more versatile architecture s b oth for cognitive mo deling and for intelligen t (...) systems. The need for such mo dels has b een slowly but steadily growing over the past 5 years. Some new, imp ortant approaches have b een prop osed and develop ed, some of which were presented at the workshop. worthwhil e further pursuing In sum. (shrink)
What is computational cognitive modeling? What exactly can it contribute to cognitive science? What has it contributed thus far? Where is it going? Answering such questions may sound overly defensive to the insiders of computational cognitive modeling, and may even seem so to some other cognitive scientists, but they are very much needed in a volume like this—because they lie at the very foundation of this field. Many insiders and outsiders alike would like to take a balanced and rational look (...) at these questions, without indulging in excessive cheer-leading, which, as one would expect, happens sometimes amongst computational modeling enthusiasts. (shrink)
This pap er advo cates a microfeature based approach towards developing computational mo dels for metaphor interpretation. It is argued that the existing mo dels based on semantic networks and mappings of complex symb olic structures are insucient and inappropriate for mo deling metaphors.
This paper discusses essential motivational representations necessary for a comprehensive computational cognitive architecture. It hypothesizes the need for implicit drive representations, as well as explicit goal representations. Drive representations consist of primary drives — both low-level primary drives (concerned mostly with basic physiological needs) and high-level primary drives (concerned more with social needs), as well as derived (secondary) drives. On the basis of drives, explicit goals may be generated on the ﬂy during an agent’s interaction with various situations. These motivational (...) representations help to make cognitive architectural models more comprehensive and provide deeper explanations of psychological processes. This work represents a step forward in making computational cognitive architectures better reﬂections of the human mind and all its motivational complexity and intricacy. (shrink)
Straightforward reinforcement learning for multi-agent co-learning settings often results in poor outcomes. Meta-learning processes beyond straightforward reinforcement learning may be necessary to achieve good (or optimal) outcomes. Algorithmic processes of meta-learning, or "manipulation", will be described, which is a cognitively realistic and effective means for learning cooperation. We will discuss various "manipulation" routines that address the issue of improving multi-agent co-learning. We hope to develop better adaptive means of multi-agent cooperation, without requiring a priori knowledge, and advance multi-agent co-learning beyond (...) existing theories and techniques. (shrink)
Perhaps the most significant feature of current artificial intelligence research is the co-existence of a number of vastly different and often seriously conflicting paradigms, competing for the attention of the research community (as well as research funding). In this article, two competing paradigms of artificial intelligence, the connectionist and the symbolic approach, will be described. Brief analysis and criticism of each paradigm will be provided, and possible integration of the two will also be discussed as a result of the analysis (...) on their respective shortcomings. (shrink)
top-down approach (that is, turning declarative knowledge into procedural knowledge), we adopt a bottom-up approach toward lowlevel skill learning, where procedural knowledge develops rst and declarative knowledge develops from it. Clarionwhich follows this approach is formed by integrating connectionist, reinforcement, and symbolic learning methods to perform on-line learning. We compare the model with human data in a mine eld navigation task. A match between the model and human data is observed in several comparisons.
Research in computational cognitive modeling investigates the nature of cognition through developing process-based understanding by specifying computational models of mechanisms (including representations) and processes. In this enterprise, a cognitive architecture is a domaingeneric computational cognitive model that may be used for a broad, multiple-level, multipledomain analysis of behavior. It embodies generic descriptions of cognition in computer algorithms and programs. Developing cognitive architectures is a difficult but important task. In this article, discussions of issues and challenges in developing cognitive architectures will (...) be undertaken, and an example cognitive architecture (CLARION) will be described. (shrink)
In developing autonomous agents, one usually emphasizes only (situated) procedural knowledge, ignoring more explicit declarative knowledge. On the other hand, in developing symbolic reasoning models, one usually emphasizes only declarative knowledge, ignoring procedural knowledge. In contrast, we have developed a learning model Clarion, which is a hybrid connectionist model consisting of both localist and distributed representations, based on the two-level approach proposed in Sun (1995). learns and utilizes both procedural and declarative knowledge, tapping into the synergy of..
This paper addresses weighting and partitioning in complex reinforcement learning tasks, with the aim of facilitating learning. The paper presents some ideas regarding weighting of multiple agents and extends them into partitioning an input/state space into multiple regions with di erential weighting in these regions, to exploit di erential characteristics of regions and di erential characteristics of agents to reduce the learning complexity of agents (and their function approximators) and thus to facilitate the learning overall. It analyzes, in reinforcement learning (...) tasks, di erent ways of partitioning a task and using agents selectively based on partitioning. Based on the analysis, some heuristic methods are described and experimentally tested. We nd that some o -line heuristic methods performed the best, signi cantly better than single-agent models. (shrink)
The models used in social simulation to date have mostly been very simplistic cognitively, with little attention paid to the details of individual cognition. This work proposes a more cognitively realistic approach to social simulation. It begins with a model created by Gilbert (1997) for capturing the growth of academic science. Gilbert’s model, which was equation-based, is replaced here by an agent-based model, with the cognitive architecture CLARION providing greater cognitive realism. Using this cognitive agent model, results comparable to previous (...) simulations and to human data are obtained. It is found that while different cognitive settings may affect the aggregate number of scientific articles produced, they do not generally lead to different distributions of number of articles per author. The paper concludes with a discussion of the correspondence between our model and the constructivist view of academic science. It is argued that using more cognitively realistic models in simulations may lead to novel insights. (shrink)
Abstra,ct— This paper will discuss learning in hybrid models that goes beyond simple rule extraction from backpropagation networks. Although simple rule extraction has received a lot of research attention, to further develop hybrid learning models that include both symbolic and subsymbolic knowledge and that learn autonomously, it is necessary to study autonomous learning of both subsymbolic and symbolic knowledge in integrated architectures. This paper will describe knowledge extraction from neural reinforcement learning. It includes two approaches towards extracting plan knowledge: the (...) extraction of explicit, symbolic rules from neural reinforcement learning, and the extraction of complete plans. This work points to the creation of a general framework for achieving the subsymbolic to symbolic transition in an integrated autonomous learning framework. (shrink)
As we know, a cognitive architecture is a domain-generic computational cognitive model that may be used for a broad analysis of cognition and behavior. Cognitive architectures embody theories of cognition in computer algorithms and programs. Social simulation with multi-agent systems can benefit from incorporating cognitive architectures, as they provide a realistic basis for modeling individual agents (as argued in Sun 2001). In this survey, an example cognitive architecture will be given, and its application to social simulation will be sketched.
Computational cognitive models hypothesize internal mental processes of human cognitive activities and express such activities by computer programs Such computational models often consist of many components and aspects Claims are often made that certain aspects of the models play a key role in modeling but such claims are sometimes not well justi ed or explored In this paper we rst review some fundamental distinctions and issues in computational modeling We then discuss in principle systematic ways of identifying the source of (...) power in the models.. (shrink)
The performance of symbolic inference tasks has long been a challenge to connectionists. In this paper, we present an extended survey of this area. Existing connectionist inference systems are reviewed, with particular reference to how they perform variable binding and rule- based reasoning and whether they involve distributed or localist representations. The bene®ts and disadvantages of different representations and systems are outlined, and conclusions drawn regarding the capabilities of connectionist inference systems when compared with symbolic inference systems or when used (...) for cognitive modelling. q 2001 Elsevier Science Ltd. All rights reserved. (shrink)
Agent-based social simulation (with multi-agent systems), which is an important aspect of social computing, can benefit from incorporating cognitive architectures, as they provide a realistic basis for modeling individual agents and therefore their social interactions. A cognitive architecture is a domain-generic computational cognitive model that may be used for a broad multiple-domain analysis of individual behavior. In this article, an example of a cognitive architecture is given, and its applications to social simulation described. Some challenging issues in this regard are (...) outlined. (shrink)
Synthesizing situated cognition, reinforcement learning, and hybrid connectionist modeling, a generic cognitive architecture focused on situated involvement and interaction with the world is developed in this book. The architecture notably incorporates the distinction of implicit and explicit processes.
forcement learning algorithms that generate only reactive policies and existing probabilistic planning algorithms that requires a substantial amount of a priori knowledge in order to plan we devise a two stage bottom up learning to plan process in which rst reinforcement learn ing dynamic programming is applied without the use of a priori domain speci c knowledge to acquire a reactive policy and then explicit plans are extracted from the learned reactive policy Plan extraction is based on a beam search (...) algorithm that performs temporal projection in a restricted fashion guided by the value functions resulting from reinforcement learn ing dynamic programming.. (shrink)
Co-learning of multiple agents has been studied in co-learning settings, and how do they help, or many different disciplines under various guises. For hamper, learning and cooperation? example, the issue has been tackled by distributed • How do we characterize the process and the artiﬁcial intelligence, parallel and distributed com- dynamics of co-learning, conceptually, mathe- puting, cognitive psychology, social psychology, matically, or computationally? game theory (and other areas of mathematical econ- • how do social structures and relations interact omics), sociology, (...) anthropology, and many other with co-learning of multiple agents? related disciplines. (shrink)
This article proposes a unified framework for understanding creative problem solving, namely, the explicit–implicit interaction theory. This new theory of creative problem solving constitutes an attempt at providing a more unified explanation of relevant phenomena (in part by reinterpreting/integrating various fragmentary existing theories of incubation and insight). The explicit–implicit interaction theory relies mainly on 5 basic principles, namely, (a) the coexistence of and the difference between explicit and implicit knowledge, (b) the simultaneous involvement of implicit and explicit processes in most (...) tasks, (c) the redundant representation of explicit and implicit knowledge, (d) the integration of the results of explicit and implicit processing, and (e) the iterative (and possibly bidirectional) processing. A computational implementation of the theory is developed based on the CLARION cognitive architecture and applied to the simulation of relevant human data. This work represents an initial step in the development of process-based theories of creativity encompassing incubation, insight, and various other related.. (shrink)
This paper presents a GA-based multi-agent reinforce- ment learning bidding approach (GMARLB) for perform- ing multi-agent reinforcement learning. GMARLB inte- grates reinforcement learning, bidding and genetic algo- rithms. The general idea of our multi-agent systems is as follows: There are a number of individual agents in a team, each agent of the team has two modules: Q module and CQ module. Each agent can select actions to be performed at each step, which are done by the Q module. While the (...) CQ module determines at each step whether the agent should continue or relinquish control. Once an agent relinquishes its control, a new agent is selected by bidding algorithms. We applied GA-based GMARLB to the Backgammon game. The experimental results show GMARLB can achieve a su- perior level of performance in game-playing, outperforming PubEval, while the system uses zero built-in knowledge. (shrink)
Sequential behavior is essential to intelligence, and it is a fundamental part of human activities ranging from reasoning to language, and from everyday skills to complex problem solving. In particular, sequence learning is an important component of learning in many task domains — planning, reasoning, robotics, natural language processing, speech recognition, adaptive control, time series prediction, financial engineering, DNA sequencing, and so on. Naturally, there are many different approaches towards sequence learning, resulting from different perspectives taken in different task domains. (...) These approaches deal with somewhat differently formulated sequential learning problems (for example, some with actions and some without), and/ or different aspects of sequence learning (for example, sequence prediction vs. sequence recognition). Sequence learning is clearly a difiicult task. More powerful algorithms for sequence learning are needed in all of these afore-mentioned domains. It is our view that the right approach to develop better techniques, algorithms, models. (shrink)
traction from reinforcement learners It addresses two ap proaches towards knowledge extraction the extraction of ex plicit symbolic rules from neural reinforcement learners and the extraction of complete plans from such learners The advantages of such knowledge extraction include the improvement of learning especially with the rule extraction approach and the improvement of the usability of re sults of learning..
Most psychological theories of problem solving have focused on modeling explicit processes that gradually bring the solver closer to the solution in a mostly explicit and deliberative way. This approach to problem solving is typically inefficient when the problem is too complex, ill-understood, or ambiguous. In such a case, a ‘creative’ approach to problem solving might be more appropriate. In the present paper, we propose a computational psychological model implementing the Explicit-Implicit Interaction theory of creative problem solving that involves integrating (...) the results of implicit and explicit processing. In this paper, the new model is used to simulate insight in creative problem solving and the overshadowing effect. (shrink)
Various forms of life have been existing on earth for hundreds of millions of years and the long history has seen the development of life from single cell organisms to invertebrates to vertebrates and to humans the truly intelli gent beings The biological organizations of various species from the lowest to the highest di er in their complexities and sizes Such di erences in internal complexity manifest in the di erences in overt behaviors and intelligence and generally speaking organizational complexities (...) of various species are propor tionate with capabilities displayed by respective species However a gap seems to exist when one goes from high vertebrate animals to humans in that a con scious rational capacity is readily available to human beings that does not seem to be present in any other animals no matter how high they are on the evolutionary hierarchy There is a qualitative di erence Yet strange enough there is no known qualitative di erence between the biological make up of hu man brains and animal brains So the questions are Where does the di erence lie What is the key to the emergence of rational thinking and intelligence.. (shrink)
methods to improve reinforcement learning are identi ed and discussed in some detail Each demonstrates to some extent the advantages of combining RL and symbolic meth ods These methods point to the potentials and the chal lenges of this line of research..
Most of the work in agent-based social simulation has assumed highly simplified agent models, with little attention being paid to the details of individual cognition. Here, in an effort to counteract that trend, we substitute a realistic cognitive agent model (CLARION) for the simpler models previously used in an organizational design task. On that basis, an exploration is made of the interaction between the cognitive parameters that govern individual agents, the placement of agents in different organizational structures, and the performance (...) of the organization. It is suggested that the two disciplines, cognitive modeling and social simulation, which have so far been pursued in relative isolation from each other, can be profitably integrated. (shrink)
chical reinforcement learning that does not rely on a pri ori hierarchical structures Thus the approach deals with a more di cult problem compared with existing work It in volves learning to segment sequences to create hierarchical structures based on reinforcement received during task ex ecution with di erent levels of control communicating with each other through sharing reinforcement estimates obtained by each others The algorithm segments sequences to re duce non Markovian temporal dependencies to facilitate the learning of the (...) overall task Initial experiments demon strated the basic promise of the approach.. (shrink)
This paper explicates the interaction between the implicit and explicit learning processes in skill acquisition, contrary to the common tendency in the literature of studying each type of learning in isolation. It highlights the interaction between the two types of processes and its various effects on learning, including the synergy effect. This work advocates an integrated model of skill learning that takes into account both implicit and explicit processes; moreover, it embodies a bottom-up approach (first learning implicit knowledge and then (...) explicit knowledge on its basis) towards skill learning. The paper shows that this approach accounts for various effects in the process control task data, in addition to accounting for other data reported elsewhere. (shrink)
Implicit processes are thought to be relatively fast, inaccessible, holistic, and imprecise, while explicit processes are slow, accessible and precise (e.g., Reber, 1989, Sun 2002). This dichotomy is closely related to some other wellknown dichotomies including symbolic versus subsymbolic processing (Rumelhart et al., 1986), conceptual versus subconceptual processing (Smolensky, 1988), and conscious versus unconscious processing (Jacoby et al., 1994). This dichotomy has been justified by extensive studies of implicit and explicit learning, implicit and explicit memory, and implicit versus explicit metacognition (...) (Reder, 1996). (shrink)
This article explores the view that computational models of cognition may constitute valid theories of cognition, often in the full sense of the term ‘‘theory”. In this discussion, this article examines various (existent or possible) positions on this issue and argues in favor of the view above. It also connects this issue with a number of other relevant issues, such as the general relationship between theory and data, the validation of models, and the practical benefits of computational modeling. All the (...) discussions point to the position that computational cognitive models can be true theories of cognition. Ó 2008 Elsevier B.V. All rights reserved. (shrink)
This paper explores the interaction between implicit and explicit processes during skill learning, in terms of top-down learning (that is, learning that goes from explicit to implicit knowledge) versus bottom-up learning (that is, learning that goes from implicit to explicit knowledge). Instead of studying each type of knowledge (implicit or explicit) in isolation, we stress the interaction between the two types, especially in terms of one type giving rise to the other, and its eﬀects on learning. The work presents an (...) integrated model of skill learning that takes into account both implicit and explicit processes and both top-down and bottom-up learning. We examine and simulate human data in the Tower of Hanoi task. The paper shows how the quantitative data in this task may be captured using either top-down or bottom-up approaches, although top-down learning is a more apt explanation of the human data currently available. These results illustrate the two diﬀerent directions of learning (top-down versus bottom-up), and thereby provide a new perspective on skill learning. Ó 2003 Elsevier B.V. All rights reserved. (shrink)
This paper explores cognitively realistic social simulations by deploying the CLARION cognitive architecture in a simple organizational simulation, which involves the interaction of multiple cognitive agents. It argues for an integration of the two separate strands of research: cognitive modeling and social simulation. Such an integration could, on the one hand, enhance the accuracy of social simulation models by taking into full account the effects of individual cognitive factors, and on the other hand, it could lead to greater explanatory, predictive, (...) and prescriptive power from these models. (shrink)
The present issue is the beginning of a new journal from various sub-disciplines and paradigms in order – Cognitive Systems Research – which we have to construct a coherent picture of how the various developed in response to what we perceive to be an pieces fit together overall. Such a synthesis is unfilled niche in the current literature in the areas of essential to the discovery of designs for general Cognitive Science and Artificial Intelligence.
To deal with reactive sequential decision tasks we present a learning model which is a hybrid connectionist model consisting of both localist and distributed representations based on the two level approach proposed in..
We present a skill learning model CLARION. Different from existing models of high-level skill learning that use a topdown approach (that is, turning declarative knowledge into procedural knowledge), we adopt a bottom-up approach toward low-level skill learning, where procedural knowledge develops first and declarative knowledge develops later. CLAR- ION is formed by integrating connectionist, reinforcement, and symbolic learning methods to perform on-line learning. We compare the model with human data in a minefield navigation task. A match between the model and (...) human data is found in several respects. (shrink)
This paper introduces a hybrid model that unifies connectionist, symbolic, and reinforcement learning into an integrated architecture for bottom-up skill learning in reactive sequential decision tasks. The model is designed for an agent to learn continuously from on-going experience in the world, without the use of preconceived concepts and knowledge. Both procedural skills and high-level knowledge are acquired through an agent’s experience interacting with the world. Computational experiments with the model in two domains are reported.
This work explores the importance of similarity-based processes in human everyday reasoning, beyond purely rule-based processes prevalent in AI and cognitive science. A unified framework encompassing both rulebased and similarity-based reasoning may provide explanations for a variety of human reasoning data.