Abstract
PDP networks that use nonmonotonic activation functions often produce hidden unit regularities that permit the internal structure of these networks to be interpreted (Berkeley et al., 1995; McCaughan, 1997; Dawson, 1998). In particular, when the responses of hidden units to a set of patterns are graphed using jittered density plots, these plots organize themselves into a set of discrete stripes or bands. In some cases, each band is associated with a local interpretation. On the basis of these observations, Berkeley (2000) has suggested that these bands are both subsymbolic and symbolic in nature, and has used the analysis of one network to support the claim that there are fewer differences between symbols and subsymbols than one might expect. We suggest below that this conclusion is premature. First, in many cases the local interpretation of each band is difficult to relate to the interpretation of a network's response; a more appropriate relationship only emerges when a band associated with one hidden unit is considered in the context of other bands associated with other hidden units (i.e., interpretations of distributed representations are more useful than interpretations of local representations). Second, the content that a band designates to an external observer (i.e., the interpretation assigned to a band by the researcher) can be quite different from the content that a band designates to the output units of the network itself.. We use two different network simulations – including the one described by Berkeley (2000) – to illustrate these points. We conclude that current evidence involving interpretations of nonmonotonic PDP networks actually illustrates the differences between symbolic and subsymbolic processing.
Similar content being viewed by others
References
Bechtel, W. and Abrahamsen, A. (1991), Connectionism and the mind. Cambridge, MA: Basil Blackwell.
Berkeley, I.S.N. (2000), What the #$*%! is a subsymbol?. Minds and Machines, 10, pp. 1–14.
Berkeley, I.S.N., Dawson, M.R.W., Medler, D.A., Schopflocher, D.P. and Hornsby, L., (1995). Density plots of hidden value unit activations reveal interpretable bands. Connection Science 7 pp. 167–186.
Chambers, J.M., Cleveland, W.S., Kleiner, B. and Tukey, P.A. (1983). Graphic methods for data analysis. Belmont, CA: Wadsworth International Group.
Churchland, P.S. and Sejnowski, T.J. (1992), The computational brain. Cambridge, MA: MIT Press.
Clark, A. (1993). Associative engines. Cambridge, MA: MIT Press.
Dawson, M.R.W. (1998). Understanding Cognitive Science. Oxford, UK: Blackwell.
Dawson, M.R.W., Medler, D.A. and Berkeley, I.S.N. (1997), PDP networks can provide models that are not mere implementations of classical theories. Philosophical Psychology, 10, 25–40.
Dawson, M.R.W., Medler, D.A., McCaughan, D.B., Willson, L. and Carbonaro, M. (2000), Using extra output learning to insert a symbolic theory into a connectionist network. Minds And Machines, 10, pp. 171–201.
Dawson, M.R.W. and Schopflocher, D.P. (1992), Modifying the generalized delta rule to train networks of nonmonotonic processors for pattern classification. Connection Science 4, pp. 19–31.
Greeno, J.G. and Moore, J.L. (1993). Situativity and symbols: Response to Vera and Simon. Cognitive Science, 17, 49–59.
Hinton, G.E. (1986). Learning distributed representations of concepts. Paper presented at the The 8th Annual Meeting of the Cognitive Science Society, Ann Arbor, MI.
Inhelder, B. and Piaget, J. (1958). The Growth Of Logical Thinking From Childhood To Adolescence. New York, NY: Basic Books.
Leighton, J.P. (1999), An alternate approach to understanding formal reasoning: Thinking according to the inductive-coherence model. Unpublished Ph.D., University of Alberta, Edmonton.
McCaughan, D.B. (1997, June 9–12). On the properties of periodic perceptrons. Paper presented at the IEEE/INNS International Conference on Neural Networks (ICNN'97), Houston, TX.
Newell, A. (1980). Physical symbol systems. Cognitive Science, 4, 135–183.
Smolensky, P. (1988). On the proper treatment of connectionism. Behavioural and Brain Sciences 11, pp. 1–74.
Touretzky, D.S. and Pomerleau, D.A. (1994), Reconstructing physical symbol systems. Cognitive Science. 18, pp. 345–353.
Vera, A.H. and Simon, H.A. (1993), Situated action: A symbolic interpretation. Cognitive Science 17, pp. 7–48.
Vera, A.H. and Simon, H.A. (1994). Reply to Touretzky and Pomerlau: Reconstructing physical symbol systems. Cognitive Science, 18, pp. 355–360.
Wason, P.C. (1966). Reasoning. New York: Penguin.
Zimmerman, C.L. (1999), A network interpretation approach to the balance scale task. Unpublished Ph.D., University of Alberta, Edmonton.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Dawson, M.R., Piercey, C.D. On the Subsymbolic Nature of a PDP Architecture that Uses a Nonmonotonic Activation Function. Minds and Machines 11, 197–218 (2001). https://doi.org/10.1023/A:1011237306312
Issue Date:
DOI: https://doi.org/10.1023/A:1011237306312