Skip to main content
Log in

Neural networks discover a near-identity relation to distinguish simple syntactic forms

  • Published:
Minds and Machines Aims and scope Submit manuscript

Abstract

Computer simulations show that an unstructured neural-network model [Shultz, T. R., & Bale, A. C. (2001). Infancy, 2, 501–536] covers the essential features␣of infant learning of simple grammars in an artificial language [Marcus, G. F., Vijayan, S., Bandi Rao, S., & Vishton, P. M. (1999). Science, 283, 77–80], and generalizes to examples both outside and inside of the range of training sentences. Knowledge-representation analyses confirm that these networks discover that duplicate words in the sentences are nearly identical and that they use this near-identity relation to distinguish sentences that are consistent or inconsistent with a familiar grammar. Recent simulations that were claimed to show that this model did not really learn these grammars [Vilcu, M., & Hadley, R. F. (2005). Minds and Machines, 15, 359–382] confounded syntactic types with speech sounds and did not perform standard statistical tests of results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Anderson, J. R. (1993). Rules of the mind. Hillsdale, NJ: Erlbaum

    Google Scholar 

  • Baluja, S., & Fahlman, S. E. (1994). Reducing network depth in the cascade-correlation learning architecture. Technical Report CMU-CS-94–209, School of Computer Science, Carnegie Mellon University

  • Campbell, D. T., & Stanley, J. C. (1963). Experimental and quasi-experimental designs for research. Chicago: Rand McNally

    Google Scholar 

  • Christiansen, M. H., & Curtin, S. L. (1999). The power of statistical learning: No need for algebraic rules. Proceedings of the twenty-first annual conference of the Cognitive Science Society (pp. 114–119). Mahwah, NJ: Erlbaum

  • Fahlman, S. E., & Lebiere, C. (1990). The cascade-correlation learning architecture. In D. S. Touretzky (Ed.), Advances in neural information processing systems (Vol. 2, pp. 524–532). Los Altos, CA: Morgan Kaufmann

    Google Scholar 

  • Gómez, R. L., & Gerken, L. (2000). Infant artificial language learning and language acquisition. Trends in Cognitive Sciences, 4, 178–186

    Article  Google Scholar 

  • Jolliffe, I. T. (1986). Principle component analysis. Berlin: Springer Verlag

    Google Scholar 

  • Kaye, J. (1989). Phonology: A cognitive view. Hillsdale, NJ: Erlbaum

    Google Scholar 

  • Marcus, G. F. (2001). The algebraic mind. Cambridge, MA: MIT Press

    Google Scholar 

  • Marcus, G. F., Vijayan, S., Bandi Rao, S., & Vishton, P. M. (1999). Rule learning by seven-month-old infants. Science, 283, 77–80

    Article  Google Scholar 

  • Mehler, J., Dommergues, J., Frauenfelder, U., & Segui, J. (1981). The syllable’s role in speech segmentation. Journal of Verbal Learning and Verbal Behavior, 20, 298–305

    Article  Google Scholar 

  • Negishi, M. (1999). Do Infants learn grammar with algebra or statistics? Science, 284, 433

    Google Scholar 

  • Pinker, S. (1997). How the mind works. New York: Norton

    Google Scholar 

  • Pinker, S. (1999). Out of the minds of babes. Science, 283, 40–41

    Article  Google Scholar 

  • Quinlan, J. R. (1993). C4.5: Programs for machine learning. San Mateo, CA: Morgan Kaufmann

    Google Scholar 

  • Rivest, F., & Shultz, T. R. (2005). Learning with both adequate computational power and biological realism. Proceedings of the 2005 Canadian artificial intelligence conference: Workshop on correlation learning (pp. 15–23). Victoria, BC: University of Victoria

  • Saffran, J. R., Aslin, R. N., & Newport, E. L. (1996). Statistical learning by 8-month-old-infants. Science, 274, 1926–1928

    Article  Google Scholar 

  • Sanger, D. (1989). Contribution analysis: A technique for assigning responsibilities to hidden units in connectionist networks. Connection Science, 1, 115–138

    Google Scholar 

  • Selkirk, E. O. (1984). On the major class features and syllable theory. In M. Aronoff & R. T. Oehrle (Eds.), Language sound structure (pp. 107–136). Cambridge, MA: MIT Press

    Google Scholar 

  • Shultz, T. R. (1999). Rule learning by habituation can be simulated in neural networks. In Proceedings of the twenty-first annual conference of the Cognitive Science Society (pp. 665–670). Mahwah, NJ: Erlbaum

  • Shultz, T. R. (2001), Assessing generalization in connectionist and rule-based models under the learning constraint. In Proceedings of the twenty-third annual conference of the Cognitive Science Society (pp. 922–927). Mahwah, NJ: Erlbaum

  • Shultz, T. R. (2003). Computational developmental psychology. Cambridge, MA: MIT Press

    Google Scholar 

  • Shultz, T. R. (2006). Constructive learning in the modeling of psychological development. In Y. Munakata & M. H. Johnson (Eds.), Processes of change in brain and cognitive development: Attention and performance XXI (pp. 61–86). Oxford: Oxford University Press

    Google Scholar 

  • Shultz, T. R., & Bale, A. C. (2000). Infant familiarization to artificial sentences: Rule-like behavior without explicit rules and variables. In L. R. Gleitman & A. K. Joshi (Eds.), Proceedings of the twenty-second annual conference of the Cognitive Science Society (pp. 459–463). Mahwah, NJ: Erlbaum

  • Shultz, T. R., & Bale, A. C. (2001). Neural network simulation of infant familiarization to artificial sentences: Rule-like behavior without explicit rules and variables. Infancy, 2, 501–536

    Article  Google Scholar 

  • Shultz, T. R., Mysore, S. P., & Quartz, S. R. (2006, in press). Why let networks grow? In D. Mareschal, S. Sirois, & G. Westermann (Eds.), Constructing cognition: Perspectives and prospects. Oxford: Oxford University Press

  • Shultz, T. R., Oshima-Takane, Y., & Takane, Y. (1995). Analysis of unstandardized contributions in cross connected networks. In D. Touretzky, G. Tesauro, & T. K. Leen (Eds.), Advances in neural information processing systems (Vol. 7, pp. 601–608). Cambridge, MA: MIT Press

    Google Scholar 

  • Sirois, S., Buckingham, D., & Shultz, T. R. (2000). Artificial grammar learning by infants: An auto-associator perspective. Developmental Science, 4, 442–456

    Article  Google Scholar 

  • Vilcu, M., & Hadley, R. F. (2001). Generalization in simple recurrent networks. In Proceedings of the twenty-third annual conference of the Cognitive Science Society (pp. 1072–1077). Mahwah, NJ: Erlbaum

  • Vilcu, M., & Hadley, R. F. (2003). Two apparent “counterexamples” to Marcus: A closer look. In Proceedings of the twenty-fifth annual conference of the Cognitive Science Society (pp. 1188–1193). Mahwah, NJ: Erlbaum

  • Vilcu, M., & Hadley, R. F. (2005). Two apparent “counterexamples” to Marcus: A closer look. Minds and Machines, 15, 359–382

    Article  Google Scholar 

  • Vroomen, J., & de Gelder, B. (1997). Activation of embedded words in spoken word recognition. Journal of Experimental Psychology, 23, 710–720

    Google Scholar 

  • Vroomen, J., van den Bosch, A., & de Gelder, B. (1998). A connectionist model for bootstrap learning of syllabic structure. Language and Cognitive Processes, 13, 193–220

    Article  Google Scholar 

  • Winer, B. J. (1962). Statistical principles in experimental design. New York: McGraw-Hill

    Google Scholar 

Download references

Acknowledgments

This work was supported by a grant from the Natural Sciences and Engineering Research Council of Canada to the first author.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Thomas R. Shultz.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Shultz, T.R., Bale, A.C. Neural networks discover a near-identity relation to distinguish simple syntactic forms. Minds & Machines 16, 107–139 (2006). https://doi.org/10.1007/s11023-006-9029-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11023-006-9029-z

Keywords

Navigation