The success of deep learning in natural language processing raises intriguing questions about the nature of linguistic meaning and ways in which it can be processed by natural and artificial systems. One such question has to do with subword segmentation algorithms widely employed in language modeling, machine translation, and other tasks since 2016. These algorithms often cut words into semantically opaque pieces, such as ‘period’, ‘on’, ‘t’, and ‘ist’ in ‘period|on|t|ist’. The system then represents the resulting segments in a dense (...) vector space, which is expected to model grammatical relations among them. This representation may in turn be used to map ‘period|on|t|ist’ (English) to ‘par|od|ont|iste’ (French). Thus, instead of being modeled at the lexical level, translation is reformulated more generally as the task of learning the best bilingual mapping between the sequences of subword segments of two languages; and sometimes even between pure character sequences: ‘p|e|r|i|o|d|o|n|t|i|s|t’ → ‘p|a|r|o|d|o|n|t|i|s|t|e’. Such subword segmentations and alignments are at work in highly efficient end-to-end machine translation systems, despite their allegedly opaque nature. The computational value of such processes is unquestionable. But do they have any linguistic or philosophical plausibility? I attempt to cast light on this question by reviewing the relevant details of the subword segmentation algorithms and by relating them to important philosophical and linguistic debates, in the spirit of making artificial intelligence more transparent and explainable. (shrink)

The standard approach to model how human beings understand natural languages is the symbolic, compositional approach according to which the meaning of a complex expression is a function of the meanings of its constituents. In other words, meaning plays a fundamental role in the model. In this work, because of the polysemous, flexible, dynamic, and contextual structure of natural languages, this approach is rejected. Instead, a connectionist model which eliminates the concept of meaning is proposed.

The goal of inquiry in this essay is to ascertain to what extent the Principle of Compositionality – the thesis that the meaning of a complex expression is determined by the meaning of its parts and its mode of composition – can be justifiably imposed as a constraint on semantic theories, and thereby provide information about what meanings are. Apart from the introduction and the concluding chapter the thesis is divided into five chapters addressing different questions pertaining to the overarching (...) goal of inquiry. Chapter Two is an attempt to determine whether the Principle of Compositionality is a trivial principle. It is argued that this is not the case in the context of providing the semantics of natural languages since it is an open question whether the best theories that respect available syntactic and semantic data are compositional. Chapter Three and Chapter Four ask whether there are reasons to think that the Principle of Compositionality is true. It is argued that the three most commonly cited reasons for thinking that correct semantic theories must be compositional – Linguistic Creativity, Productivity and Systematicity – in fact only give us reasons to think that the meaning of complex expressions depend on the meanings of their parts and their modes of composition, but not that the meaning of complex expressions depend ONLY on those factors. Chapter Five asks whether there are any reasons to think that the Principle of Compositionality is false. It is argued that all the phenomena that have been suggested entail non-compositionality are in fact compatible with some versions of compositionality. Even if this conclusion is reached in this chapter, the conclusion of the previous two chapters entails that we are not justified in imposing the Principle of Compositionality as an adequacy constraint on Semantic theories. Chapter Six explores the ways in which information about what meanings are can be derived by imposing constraints on semantic theories. It is argued that the distinction emphasized in Chapter Three and Chapter Four between depending on and depending ONLY on is of critical importance. It is demonstrated that the informative potential of the Principle of Compositionality goes far beyond that of the principle we are in fact justified in imposing on semantic theories. So not only is it the case that we cannot learn anything about meanings from the justified imposition of the Principle of Compositionality – since we cannot justifiably impose it as a constraint on semantic theories – what we could have learned from it had we been justified in imposing it is very different from what we can learn from the principle that we are justified in imposing. (shrink)

When reasons are given for compositionality, the arguments usually purport to establish compositionality in an almost a priori manner. I will rehearse these arguments why one could think that compositionality is a priori true, or almost a priori true, and will find all of them inconclusive. This, in itself, is no reason against compositionality, but a reason to try to establish or defend the principle on other than quasi-a priori grounds.