Abstract
We argue that the pictorial nature of certain constructions in signs and in gestures explains surprising properties of their syntax. In several sign languages, the standard word order (e.g. SVO) gets turned into SOV (with preverbal arguments) when the predicate is a classifier, a distinguished construction with highly iconic properties (e.g. Pavlič, 2016). In silent gestures, participants also prefer an SOV order in extensional constructions, irrespective of the word order of the language they speak (Goldin-Meadow et al., 2008). But in silent gestures and in Brazilian Sign Language (Libras), intensional constructions can override these SOV preferences, yielding SVO instead (Schouwstra & de Swart, 2014; Napoli et al., 2017). This distinction was argued to be due to iconicity: arguments are expressed before the verb if they correspond to entities that are present before the action, otherwise they follow the verb. While agreeing with this intuition, we propose that the extensional/intensional distinction is neither empirically nor theoretically appropriate. In new data from American Sign Language, we replicate the distinction among extensional classifier predicates: for x ate up the ball, the ball is typically seen before the eating and a preposed object is preferred; but for x spit out the ball, the ball is typically seen after the spitting and a postposed object is preferred, although both eat up and spit out are used extensionally. We extend this finding to data involving pro-speech (= speech-replacing) gestures embedded in French sentences. We argue for a Visibility Generalization according to which arguments appear before the verb if their denotations are typically visible before the action, and we develop a new formal account within a pictorial semantics for visual animations (inspired by Greenberg and Abusch). It derives the observed word order preferences, it explains how the semantics of classifier predicates combines iconic and conventional properties, and it makes a more general point: sign language semantics combines logical semantics with pictorial semantics.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
This paragraph follows the discussion of Schlenker (2021).
See Section 20.2 of Sandler and Lillo-Martin (2006) for a discussion of other aspects of the syntax of classifier constructions.
Liddell explicitly sets aside the issue of topicalization, which is independently known to trigger word movement effects in ASL. See also Liddell's discussion of the relation between iconicity and SOV (without topicalization) on pp. 89–90 of Liddell (1980).
In the authors’ words, “in comparison to the production experiment”, in comprehension “the effect of word order on meaning in interpretation is modest”.
In Langus and Nespor's (2010) words: “SOV is the preferred constituent order in the direct interaction between the sensory-motor and the conceptual system; the SVO order is preferred by the computational system of grammar.”
In the words of Christensen et al. (2016), “structural iconicity is a particular type of iconicity in which the structure of events or relations between referents is replicated in the syntax of a spoken or signed utterance. It can be defined as a non-arbitrary, motivated relationship between form and meaning, which is established when the arrangement of individual signs mirrors actual properties of the relations between their referents, i.e. in transitive events.” As the authors note in relation to Schouwstra and de Swart (2014), “concerns have been raised (…) regarding whether construction events fit the general definition of ‘intensional’, since the patients in these events are both concrete and have ‘extensional’ properties (Parsons, 1990)”. This, in turn, may justify a generalization in terms of structural iconicity rather than intensionality.
We note that not all the situations described in (5) would lead one to expect an intensional behavior, at least according to (4): when one sees or hears a saxophone, there is a typically a certain saxophone that one sees or hears, unless one is under an illusion of some sort. Visually representing this as in (5)b(ii) (where the saxophone being heard is real) only strengthens the inference that there is a certain saxophone which the cook heard, and which was present in the scene before the action. And in fact, ‘hear’ and ‘see’ belong to the exceptions to the authors’ generalization that the object should follow the verb, as seen in the authors’ Table A2; this might be because the vignettes failed to elicit an intensional construction (and also one whose object would typically be seen after the action).
It should be added that Koraka (2021) (citing precedents in other sign languages such as BSL [Sutton-Spence & Woll, 1999]) found related results in Greek Sign Language: with predicates such as BUILD, MAKE and BAKE, SVO order was preferred (see also Christensen et al., 2016 for an SVO preference in sequences of gestures describing construction events). We note that these predicates are straightforward to analyze in terms of an iconic semantics as developed in this paper: the object of the creation is typically visible after the action and not before.
Without developing a formal account, they write: “Signing space may be likened to a canvas, with time as an added dimension. This iconicity is fundamental to understanding how sign languages work.”
We use the term consultant to refer to a collaborator that assesses sentences, including if this person is also a contributor to the article.
The two steps are discussed separately because they have a slightly different status, for two reasons. First, unlike the first step, the second had to be conducted long-distance and on Zoom, with the consultant recording himself separately to have high quality videos. Second, the first step was completed before the article was written and thus in the absence of any theoretical discussion, contrary to the second step.
Two remarks should be added.
First, as we briefly discuss below, our Coda ASL consultant prefers SOV order for a-LOOK-b, but not for a-HIT-b, hence partly disagreeing with our main consultant about a-LOOK-b.
Second, there is also a possible discrepancy between our generalization (based on our main ASL consultant) and the findings of Fischer (1975), who argues that when loci are used pre-verbal arguments are more natural. As she writes (p. 19) about the English sentence the girl kicked the boy, “the most straightforward way” of translating it “if the sentence occurs in isolation” (her emphasis) is by way of (i), which presumably involves a neutral locus:
(i) GIRL KICK BOY
But “if the sentence occurs in a more extended discourse, especially if one will wish to refer to the boy and/or the girl again”, the “most natural way” to sign the sentence would be with (ii)a, which “gives us an OSV structure”, although reversal of “the order of the subject and the object” is possible as in (ii)b; we take these to correspond to contemporary glosses as in (iii)a-b respectively.
(ii)
a. BOY (HERE)
GIRL (HERE)
SHE-KICK-HIM
(right hand)
(left hand)
left “kicks” right, from direction of location of girl to direction of location of boy
b. GIRL (HERE)
BOY (HERE)
SHE-KICK-HIM
(left hand)
(right hand)
left “kicks” right, from girl to boy)
(iii)
a. BOYa GIRLb
b-KICK-a
b. GIRLb BOYa
b-KICK-a
(An anonymous reviewer suggests, following Padden (1983), that (ii) might be analyzed as involving a multi-clausal structure, tantamount to: Boy is here. Girl is here. (she-) kicks (-him). If so, the agreement verb might form a separate clause that might not involve a pre-verbal object. The wording of Fischer's discussion indicates that she takes her examples to involve genuine OSV and SOV structure, however.)
The fact that the context was uniformly SVO might have created a small bias for SVO order in the target sentences, but this would likely have been the same bias for classifier predicates and plain verbs. Importantly, this could not explain the interpretive differences between the two cases.
There are precedents for the story in (13), not all of which are of dubious authenticity; see for instance the Old Testament, Book of Jonah; and the BBC's story entitled “Humpback whale gulps and spits out Cape Cod lobsterman” (retrieved on June 13, 2021 at https://www.bbc.com/news/world-us-canada-57450685).
The consultant forgot to explicitly select a ‘best’ sentence among (13)(ii)a, b, c in the session of 21/04/25, but since the acceptability ratings were respectively 6, 4, 5, it is clear that a was ‘best'.
The signer did not rotate his body or redirect his eyegaze, so this is not an instance of Role Shift.
In the second round of judgments, conducted by email (= [JL 22.01.30] in the raw data), the consultant answered that among d., e. and f., “a is best”. This is evidently a typo. JL took a further look at the original table and wrote by email on 22.02.14: “I'm sure I meant to type d rather than a.” In any event, none of the reported results changes whether one includes or excludes this particular answer.
Coda stands for child of deaf adult.
If anything, integrating gestural sequences to a full-fledged linguistic environment might be expected to strengthen an SVO bias. It is thus interesting that despite this pressure, we found contrasts across the eat-up- and spit-out-type constructions.
An anonymous reviewer asks why we do not have the same number of consultants for ASL as for gestures. The two cases are different along several dimensions: (i) ASL work is based on elicitation, with 3 judgment tasks for each data point, and numerous open questions; as can be seen in the Supplementary Materials, the information is just far richer than that provided by the gesture survey. We are thus confident that our methods are highly appropriate to investigate our consultant's idiolect. Our gesture survey with a single consultant would not be comparably appropriate to study the respondent's idiolect. (ii) Our ASL sentences are entirely standard, whereas our sequences of pro-speech gestures in French are unusual, which justified our caution in obtaining judgments from several consultants. (iii) Feasibility is different across the two cases: It was easy for us to find a significant number of native French speakers with prior experience with linguistic judgments; something comparable would have been much harder for us in ASL. In other words, there is a trade-off between depth and number: we have considerable depth in our ASL description, far less so in our gesture survey, but the latter has more consultants. Note that the independent assessment of our ASL data by two additional native signers functions as a ‘sanity check’ and gives us an idea of what might generalize to further idiolects.
As noted in Schlenker (2020), applying standard elicitation methods to sentences with gestures is justified. First, Sprouse and Almeida (2012, 2013) and Sprouse et al. (2013) argued for the general validity of introspective methods in standard linguistic judgments. Second, Tieu et al. (2017, 2018) largely confirmed with experimental means early semantic judgments on co-speech gestures that appeared in the literature (Schlenker, 2018c). Third, Tieu et al. (2019) did the same thing for semantic judgments on pro-speech gestures (Schlenker, 2019a, b), as did Schlenker and Chemla (2018) for more grammatical judgments.
Last column of b: average over 9 consultants only, as 1 consultant failed to enter an answer.
Last column of a: average over 9 consultants only, as 1 consultant failed to enter an answer.
The new consultants, all different from the earlier ones, had experience with linguistic judgments (and in this case, with linguistics). Some consultants for the second gesture survey were also consultants for the visual narrative survey of Sect. 10, conducted approximately two months earlier. One of the new consultants had seen a talk that included a discussion of earlier results of this paper more than 3 1/2 months before taking the survey.
Line c, last column: average over 10 consultants only, as 1 consultant failed to enter an answer.
In the framework of Cognitive Linguistics, temporal coherence is used as an iconic criterion to determine the order between main and subordinate adjunct clauses in spoken languages (Givón, 1993, Haiman, 1983; Li, 2016). In a sense, we are using related ideas to determine the order of predicates and arguments in highly iconic constructions in signs and gestures.
Two remarks should be added.
(i) We could replace … is true of time t… with … is true at time t…, treating t as an additional parameter with respect to which truth is relativized. Nothing hinges on this notational point; we treat times separately because we need to quantify over them when we analyze visual narratives.
(ii) Schlenker (2022) made use of eventualities to allow for a unification between pictorial semantics and music semantics, which in turn proved helpful to give a semantics for mixed sequences—such as cartoons accompanied with music. We are not concerned with the interaction between pictures and music in the present piece.
Schlenker (2022) discusses a similar example from Abusch, but involving more complex pictures.
We could further existentially quantify over viewpoints to obtain a definition of truth relative to a world alone.
We use the term “situations” informally here, to mean: whatever is denoted by the pictures.
Here it must be remembered that a is not in the same position in P1 (on the right) and in P2 (towards the center because the crocodile moved).
The requirement that t3 < t4 < t5 is already stated because <t3, t4, t5> is defined from the start as an ordered sequence.
In our informal paraphrase, we talk of the ball and the crocodile; the fact that these are the same ball and crocodile depicted by the earlier pictorial components is guaranteed by the assignment function.
As Paul Portner (p.c.) notes, our semantics predicts an important difference between a classifier predicate and a pro-speech gesture. Unless the latter is conventionalized, it should just mean that something that resembles the manual shape did something. By contrast, classifier predicates usually have a conventional lexical component and should thus imply that the denoted objects genuinely have certain properties—e.g. that something is a vehicle, or a whale, or a crocodile. While we believe this contrast to be real, we have not specifically tested it.
As things stand, our analysis does not derive the observation, discussed in Krebs et al. (2021), that with classifier predicates OSV order incurs a higher processing cost than SOV does.
For reasons we come to below, we can't exclude that there is also a way to treat classifier predicates as standard verbs, but this is not the preferred option for our main consultant.
Schlenker (2018b) concluded: “Now our controls do not literally involve a ‘like this’ modifier, but rather display the relevant path after the word WITH. One could of course test closer analogues of ‘like this’ in ASL. But in any event the presuppositional or cosupposition behavior of some classifier predicates is not expected under the current like this analysis.”
See Schlenker and Lamberton (2019) for a different case in which some discourse referents are created by iconic representations (at the edge of repetitions).
We have slightly adjusted the transcription conventions, putting loci introduced by expressions to their right rather than to their left. Schlenker (2013) wrote a+ and c+ to indicate that pointing is towards a position slightly higher than loci a and c (a notation we have preserved here). This might serve to distinguish between the person who is at the location—namely John—and the location itself. But cases of clear ambiguity are described in the literature (see Schlenker, 2018a for examples and references).
Two remarks should be added. First, ‘maximal iconicity’ is a narrow principle applying to classifier predicates, not something about sign languages in general. Second, the LIS case with SOV yielding the target ‘x spit out y’ meaning when the object is in a neutral locus raises two questions about ASL. The first is about spit-out-type classifier predicates: Does SOV also yield the target ‘x spit out y’ meaning when the object of a classifier predicate is signed in a neutral locus? Unless the object is moved (e.g. by topicalization), we would expect that the answer is ‘no’ because the basic word order of ASL is SVO, not SOV (in our account of LIS, it is the possibility of treating the classifier predicate as a normal verb that is responsible for the availability of SOV in this case). The second question is about eat-up-type classifier predicates: Does SVO order improve when the object is signed in a neutral locus? By analogy with the LIS case, we would expect that the answer is ‘yes': signing the object in a neutral locus should make it easier to obviate the need for an iconic composition between the classifier and its arguments. Our main consultant made a remark to that effect about (16)b, which does not yield the intended reading but “probably” would if the final PERSON were “signed in neutral space” ([JL 22.01.14]). We leave these issues for future research.
Presuppositions will have to be taken into account to develop a more adequate semantics. We conjecture that a-SWIM-EAT-b presupposes (rather than asserts) that a denotes an animal with a large mouth (because of the classifier shape), and that b denotes a large object (because of the size of the crocodile's open mouth). If so, the analysis would have to be developed within a trivalent account that can capture presuppositions. See also Schlenker (2019a) and Tieu et al. (2019) for an argument that iconic representations (including gestures and classifier predicates) productively trigger presuppositions.
A rule of ‘maximal iconicity’ as in (i) below was posited in the analysis of Role Shift in Schlenker (2017). But it pertained to the interpretation of certain structures, not to the competition between iconic and non-iconic structures (unlike the competition rule we tentatively posit in this paragraph).
(i) In ASL and LSF Action Role Shift, expressions that can be interpreted iconically must be so interpreted.
As mentioned in fn. 43, there is no claim whatsoever that sign language generally prefers iconic over non-iconic structures; ‘maximal iconicity’ is restricted to very special cases, involving classifier predicates and Role Shift in certain sign languages.
The picture is from Geluck's Le Chat, here modified from: https://www.pinterest.fr/pin/402227810477055597/, accessed on April 20, 2021.
There might well be more complex cases in which the prior representation of a thought or spoken word is salient enough that the thought can be depicted as existing before the action (e.g. for a thought or phrase which is repeated from individual to individual). We expect that in such cases gestural sequences and ASL classifier predicates allow for pre-verbal intensional objects. We leave a test of this prediction for future research.
Four further questions are left for future research.
(i) We mentioned in Sect. 8.2 that neutral loci make it possible to regain a non-iconic word order in LIS, and we mentioned in fn. 43 that a similar effect is not inconceivable in ASL. This requires further study.
(ii) We wrote as if classifier/gesture movement is interpreted fully iconically, but it might be that things are more complicated and that some movements are ‘unmarked’ and do not convey as precise iconic information as others.
(iii) On the gestural side, we wrote as if the gestures under study are fully iconic, but this is a simplification: first, some gestures are conventionalized or semi-conventionalized; second, in our target sentences we took the precaution of first introducing the gestures as co-speech elements, thus clarifying their meaning and possibly giving them a near-conventional meaning within the discourse. Conventionalized gestures might require refinements.
(iv) Once the possibility of λ-abstraction is granted, it could in principle apply to ASL spit-out-type classifier predicates to yield the target reading with pre-verbal objects (assuming these are moved by information-theoretic considerations). But our most robust finding across consultants and constructions is that preverbal objects do not yield the target reading in spit-out-type classifier constructions. This has yet to be explained.
In the cases we consider, the classifier predicate is not just conventional but also morphologically simple. Not all examples are of this type, however. For instance, in several sign languages two fingers can be used to represent two individuals, three to represent three, and these might be cases of number incorporation within the classifier. There might even be cases in which the classifier is itself the result of a syntactic operation. Our semantic analysis is neutral on this point, although it would have to take into account the semantics of these conventional forms—e.g. a classifier representing two individuals should of course have a different conventional specification than one representing one person, and the conventional part might be derived by compositional rules applying below or above the word level.
The car image is from https://www.cocktail-distribution.com/p/26617-citroen-dyane-6-bleue-118-3663506002267.html (retrieved 04/09/21).
References
Abusch, D. (2013). Applying discourse semantics and pragmatics to co-reference in picture sequences. In E. Chemla, V. Homer, G. Winterstein (Eds.), Proceedings of Sinn und Bedeutung (Vol. 17, pp. 9–25). https://ojs.ub.uni-konstanz.de/sub/index.php/sub/article/view/330
Abusch, D., & Rooth, M. (2017). The formal semantics of free perception in pictorial narratives. In A. Cremers, T. van Gessel & F. Roelofsen (Eds.), Proceedings of the 21st Amsterdam Colloquium (pp. 85–94). Amsterdam: ILLC.
Abusch, D. (2020). Possible worlds semantics for pictures. In L. Matthewson, C. Meier, H. Rullmann, & T. E. Zimmermann (Eds.), The Wiley Blackwell companion to semantics. New York: Wiley.
Aristodemo, V. (2017). Gradable constructions in Italian Sign Language. PhD dissertation, EHESS, Paris.
Cecchetto, C., Geraci, C., & Zucchi, S. (2006). Strategies of relativization in Italian Sign Language. Natural Language & Linguistic Theory, 24(4), 945–975.
Christensen, P., Fusaroli, R., & Tylén, K. (2016). Environmental constraints shaping constituent order in emerging communication systems: Structural iconicity, interactive alignment and conventionalization. Cognition, 146, 67–80.
Cogill-Koez, D. (2000). Signed language classifier predicates: Linguistic structures or schematic visual representation? Sign Language & Linguistics, 3(2), 153–207.
Conte, G., Santoro, M., Geraci, C., & Cardinaletti, A. (2010). Why are you raising your eyebrows? In LREC 2010 conference proceedings (pp. 53–56). Valetta, Malta: ELRA.
Davidson, K. (2015). Quotation, demonstration, and iconicity. Linguistics & Philosophy, 38(6), 477–520.
Emmorey, K., & Falgier, B. (2004). Conceptual locations and pronominal reference in American Sign Language. Journal of Psycholinguistic Research, 33(4), 321–331.
Emmorey, K., & Herzig, M. (2003). Categorical versus gradient properties of classifier constructions in ASL. In K. Emmorey (Ed.), Perspectives on classifier constructions in Signed Languages (pp. 222–246). Mahwah, NJ: Lawrence Erlbaum Associates.
Fischer, S. D. (1975). Influences on word order change in American Sign language. In C. N. Li (Ed.), Word order and word order change (pp. 1–25). Austin, TX: University of Texas Press.
Forbes, G. (2020). Intensional transitive verbs. In E. N. Zalta (Ed.), The Stanford Encyclopedia of Philosophy. https://plato.stanford.edu/entries/intensional-trans-verbs/.
Gibson, E., Piantadosi, S. T., Brink, K., Bergen, L., Lim, E., & Saxe, R. (2013). A noisy-channel account of crosslinguistic word order variation. Psychological Science. https://doi.org/10.1177/0956797612463705
Givón, T. (1993). English grammar: A Function-based introduction (Vol. 1). Amsterdam/ Philadelphia: John Benjamins.
Goldin-Meadow, S., So, W. C., Özyürek, A., & Mylander, C. (2008). The natural order of events: How speakers of different languages represent events nonverbally. Proceedings of the National Academy of Sciences, 105(27), 9163–9168.
Greenberg, G. (2014). Reference and predication in pictorial representation. Handout of a talk given at the London Aesthetics Forum (February 19, 2014).
Greenberg, G. (2019). Tagging: Semantics at the iconic/symbolic interface. In J. Schlöder, D. Mc Hugh, & F. Roelofsen (Eds.), Proceedings of the Amsterdam Colloquium 2019 (pp. 11–20). Amsterdam: ILLC.
Greenberg, G. (2013). Beyond resemblance. Philosophical Review, 122(2), 2013.
Greenberg, G. (2021). Semantics of pictorial space. Review of Philosophy and Psychology. https://doi.org/10.1007/s13164-020-00513-6
Haiman, J. (1983). Iconic and economic motivation. Language, 59, 781–819.
Hall, M. L., Mayberry, R. I., & Ferreira, V. S. (2013). Cognitive constraints on constituent order: Evidence from elicited pantomime. Cognition, 129(1), 1–17.
Hall, M. L., Mayberry, R. I., & Ferreira, V. S. (2015). Production and comprehension show divergent constituent order preferences: Evidence from elicited pantomime. Journal of Memory and Language, 81, 16–33.
Hendriks, B. (2007). Negation in Jordanian Sign Language: A cross-linguistic perspective. In P. Perniss, R. Pfau, & M. Steinbach (Eds.), Visible variation: Comparative studies on Sign Language Structure (pp. 103–128). Berlin: Mouton de Gruyter.
Kimmelman, V. (2012). Word order in Russian Sign Language. Linguistics in Amsterdam, 5(1), 1–56.
Koraka, M. (2021). On word order in Greek Sign Language. Poster, FEAST (Formal and Experimental Advances in Sign language Theory), Hongkong, June 2, 2021.
Krebs, J., Malaia, E. W., Ronnie, B., & Roehm, D. (2021). Psycholinguistic mechanisms of classifier processing in Sign Language. Journal of Experimental Psychology: Learning, Memory, and Cognition, 47(6), 998–1011.
Kuhn, J. (2017). Dependent indefinites: The view from sign language. Journal of Semantics, 34(3), 407–446. https://doi.org/10.1093/jos/ffx007
Kuhn, J. (2022). Disjunctive discourse referents in French Sign Language. Proceedings of SALT, 31 https://doi.org/10.3765/salt.v31i0.5116
Langus, A., & Nespor, M. (2010). Cognitive systems struggling for word order. Cognitive Psychology, 60(4), 291–318.
Li, X. (2016). The ordering distribution of temporal adverbial clauses of Chinese EFL learners and L1 conceptual transfer. Modern Foreign Languages, 5, 682–692.
Liddell, S. K. (1980). American Sign Language syntax. Hague: Mouton Publishers.
Liddell, S. K. (2003). Grammar, gesture, and meaning in American Sign Language. Cambridge: Cambridge University Press.
Meir, I., Lifshitz, A., Ilkbasaran, D., & Padden, C. A. (2010). The interaction of animacy and word order in human languages: A study of strategies in a novel communication task. Paper presented at the 8th International Conference on the Evolution of Language, Utrecht, The Netherlands.
Napoli, D. J., Spence, R. S., & de Quadros, R. M. (2017). Influence of predicate sense on word order in sign languages: Intensional and extensional verbs. Language, 93(3), 641–670. https://doi.org/10.1353/lan.2017.0039
Oviedo, A. (2003). Apuntes para una Gramática de la Lengua de Señas Colombiana. Cali: INSOR/Universidad del Valle.
Padden, C. (1983). Interaction of morphology and syntax in American Sign Language. Ph.D. dissertation, University of California, San Diego.
Parsons, T. (1990). Events in the Semantics of English. Cambridge, MA: MIT Press.
Patel-Grosz, P., Grosz, P. G., Kelkar, T., & Jensenius, A. R. (2022). Steps towards a Semantics of Dance. Journal of Semantics, 39(4), 693–748.
Pavlič, M. (2016). The word order parameter in Slovenian Sign Language transitive, ditransitive, classifier and locative constructions. PhD dissertation, University of Venice.
Sandler, W., & Lillo-Martin, D. (2006). Sign language and linguistic universals. Cambridge: Cambridge University Press.
Schlenker, P. (2013). Temporal and modal anaphora in sign language (ASL). Natural Language and Linguistic Theory, 31(1), 207–234.
Schlenker, P. (2017). Super monsters II: Role shift, iconicity and quotation in sign language. Semantics & Pragmatics 9:EA.
Schlenker, P. (2018a). Locative shift. Glossa: A journal of general linguistics, 3(1), 115. https://doi.org/10.5334/gjgl.561
Schlenker, P. (2018b). Sign language semantics: Problems and prospects [replies to peer commentaries]. Theoretical Linguistics, 44(3–4), 295–353.
Schlenker, P. (2018c). Gesture projection and cosuppositions. Linguistics & Philosophy, 41(3), 295–365.
Schlenker, P. (2019a). Gestural semantics: Replicating the typology of linguistic inferences with pro- and post-speech gestures. Natural Language & Linguistic Theory, 37(2), 735–784.
Schlenker, P. (2019b). What is super semantics? Philosophical Perspectives, 32(1), 365–453. https://doi.org/10.1111/phpe.12122
Schlenker, P. (2020). Gestural grammar. Natural Language & Linguistic Theory, 38, 887–936.
Schlenker, P. (2021). Iconic presuppositions. Natural Language & Linguistic Theory, 39, 215–289.
Schlenker, P. (2022). Musical meaning within super semantics. Linguistics & Philosophy, 45, 795–872.
Schlenker, P., & Chemla, E. (2018). Gestural agreement. Natural Language & Linguistic Theory, 36(2), 87–625587. https://doi.org/10.1007/s11049-017-9378-8
Schlenker, P., & Lamberton, J. (2019). Iconic plurality. Linguistics & Philosophy, 42, 45–108.
Schlenker, P., & Lamberton, J. (2022). Meaningful blurs: The sources of repetition-based plurals in ASL. Linguistics & Philosophy, 45, 201–264.
Schlenker, P., Lamberton, J., & Santoro, M. (2013). Iconic variables. Linguistics & Philosophy, 36(2), 91–149.
Schouwstra, M., & de Swart, H. (2014). The semantic origins of word order. Cognition, 131(3), 431–436.
Schouwstra, M., de Swart, H., & Thompson, B. (2019). Interpreting silent gesture: Cognitive biases and rational inference in emerging language systems. Cognitive Science, 43(7), e12732.
Sprouse, J., & Almeida, D. (2012). Assessing the reliability of textbook data in syntax: Adger’s Core Syntax. Journal of Linguistics, 48, 609–652.
Sprouse, J., & Almeida, D. (2013). The empirical status of data in syntax: A reply to Gibson and Fedorenko. Language and Cognitive Processes., 28, 222–228.
Sprouse, J., Schütze, C. T., & Almeida, D. (2013). A comparison of informal and formal acceptability judgments using a random sample from Linguistic Inquiry 2001–2010. Lingua, 134, 219–248.
Supalla, T. (1982) Structure and acquisition of verbs of motion and location in American Sign Language. PhD dissertation, University of California at San Diego.
Sutton-Spence, R., & Woll, B. (1999). The Linguistics of British Sign Language: An Introduction. Cambridge: Cambridge University Press.
Sze, F. Y. B. (2003). Word order of Hong Kong Sign Language. In A. E. Baker, B. van den Bogaerde, & O. A. Crasborn (Eds.), Cross-linguistic perspectives in sign language research. Selected papers from TISLR (pp. 163–192). Hamburg: Signum.
Tieu, L., Pasternak, R., Schlenker, P., Chemla, E. (2017). Co-speech gesture projection: Evidence from truth-value judgment and picture selection tasks. Glossa: A journal of general linguistics, 2(1):102. https://doi.org/10.5334/gjgl.334
Tieu, L., Pasternak, R., Schlenker, P., & Chemla, E. (2018). Co-speech gesture projection: Evidence from inferential judgments. Glossa: A journal of general linguistics, 3(1): 109. https://doi.org/10.5334/gjgl.580
Tieu, L., Schlenker, P., & Chemla, E. (2019). Linguistic inferences without words. Proceedings of the National Academy of Sciences, 116(20), 9796–9801.
Valli, C., & Lucas, C. (2000). Linguistics of American Sign Language: an Introduction (3rd ed.). Washington, DC: Gallaudet University Press.
Vermeerbergen, M. (2004). The quest for basic word order in Flemish Sign Language. In A.-M. Bertonneau, & G. Dal (Eds.), La linguistique de la LSF: recherches actuelles (Silexicales, Vol. 4) (pp. 257–267) Villeneuve d’Ascq: Université de Lille 3.
Zucchi, S. (2011). Event descriptions and classifier predicates in sign languages. Presentation at FEAST (Formal and Experimental Advances in Sign language Theory, Venice, June 21, 2011.
Zucchi, S. (2012). Formal semantics of sign languages. Language and Linguistics Compass, 6(11), 719–734.
Zucchi, S. (2017). Event categorization in sign languages. In H. Cohen & C. Lefebvre (Eds.), The handbook of categorization in cognitive science (pp. 377–396). Amsterdam: Elsevier.
Acknowledgments
For help with the ASL data, we are extremely grateful to our Coda consultant. We also thank Jeremy Kuhn for helpful references. For discussion and advice on the gesture survey, we are grateful to Nadine Bade, Janek Guerrini, Léo Migotti, Luigi Rizzi and Anastasia Tsilia. Special thanks to Lucie Ravaux for helping with the references, Excel sheets, videos, and proofs. We are also grateful to Susanne Trissler for her wonderful copyediting work, and for her extraordinary patience and attention to detail.
Acknowledgment of the participants to the gesture and visual narrative surveys—with our heartfelt thanks: Gesture Survey 1: Clémence Bonnet, Yanis da Cunha, Marie Legenti, Clément Maclou, Baptiste Morel, Anthony Redon, Marie-Laure Redon-Bonnet. Gesture Survey 2: Isabelle Charnavel Alexandre Cremers, Anouk Dieuleveut, Paul Egré, Valentine Hacquard, Vincent Homer, Adèle Mortier, Léo Zaradzki. Visual Narrative Survey: Émile Enguehard, Diego Feinmann, Janek Guerrini, Jeremy Kuhn, Nur Lan, Matt Mandelkern, Justine Mertz, Benjamin Spector, Lyn Tieu, Anastasia Tsilia. Finally, we are very grateful to handling Editor Paul Portner and to the anonymous reviewers for detailed and constructive comments and criticisms.
Funding
Schlenker: This research received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 788077, Orisem, PI: Schlenker).
Chemla, Geraci, Schlenker: Research was conducted at DEC, Ecole Normale Supe´rieure – PSL Research University. DEC is supported by grant FrontCog ANR-17-EURE-0017.
Author information
Authors and Affiliations
Contributions
Schlenker designed the study and wrote the paper, with the exception of parts of Sect. 4.6 and some footnotes. Geraci coordinated work on LIS and co-advised Bonnet's internship on the gesture study. Bonnet designed the first gesture survey in collaboration with Schlenker and Geraci, collected the data, and computed the results. Jonathan Lamberton was the main ASL consultant and helped construct and transcribe all examples. He informally collected Jason Lamberton's ASL judgments. Santoro constructed and assessed the LIS data in collaboration with Geraci. Chemla provided methodological and theoretical advice throughout.
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
Appendices
Appendix 1: Pictorial semantics for mixed representations
In Sect. 6, we explained how gestural sequences can be treated (in a simplified fashion) as visual animation. In this appendix, we refine our iconic semantics for the more complex case of sign language classifier predicates. These require special measures because their lexical form is conventional, and only their position and movement to be interpreted iconically (see also Greenberg, 2019 for another perspective).
-
Mixed representations
The formal problem we must address is this: we must allow for mixed representations in which some symbols are non-iconic, but their position and movement is. A similar issue arose in a study of repetition-based plurals in ASL (Schlenker & Lamberton, 2022). In an ASL description of a drawing that involved 4 letters G arranged in a circular fashion, what appeared in signing space was (very roughly) as in (78)b on the left. These were four iterations of the manual letter 
, which looks nothing like the Latin letter it denotes, G. Still, the inferential judgments obtained suggested that the circular arrangement of the four manual letters 
on the left-hand side tracked the arrangement of the G's in the drawing on the right-hand side.
(78) | a. Purely iconic representation |
| |
b. Semi-iconic and semi-symbolic representation | |
|
In a very simplified analysis, one can take the signing space to be divided into 9 elementary components or ‘pixels', as shown in (78)b, with the rule that “a 
will appear in a pixel just in case a G from the drawing projects onto it”. By contrast, in a related case that involved a gestural representation of a little circle in an ASL sentence, a purely iconic rule could be followed: the four little circles appearing in signing space on the left-hand side of (78)a were taken to represent similar-looking shapes on the drawing, as displayed on the right-hand side of (78)a.
Schlenker and Lamberton (2022) handled mixed cases such as (78)b by modifying the marking rules that determine what appears in an iconic representation. In a purely iconic projection, corresponding to a case discussed by Greenberg (2021), the features associated with any given element in the picture correspond to those features of the environment that project to that point. An illustration appears in (79): only visible edges are represented (as lines) on the picture (further possible marking conventions are discussed in Greenberg (2014).
(79) | Illustration of a simple marking convention: only visible edges are marked (Greenberg, 2014) |
|
This simple marking rule can yield a simplified analysis of (78)a by way of the purely iconic rule in (80)a. To handle (78)b, by contrast, we need the mixed marking rule in (80)b.
(80) | a. Purely iconic marking rule: Assuming a method of projection, mark a pixel as ‘black’ if and only if its projection line meets the edge of an object. |
b. Mixed marking rule: Assuming a method a projection, mark a pixel (which may be taken to be a large part of the 2-dimensional space) with a given word W if and only if its projection line meets an object that lies in the denotation of W. |
-
Application to singular nouns
Our mixed marking rule was motivated by iconically interpreted repetition-based plurals, as in (78)a. But an immediate consequence of the mixed marking rule is that, in principle at least, an unrepeated sign, for instance the manual 
, could be interpreted semi-iconically as well, in the sense that its position in signing space could provide information about the position of its denotation, as schematized in the (imaginary) example below, where 
is signed high to suggest that its denotation (G in the relevant drawing) appears high as well. In other words, although the sign is conventional, its position in signing space is interpreted iconically.
(81) | An imaginary case in which the position of a single manual |
|
is interpreted by the mixed marking rule in (80)b
This option is important to analyze the arguments of classifier predicates: even a conventional, non-iconic sign can be interpreted iconically with respect to its position in signing space. This possibility is particularly salient for arguments of classifier predicates: their position and movement are interpreted iconically, and thus it is natural to also interpret iconically the position of their arguments in signing space.
-
Application to classifier predicates
To apply these ideas to classifier predicates, it is best to consider a maximally simple case, for instance one in which a vehicle classifier is used to describe the movement of a car in a cartoon or in a comic.Footnote 49 We can make use once again of the 9-pixel square in (78), now featuring a vehicle classifier in three different positions to represent three stages of a movement (in a less simplified analysis, the classifier movement would be made continuous or pseudo-continuous).
(82) | Illustration of a mixed marking rule for the movement of a vehicle classifier (the shape of is conventional but its movement is iconicFootnote 50) |
|
Our earlier rules for pictorial semantics will continue to apply, but with the proviso that a mixed marking rule must be used. For instance, if we assume that the vehicle classifier in picture P1 in (82) is associated with locus/variable a, it will give rise to truth conditions akin to (83); they are similar to what was delivered by (42) in the main text, except that the marking rule is the mixed one described above, hence the informal truth conditions we end up with (we write P1[a] because the picture includes a locus/variable a).
(83) | P1[a] is true of time t relative to π, w, s iff relative to π, at t: w projects (according to the mixed marking rule) to P1 and s(a) projects to variable a of P1, |
which roughly corresponds to: relative to π, at t: in w there is a blue car s(a) in a position corresponding to the right-most square of the middle row. |
With these adjustments, we can give the same general analysis to the crocodile-related classifier construction as we did to the crocodile-related gestural construction discussed above: the difference solely lies in the marking rule used for classifiers, as the crocodile-denoting manual shape is in principle allowed to have a conventional rather than a purely iconic form (of course nothing prevents a classifier lexical form from being to some extent iconic, and this is in fact the case of our CROCODILE example; the point is that nothing in the analysis requires this to be the case, and this is important if the theory is to apply to the vehicle classifier in (1) in the main text). As noted in Sect. 6.5.1, on the present semantics, the SOV order in (84) means that one first saw the crocodile, then the ball, then the crocodile eating the ball; and the meaning of the OSV order in (85) is that one first saw the ball, then the crocodile, and then the crocodile eating the ball. An SVO order would imply, oddly, that one saw a crocodile, then the crocodile eating the ball, then the ball.
(84) | a. CROCODILEa BALLb a-CRAWL-SWALLOW-b. | |
b. <C[a], B[b], Sτ[a, b]> | ||
(85) | a. BALLb CROCODILEa a-CRAWL-SWALLOW-b. | |
b. <B[b], C[a], Sτ[a, b]> | ||
For a crocodile spitting out a ball, as in (14), it is of course natural to present the ball as becoming visible after it was spit out, hence the preference for SVO.
Appendix 2: Linearizing preferences in 2-frame comics representing propositional attitudes
Starting from some of Langus and Nespor's Schouwstra and de Swart's own stimuli, reproduced in (86), we sought to assess whether linearizing the bubble before or after the thinking or speaking agent gave rise to ordering preferences.
If we had consistently tested linearized versions of the originals, we would have obtained inconsistent bubble tails, as in (87), where the tails do not point towards the attitude holders.
(87) | Tail directionality incoherence when linearized versions of the original pictures are consistently used |
a. Bubble after speech event | |
| |
b. Bubble before thought event | |
|
To address this problem, we tested the comics that sometimes involved mirror images of the original pictures (they appear in (90) and (91) below; half of them were mirror images). For this reason, we started by assessing the acceptability of the original pictures relative to their mirror images, expecting to find little or no difference. This was indeed the case, as seen (88)–(89).
(88) | Complex vignette used to elicit gestural embeddings (Langus & Nespor, 2010, Appendix) | |
10 consultants | ||
Number of consultants that prefer a to b: 0 | ||
Number of consultants that prefer b to a: 2 | ||
a. Original | b. Mirror image | |
| ||
(89) | Stimulus corresponding to a cook thinking of a sock (Schouwstra & de Swart, 2014, Fig. 2) | |
Number of consultants that prefer a to b: 0 | ||
Number of consultants that prefer b to a: 1 | ||
a. Original | b. Mirror image | |
| ||
As can be seen, the postposed bubble gets better ratings that the preposed bubble in (90), with all consultants expressing a preference going for the postposed bubble. Averages show little difference in (91), and half the participants expressed no preference. Among the 5 who did, 4 preferred the postposed bubble.
(90) | Transformation of (88) into a simple visual narrative made of 2 pictures |
10 consultants | |
Number of consultants that prefer a to b: 8 | |
Number of consultants that prefer b to a: 0 | |
a. Speech event before speech content (mirror image of the original) | |
| |
b. Speech content before speech event (original) | |
|
(91) | Transformation of (89) into a simple visual narrative made of 2 pictures |
10 consultants | |
Number of consultants that prefer a to b: 4 | |
Number of consultantsthat prefer b to a: 1 | |
a. Thinking event before thought content (original) | |
| |
b. Thought content before thinking event (mirror image of the original) | |
|
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Schlenker, P., Bonnet, M., Lamberton, J. et al. Iconic Syntax: sign language classifier predicates and gesture sequences. Linguist and Philos 47, 77–147 (2024). https://doi.org/10.1007/s10988-023-09388-z
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10988-023-09388-z