Abstract
Intersemiotic projection is one of the most common configurations in the knowledge construction process of academic comics. Although previous studies address some general features of intersemiotic projection, further research on interdependency relations of intersemiotic projection is needed in order to map out the whole system. This study, based on the social-semiotic approach to multimodal studies, proposes a systemic framework of image-text paratactic and hypotactic projection in academic comics. This framework identifies three sub-categories of paratactic projection and hypotactic projection, respectively: (1) Emergent, (2) Adjoined, (3) Sequential, (4) Compound, (5) Subsumed, and (6) Fused. The use of these projection configurations contributes to the realization of the communicative purposes of academic comics to evidence, exemplify, emphasize, explain, elaborate, and visualize the discussed knowledge. This research attempts not only to contribute to the theoretical development of multimodality from the social-semiotic perspective by refining and enlarging the multimodal hypotactic projection framework but also to provide an effective and applicable tool for comics analysis.
Funding source: The “Thirteenth-Five-year Plan” Project of Guangdong Planning office of Philosophy and Social Science
Award Identifier / Grant number: GD18CWW06
-
Research funding: This work was supported by The “Thirteenth-Five-year Plan” Project of Guangdong Planning office of Philosophy and Social Science (grant number GD18CWW06).
References
Barthes, Roland. 1997 [1961]. The photographic message. In Image-music-text, 15–21. London: Fontana.Search in Google Scholar
Barthes, Roland. 1997 [1964]. Rhetoric of the image. In Image-music-text, 32–21. London: Fontana.Search in Google Scholar
Bateman, John A. 2011. The decomposability of semiotic modes. In Kay L. O’Halloran & Bradley A. Smith (eds.), Multimodal studies: Exploring issues and domains, 17–28. New York: Routledge.Search in Google Scholar
Bateman, John A. & Karl H. Schmidt. 2012. Multimodal film analysis: How films mean. London: Routledge.10.4324/9780203128220Search in Google Scholar
Bateman, John A. 2014a. Using multimodal corpora for empirical research. In Jewitt Carey (ed.), The Routledge handbook of multimodal analysis, 238–252. London: Routledge.Search in Google Scholar
Bateman, John A. 2014b. Text and image: A critical introduction to the visual/verbal divide. Abingdon & New York: Routledge.10.4324/9781315773971Search in Google Scholar
Bateman, John A. & Janina Wildfeuer. 2014a. Defining unites of analysis for the systematic analysis of comics: A discourse-based approach. Studies in Comics 5(2). 373–203. https://doi.org/10.1386/stic.5.2.373_1.Search in Google Scholar
Bateman, John A. & Janina Wildfeuer. 2014b. A multimodal discourse theory of visual narrative. Journal of Pragmatics 74. 180–208. https://doi.org/10.1016/j.pragma.2014.10.001.Search in Google Scholar
Bateman, John A., Francisco O. D. Veloso, Janina Wildfeuer, Felix H. Cheung & Nancy Songdan Guo. 2017a. An open multilevel classification schema for the visual layout of comics and graphic novels: Motivation and design. Digital Scholarship in the Humanities 32(3). 476–210.10.1093/llc/fqw024Search in Google Scholar
Bateman, John A., Janina Wildfeuer & Tuomo Hiippala. 2017b. Multimodality: Foundations, research, and analysis. Berlin & Boston: Walter de Gruyter.10.1515/9783110479898Search in Google Scholar
Cohn, Neil. 2013a. The visual language of comics: Introduction to the structure and cognition of sequential images. London: Bloomsbury.Search in Google Scholar
Cohn, Neil. 2013b. Beyond speech balloons and thought bubbles: The integration of text and image. Semiotica 197(1/4). 35–23. https://doi.org/10.1515/sem-2013-0079.Search in Google Scholar
Cohn, Neil. 2014. The architecture of visual narrative comprehension: The interaction of narrative structure and page layout in understanding comics. Frontiers in Psychology 5. 1–2. https://doi.org/10.3389/fpsyg.2014.00680.Search in Google Scholar
Davies, Paul F. 2019. New choices of the comics creator. The Comics Grid 9(1). 1–9. https://doi.org/10.16995/cg.153.Search in Google Scholar
Eggins, Suzanne. 2004. An introduction to systemic functional linguistics. London: Continuum.Search in Google Scholar
Eisner, Will. 1985. Comics and sequential art. Florida: Poorhouse Press.Search in Google Scholar
Feng, Dezheng & Kay L. O’Halloran. 2012. Representing emotive meaning in visual images: A social semiotic approach. Journal of Pragmatics 44. 2067–2084. https://doi.org/10.1016/j.pragma.2012.10.003.Search in Google Scholar
Feng, Dezheng, Delu Zhang, & Kay L. O’Halloran. 2014. Advances and frontiers of multimodal discourse analysis. Contemporary Linguistics 1. 88–99.Search in Google Scholar
Forceville, Charles. 1996. Pictorial metaphor in advertising. London: Routledge.10.4324/9780203272305Search in Google Scholar
Halliday, Michael A. K. 1978. Language as social semiotic: The social interpretation of language and meaning. London: Edward Arnold.Search in Google Scholar
Halliday, Michael A. K. & Ruqaiya Hasan. 1985. Language, text, and context. Victoria: Deakin University.Search in Google Scholar
Halliday, Michael A. K. 1994. Introduction to functional grammar, 2nd edn. London: Arnold.Search in Google Scholar
Halliday, Michael A. K. & Christian M. I. M. Matthiessen. 2014. Introduction to functional grammar, 4th edn. Abingdon: Routledge.10.4324/9780203783771Search in Google Scholar
Helms, Jason M. 2015. Is this article a comic. Digital Humanities Quarterly 9(3). http://www.digitalhumanities.org/dhq/vol/9/4/000230/000230.html (accessed 1 January 2015).Search in Google Scholar
Humphrey, Aaron. 2015. Visual and spatial language: The silent voice of Woodstock. Composition Studies 43(1). 19–30.Search in Google Scholar
Kashtan, Aaron J. 2015. Materiality comics. Digital Humanities Quarterly 9(4). http://www.digitalhumanities.org/dhq/vol/9/4/000212/000212.html (accessed 1 January 2015).Search in Google Scholar
Kress, Gunther & Theo van Leeuwen. 1996. Reading images: The grammar of visual design. London: Routledge.Search in Google Scholar
Kress, Gunther & Theo van Leeuwen. 2006. Reading images: The grammar of visual design, 2nd edn. London & New York: Routledge.10.4324/9780203619728Search in Google Scholar
Kress, Gunther & Theo van Leeuwen. 2021. Reading images: The grammar of visual design, 3rd edn. Abingdon & New York: Routledge.10.4324/9781003099857Search in Google Scholar
Kress, Gunther. 2010. Multimodality: A social semiotic approach to contemporary communication. London & New York: Routledge.Search in Google Scholar
Kuhn-Osius, Eckhard. 2016. Before they were art: WestGerman proto-comics and comics: A brief and somewhat subjective survey. In Lynn M. Kutch (ed.), Novel perspectives on German-language comics studies: History, pedagogy, theory, 37–65. London: Lexington.Search in Google Scholar
Martin, James R., Christian M. I. M. Matthiessen & Clare Painter. 2010. Deploying functional grammar. Beijing: Commercial Press.Search in Google Scholar
Martinec, Radan & Andrew Salway. 2005. A system for image-text relations in new (and old) media. Visual Communication 4(3). 337–270. https://doi.org/10.1177/1470357205055928.Search in Google Scholar
McCloud, Scott. 1993. Understanding comics: The invisible art. New York: Harper Perennial.Search in Google Scholar
McCloud, Scott. 2006. Making comics: Storytelling secrets of comics, manga, and graphic novels. New York: Harper.Search in Google Scholar
Meskin, Aaron. 2007. Defining comics? The Journal of Aesthetics and Art Criticism 65(4). 369–279. https://doi.org/10.1111/j.1540-594x.2007.00270.x.Search in Google Scholar
O’Halloran, Kay L. 2005. Mathematical discourse: Language, symbolism, and visual images. London: Continuum.Search in Google Scholar
Painter, Clare, James Martin & Len Unsworth. 2013. Reading visual narratives: Image analysis in children’s picture books. Sheffield: Equinox.Search in Google Scholar
Palmer, Stephen. 1992. Common region: A new principle of perceptual grouping. Cognitive Psychology 24. 436–447. https://doi.org/10.1016/0010-0285(92)90014-s.Search in Google Scholar
Royce, Terry D. 1998. Synergy on the page: Exploring inter-semiotic complementarity in page-based multimodal text. JASFL Occasional Papers 1(1). 25–29.Search in Google Scholar
Royce, Terry D. 2007. Inter-semiotic complementarity: A framework for multimodal discourse analysis. In Terry D. Royce & Wendy L. Bowcher (eds.), New directions in the analysis of multimodal discourse, 63–209. Mahwah, NJ: Lawrence Erlbaum Associates.Search in Google Scholar
Tsai, Chinchung. 2005a. Zhuang Zi speaks Brian Bruya (trans.). Beijing: Modern Press.Search in Google Scholar
Tsai, Chinchung. 2005b. The Dao speaks Brian Bruya (trans.). Beijing: Modern Press.Search in Google Scholar
Tseng, Chiao-I & John A. Bateman. 2018. Cohesion in comics and graphic novels: An empirical comparative approach to transmedia adaptation in City of Glass. Adaptation 11(2). 122–243. https://doi.org/10.1093/adaptation/apx027.Search in Google Scholar
Vales, Elisa. 2015. Narrative process annotation of comic strips in corpus analysis. In Janina Wildfeuer (ed.), Building bridges for multimodal research: International perspectives on theories and practices of multimodal analysis, 267–286. Frankfurt am Main: Peter Lang.Search in Google Scholar
Wertheimer, Max. 1923. Untersuchungen zur Lehre von der Gestalt [Laws of organization in perceptual forms]. Psychologische Forschung 4. 301–350. https://doi.org/10.1007/bf00410640.Search in Google Scholar
Yang, Gene. 2008. Graphic novels in classroom. Language Arts 85. 185–292.Search in Google Scholar
Zeng, Lei & Muwen Yang. 2016. A comprehensive analytic framework for multimodal projection in verbal-image texts. Journal of Xi’an International Studies University 4. 35–28.Search in Google Scholar
Zeng, Lei & Muwen Yang. 2019. A study of projection system in academic comics. Modern Foreign Languages 42(5). 610–222.Search in Google Scholar
© 2022 Walter de Gruyter GmbH, Berlin/Boston
