INTERSEMIOTIC THEORIES: TOWARDS A FRAMEWORK FOR THE ANALYSIS OF THE INTERPLAY OF LANGUAGE AND VISUAL IMAGES IN MOVIE POSTERS
Main Article Content
Abstract
The present paper aims to propose a preliminary adapted framework to analyze the relations between language and images in movie posters. After critically reviewing some major approaches to intersemiosis with both their strengths and weaknesses, the study decides to employ the Rhetorical Structure Theory (RST) discoursal approach in general and the specific framework of Bateman’s (2008) Genre and Multimodality (GeM) for the purposes of the current study. The review also suggests the need to include compositional features into the discussion of text-image relations. Putting all these issues into consideration, the paper finally develops an adapted analytical framework that employs Bateman’s (2008) GeM as a platform and combines the layout and RST layers of GeM with compositional features for a more thorough examination of the roles of the visual and verbal elements. The application of the adapted model is then illustrated by the analysis of the theatrical release poster of What women want (2000).
Article Details
Keywords
intersemiotic theories, movie posters, framework, Bateman’s (2008) Genre and Multimodality
References
Baldry, A., & Thibault, P. J. (2006). Multimodal transcription and text analysis. Equinox.
Bateman, J. (2008). Multimodality and genre: A foundation for the systematic analysis of multimodal documents. Palgrave Macmillan.
Bateman, J. (2014). Text and image: A critical introduction to the visual/verbal divide. Routledge.
Bateman, J. (2013). Using multimodal corpora for empirical research. In C. Jewitt, (ed.), The Routledge handbook of multimodal analysis (2nd edition) (pp. 238–252). Routledge.
Caple, H. (2013). Photojournalism: A social semiotic approach. Palgrave Macmillan.
Caple, H. (2019). “Lucy says today she is a Labordoodle”: How the dogs-of-Instagram reveal voter preferences. Social Semiotics, 29(4), 427–447.
Chen, Y., & Gao, X. (2013). Interpretation of movie posters from the perspective of multimodal discourse analysis. GSTF International journal on education (JEd), 1(1), 1-24.
Forceville, C. J. (1996). Pictorial metaphor in advertising. Routledge.
Halliday, M. A. K., & Hasan, R. (1976). Cohesion in English. Longman.
Halliday, M. A. K., & Matthiessen, C. M. (2004). An Introduction to functional grammar (3rd edition). Edward Arnold.
Iftikhar, S., Shahnaz, A., & Masroor, F. (2019). Multimodal discourse analysis of the poster covers of Academy award winning animated feature movies. PUTAJ – Humanities and Social sciences, 26(2), 49-80.
Jewitt, C., Bezemer, J., & O'Halloran, K. (2016). Introducing multimodality (1st ed.). Routledge. https://doi.org/10.4324/9781315638027
Knox, J. (2007). Visual-verbal communication on online newspaper homepages. Visual communication, 6(1), 19-53.
Kong, K. (2006). A taxonomy of the discourse relations between words and visual. Information design journal, 14(3), 207-230.
Kress, G., & van Leeuwen, T. (2006). Reading images: the grammar of visual design (2nd ed.). Routledge. https://doi.org/10.4324/9780203619728
Kress, G., & van Leeuwen, T. (2021). Reading Images: The Grammar of Visual Design (3rd ed.). Routledge. https://doi.org/10.4324/9781003099857
Linh, N. T. T. (2021). A multimodal discourse analysis of romantic comedy movie posters. VNU Journal of Foreign Studies, 37(3), 79-93. https://doi.org/10.25073/2525-2445/vnufs.4647
Liu, Y., & O’Halloran, K. (2009). Intersemiotic texture: analyzing cohesive devices between language and images. Social semiotics, 19(4), 367-388.
Maiorani, A. (2007). ‘Reloading’ movies into commercial reality: A multimodal analysis of The Matrix trilogy’s promotional posters. Semiotica, 166(1/4), 45-67.
Maiorani, A. (2014). The Matrix phenomenon. In S. Norris & C. Maier (eds.), Integrations, images and text: A reader in multimodality (pp. 269-279). De Gruyter Mouton.
Mann, W. C., & Thompson, S. A. (1988). Rhetorical Structure Theory: toward a functional theory of text organization, Text, 8(3), 243-281.
Martin, J. R. (1992). English Text: systems and structure. Benjamins.
Martin, J. R., & Rose, D. (2008). Genre relations: mapping culture. Equinox.
Martinec, R. (1998). Cohesion in action. Semiotica, 120(1-2), 161-180.
Martinec, R., & Salway, A. (2005). A system for image_text relations in new (and old) media. Visual Communication, 4(3), 337-371.
Matthiessen, C.M.I.M. (2009). Multisemiosis and context-based register typology: Registerial variation in the complementarity of semiotic systems. In E. Ventola & J. Moya (eds.), The world told and the world shown (pp. 11- 38). Palgrave Macmillan.
Peng, Z. (2022). A multimodal discourse analysis of movie posters from the perspective of visual grammar — A case study of "Hi, Mom". Theory and Practice in Language Studies, 12(3), 605-609. https://doi.org/10.17507/tpls.1203.22
Rondon, T., & Heberle, V. (2022). Marvel's captain america civil war poster: an analysis based on the grammar of visual design. Revista X, 17(3), 726-751.
Royce, T. (1998). Synergy on the page: exploring intersemiotic complementarity in page-based multimodal text. Japan Association for Systemic Functional Linguistics Occasional Papers, 1(1), 25–49.
Royce, T. (2007). Inter-semiotic complementarity: A framework for multimodal discourse analysis. In T. Royce and W. Bowcher (Eds.), New directions in the analysis of multimodal discourse (pp. 63-109). Lawrence Erlbaum Associates.
Stockl, H. (2004). In between modes: language and image in printed media. In E. Ventola, C. Charles & M. Kaltenbacher (eds.), Perspectives on multimodality (pp. 9–30). John Benjamins.
Stöckl, H. (2019a). Image-centricity – When visuals take center stage: analyses and interpretations of a current (news) media practice. In H. Stöckl, H. Caple & J. Pflaeging (eds.), Shifts towards image-centricity in contemporary multimodal practices (1st ed.) (pp. 19-41). Routledge.
https://doi.org/10.4324/978042948796 5
Stöckl, H. (2019b). Linguistic multimodality – multimodal linguistics: a state-of-the-art sketch. In J.Wildfeuer, J. Pflaeging, J. A. Bateman, O. Seizov, & C.I. Tseng (Eds.), Multimodality: Disciplinary thoughts and the challenge of diversity (pp. 41–68). De Gruyter.
Stöckl, H., Caple, H., & Pflaeging, J. (Eds.). (2019). Shifts towards Image-centricity in contemporary multimodal practices (1st ed.). Routledge. https://doi.org/10.4324/9780429487965
Taboada, M., & Habel, K. (2013). Rhetorical relations in multimodal documents. Discourse Studies, 15(1), 65-89.
Unsworth, L. (2007). Image/text relations and intersemiosis: towards multimodal text description for multiliteracies education. In L. Barbara, & T. B. Sardinha (eds.), Proceedings of the 33rd International Systemic Functional Congress (33rd ISFC) (pp. 1165-1205). Pontificia Universidade Catolica De Sao Paulo (PUCSP), Sao Paulo, Brazil.
http://www.pucsp.br/isfc/proceedings/Artigos%20pdf/59pl_unsworth_1165a1205.pdf
van Leeuwen, T. (2005). Introducing Social Semiotics. Routledge.
Zappavigna, M. (2019). “And then he said …No one has more respect for women than I do” Intermodal relations and intersubjectivity in image macros. In H. Stöckl, H. Caple, & J. Pflaeging (eds.), Shifts towards image-centricity in contemporary multimodal practices (1st ed.) (pp. 204-225). Routledge. https://doi.org/10.4324/9780429487965
Bateman, J. (2008). Multimodality and genre: A foundation for the systematic analysis of multimodal documents. Palgrave Macmillan.
Bateman, J. (2014). Text and image: A critical introduction to the visual/verbal divide. Routledge.
Bateman, J. (2013). Using multimodal corpora for empirical research. In C. Jewitt, (ed.), The Routledge handbook of multimodal analysis (2nd edition) (pp. 238–252). Routledge.
Caple, H. (2013). Photojournalism: A social semiotic approach. Palgrave Macmillan.
Caple, H. (2019). “Lucy says today she is a Labordoodle”: How the dogs-of-Instagram reveal voter preferences. Social Semiotics, 29(4), 427–447.
Chen, Y., & Gao, X. (2013). Interpretation of movie posters from the perspective of multimodal discourse analysis. GSTF International journal on education (JEd), 1(1), 1-24.
Forceville, C. J. (1996). Pictorial metaphor in advertising. Routledge.
Halliday, M. A. K., & Hasan, R. (1976). Cohesion in English. Longman.
Halliday, M. A. K., & Matthiessen, C. M. (2004). An Introduction to functional grammar (3rd edition). Edward Arnold.
Iftikhar, S., Shahnaz, A., & Masroor, F. (2019). Multimodal discourse analysis of the poster covers of Academy award winning animated feature movies. PUTAJ – Humanities and Social sciences, 26(2), 49-80.
Jewitt, C., Bezemer, J., & O'Halloran, K. (2016). Introducing multimodality (1st ed.). Routledge. https://doi.org/10.4324/9781315638027
Knox, J. (2007). Visual-verbal communication on online newspaper homepages. Visual communication, 6(1), 19-53.
Kong, K. (2006). A taxonomy of the discourse relations between words and visual. Information design journal, 14(3), 207-230.
Kress, G., & van Leeuwen, T. (2006). Reading images: the grammar of visual design (2nd ed.). Routledge. https://doi.org/10.4324/9780203619728
Kress, G., & van Leeuwen, T. (2021). Reading Images: The Grammar of Visual Design (3rd ed.). Routledge. https://doi.org/10.4324/9781003099857
Linh, N. T. T. (2021). A multimodal discourse analysis of romantic comedy movie posters. VNU Journal of Foreign Studies, 37(3), 79-93. https://doi.org/10.25073/2525-2445/vnufs.4647
Liu, Y., & O’Halloran, K. (2009). Intersemiotic texture: analyzing cohesive devices between language and images. Social semiotics, 19(4), 367-388.
Maiorani, A. (2007). ‘Reloading’ movies into commercial reality: A multimodal analysis of The Matrix trilogy’s promotional posters. Semiotica, 166(1/4), 45-67.
Maiorani, A. (2014). The Matrix phenomenon. In S. Norris & C. Maier (eds.), Integrations, images and text: A reader in multimodality (pp. 269-279). De Gruyter Mouton.
Mann, W. C., & Thompson, S. A. (1988). Rhetorical Structure Theory: toward a functional theory of text organization, Text, 8(3), 243-281.
Martin, J. R. (1992). English Text: systems and structure. Benjamins.
Martin, J. R., & Rose, D. (2008). Genre relations: mapping culture. Equinox.
Martinec, R. (1998). Cohesion in action. Semiotica, 120(1-2), 161-180.
Martinec, R., & Salway, A. (2005). A system for image_text relations in new (and old) media. Visual Communication, 4(3), 337-371.
Matthiessen, C.M.I.M. (2009). Multisemiosis and context-based register typology: Registerial variation in the complementarity of semiotic systems. In E. Ventola & J. Moya (eds.), The world told and the world shown (pp. 11- 38). Palgrave Macmillan.
Peng, Z. (2022). A multimodal discourse analysis of movie posters from the perspective of visual grammar — A case study of "Hi, Mom". Theory and Practice in Language Studies, 12(3), 605-609. https://doi.org/10.17507/tpls.1203.22
Rondon, T., & Heberle, V. (2022). Marvel's captain america civil war poster: an analysis based on the grammar of visual design. Revista X, 17(3), 726-751.
Royce, T. (1998). Synergy on the page: exploring intersemiotic complementarity in page-based multimodal text. Japan Association for Systemic Functional Linguistics Occasional Papers, 1(1), 25–49.
Royce, T. (2007). Inter-semiotic complementarity: A framework for multimodal discourse analysis. In T. Royce and W. Bowcher (Eds.), New directions in the analysis of multimodal discourse (pp. 63-109). Lawrence Erlbaum Associates.
Stockl, H. (2004). In between modes: language and image in printed media. In E. Ventola, C. Charles & M. Kaltenbacher (eds.), Perspectives on multimodality (pp. 9–30). John Benjamins.
Stöckl, H. (2019a). Image-centricity – When visuals take center stage: analyses and interpretations of a current (news) media practice. In H. Stöckl, H. Caple & J. Pflaeging (eds.), Shifts towards image-centricity in contemporary multimodal practices (1st ed.) (pp. 19-41). Routledge.
https://doi.org/10.4324/978042948796 5
Stöckl, H. (2019b). Linguistic multimodality – multimodal linguistics: a state-of-the-art sketch. In J.Wildfeuer, J. Pflaeging, J. A. Bateman, O. Seizov, & C.I. Tseng (Eds.), Multimodality: Disciplinary thoughts and the challenge of diversity (pp. 41–68). De Gruyter.
Stöckl, H., Caple, H., & Pflaeging, J. (Eds.). (2019). Shifts towards Image-centricity in contemporary multimodal practices (1st ed.). Routledge. https://doi.org/10.4324/9780429487965
Taboada, M., & Habel, K. (2013). Rhetorical relations in multimodal documents. Discourse Studies, 15(1), 65-89.
Unsworth, L. (2007). Image/text relations and intersemiosis: towards multimodal text description for multiliteracies education. In L. Barbara, & T. B. Sardinha (eds.), Proceedings of the 33rd International Systemic Functional Congress (33rd ISFC) (pp. 1165-1205). Pontificia Universidade Catolica De Sao Paulo (PUCSP), Sao Paulo, Brazil.
http://www.pucsp.br/isfc/proceedings/Artigos%20pdf/59pl_unsworth_1165a1205.pdf
van Leeuwen, T. (2005). Introducing Social Semiotics. Routledge.
Zappavigna, M. (2019). “And then he said …No one has more respect for women than I do” Intermodal relations and intersubjectivity in image macros. In H. Stöckl, H. Caple, & J. Pflaeging (eds.), Shifts towards image-centricity in contemporary multimodal practices (1st ed.) (pp. 204-225). Routledge. https://doi.org/10.4324/9780429487965