In the previous lesson you have been sensitized to the concept of multimodal literacy by interacting with various modes of communication in a playful way. This lesson supports the practical approach of the previous one by focusing more on the theoretical framework. What do we get when we untangle the compound 'multimodal literacy' ? What does it tell us about the context in which the term evolved? What kind of scholarly views are there on the concept of multimodal literacy?

Watch the clip by (Enle Li & Liz Xiong)

This clip offers an impression of how words refer to specific things that have a spatial dimension. The words "Car", "Sun", or "Rain", for example, are not merely representations or symbols that exist as such in our minds. By using them in the form of language to communicate, we leave out much of their contextual value, as this clip demonstrates. Multimodal literacy tries to address the other kinds of information that we unconsciously use as tools in our communication, especially with the advent of digital technology.

Defining Multimodal literacy

The term ‘multimodal literacy’ is recurrent in many fields of research. It is essential in the study of Literacy, Education, Media, Communication, Design, Information, Semiotics and Digital Humanities, to name but a few fields. This does not mean that scholars from these different disciplines attribute the same meaning to the term. The best way to analyse it is by examining its components. Let's start with analysing the components of the concept.

Multi, refers to many, as opposed to few. It means that multimodal literacy is not limited to mastering speech and writing, but includes knowing how to make use of facial and bodily gestures, sound, image, design, colour, space,  and many other features (Kress, 2003). The second component, modality, refers to ways in which something is done or expressed; the way of operating.

The third component, literacy, is the most complex term. Its traditional interpretation refers to the technical capacity to read and write (Ferrari et al., 2014). Yet from the 1970s onwards, a broader definition was embraced, in which literacy was seen as a dynamic practice embedded in social and cultural life and thus subject to constant change (Street 1984, p. 1). Think of the difference between the skills needed to write a handwritten letter and those required to send a WhatsApp or Snapchat message accompanied by fragments of sound or video. Some scholars even extend the meaning of the concept of literacy to familiarity with “reading the book of nature”, e.g. the act of observing the stars to determine the right time to plant a crop, or observing the tides to get the best catch of fish (Bart and Hamilton, 2000).

Multimodal + Literacy

When Gunther Kress and Carey Jewitt proposed adding the prefix multimodal to the term literacy in their seminal work "Multimodal Literacy", they had a broader definition of literacy in mind, one that reflected societal changes related to the digital turn in the social sciences (Jewitt and Cress, 2003). In doing so, they drew on the work of the New London Group, of which Kress was a member, which already in 1996 had introduced the related term multiliteracies (New London Group, 1996). The difference is that the latter term extends the meaning of literacy beyond a variety of means of knowledge production. It is intended to cover a very broad range of related capacities and skills: the various modes of communication (linguistic, visual, audio, gestural), the various media (television, radio, smartphone, computer, augmented reality), the linguistic and cultural diversity brought about by the migration of people on a global scale, and finally the different environments in which literacy practices are put into operation: school, home and other informal settings (Cope and Kalantizis, 2000; Fantin, 2013).

Digital Literacy

The related term digital literacy also covers the notion of conscious use of modes of representation, but it emphasises the need for specific knowledge of the digital realm. A digitally literate person is able to search, access, process and create information through the use of digital technology and the World Wide Web (Ferrari et al. 2014).

Is Multimodal Literacy New?

Literacy has always been multimodal to some extent. The act of writing involves visual and actional modes, and in the act of speaking, pace, rhythm and tone also contribute to creating meaning. Moreover, when we watch television or films we also listen to sound (Kress, 2003). The reason for addressing this topic in a more systematic way is the spectacular increase in the speed and scale at which modes of expression can be produced, combined and “re-mediated”. Taken in conjunction with the shifting balance between textual and non-textual representations of knowledge, this calls for a critical reflection on how we deal with information. We are moving from the logic of writing on a piece of paper to the logic of creating malleable digital images on a screen.

Exercise : Reflection

References

• Barton, D. and Hamilton, M. (2000) Literacy Practices. In D. Barton, M. Hamilton and R. Ivanic (Eds), Situated Literacies: Reading and Writing in Context. London: Routledge.
• Cope, B. and Kalantzis, M. (Eds.) (2000) Multiliteracies: Literacy Learning and the Design of Social Futures. South Yarra, Melbourne: Macmillan Publishers.
• Fantin Monica (2013), Beyond Babel, in: International Journal of Digital Literacy and Digital Competence 2(1):1-10.4018/jdldc.2011010101
• Ferrari Anusca, Barbara Neža Brečko, Yves Punie, ‘DIGCOMP: A Framework for Developing and Understanding Digital Competence in Europe, in: eLearning Papers 38, May 2014. https://op.europa.eu/s/pdtR
• Jewitt, C., & Kress, G. (Eds.)(2003). Multimodal Literacy. New York: Peter Lang.
• Street, B. V. (1984) Literacy in Theory and Practice. Cambridge: Cambridge University Press.
• The New London Group (1996). A Pedagogy of Multiliteracies:Designing Social Futures. Harvard Educational Review: April 1996, Vol. 66, No. 1, pp. 60-93. [PDF]; https://doi.org/10.17763/haer.66.1.17370n67v22j160u