EL.LE

Educazione Linguistica. Language Education

Rivista | EL.LE
Fascicolo | 13 | 3 | 2024
Articolo | Digital Storytelling as Multimodal Response to Young Adult Literature

Digital Storytelling as Multimodal Response to Young Adult Literature

Promoting EFL Students’ Multiliteracies, Global Citizenship, and Mediation Skills

Abstract

This study examines how Digital Storytelling enhanced upper secondary students’ multiliteracy and mediation skills in EFL and their reflections on global issues. Thirteen digital stories, created in response to Young Adult Literature, were analysed using Visual Communication Grammar and Soundscape. This analysis was complemented by students’ diaries and classroom field notes. Results show that students used different remix strategies to reinterpret novel themes, enjoying collaboration and fostering critical and creative skills but struggling with brainstorming, research, and editing.


Open access | Peer reviewed

Presentato: 30 Aprile 2024 | Accettato: 15 Settembre 2024 | Pubblicato 03 Dicembre 2024 | Lingua: en

Keywords Digital storytellingAnalysisMultiliteraciesEFLMediationYoung adult literatureGlobal citizenship education

1 Introduction

Digital storytelling (DST) is defined as “short personal stories created with digital technology tools that are then shared with others in order to present information, ideas and opinions on a range of topics and themes” (Robin 2016, 19). This definition extends the original focus of DST on personal narratives to also include other genres, such as historical and socio‑political documentaries, commercials, tutorials, or news broadcasts (Christiansen, Koelzer 2016; Oskoz, Elola 2022). These stories are not only meant as tools of personal expression but also, and mainly, of education (Gregory‑Signes 2014). Also the range of technologies to create a digital story (DS) has dramatically changed, moving from computer‑based to mobile‑based applications, which have opened a whole range of other digital and multimodal affordances for learners as well as teachers (Lambert 2018; Kim, Yatsu, Li 2021).

In the context of language education, DST has been used with young students (Kim, Yatsu, Li 2021; Kim, Jia 2020) as well as university students (Oskoz, Elola 2014; 2016a), and in the context of different foreign languages (FL), showing the many affordances for both students’ FL and multiliteracy skills development.1 Also research has shown that DST can be an invaluable tool in promoting students’ deeper understanding of difficult topics, self‑reflection, and empathy (Kim, Jia 2020; Jiang 2022) in line with the premises of Global Citizenship Education (GCE; Unesco 2015). On the other hand, studies are showing that DST can also support language students’ development of “mediation of a text” skills (Council of Europe 2020) when used as a multimodal response to literature (Horne 2021). However, more research is needed to explore how students combine different modes (Oskoz, Elola 2022) and use remix strategies (Hafner 2015; see § 3.4.1) to create new meanings that reflect their understanding and interpretation of the text, while moving beyond it (Dail, Vásquez 2018).

Although the research was carried out jointly by the three authors, Maria Elisa Fina wrote §§ 3.4.1, 4.1, and 4.2, Fabiana Fazzi wrote §§ 2, 3.1, 3.2, 3.3, 3.4.2, 4.3, and 5, and Elisa Da Lio wrote §§ 1 and 6.

1  Abidin et al. 2011; Fu, Yang, Yeh 2022; Oskoz, Elola 2016a; 2016b; Yang, Chen, Hung 2020.

Our research aims to fill this gap by reporting on the multimodal analysis of thirteen digital stories designed by upper secondary students in response to the reading of Young Adult Literature (YAL) in English as a FL. The multimodal analysis is complemented by the qualitative analysis of the classroom observations and artefacts showing how DST is in line with both the principles and objectives of multiliteracies pedagogy and GCE, while also contributing to enhancing students’ skills regarding the ‘mediation of a creative text’.

2 Literature Review

2.1 Digital Storytelling to Promote Multiliteracy and Global Citizenship Skills Development in a FL

Digital Storytelling is today a well‑established pedagogical tool across different disciplines. Its popularity is due to the power of storytelling as inherent to human nature combined with the creative use of digital tools and resources with which students are somewhat already familiar. In fact, as Kim, Yatsu, Li (2021) point out, young students already engage with digital practices that underpin DST, such as selecting pictures and sounds, filming, and editing, when creating short videos on TikTok and Instagram. This is why using DST is believed to lower the distance between formal and informal ways of knowing and learning, promoting students’ motivation and active engagement with content. In this context, studies have shown that when DST is employed in the FL classroom, it can lead to different gains for students, including the enhancement of listening (Abidin et al. 2011), speaking (Fu, Yang, Yeh 2022) and writing skills (Oskoz, Elola 2016a; Tanrıkulu 2022), the increase of motivation (Yang, Chen, Hung 2020; Hava 2021) and willingness to communicate (Huang 2023), as well as the development of digital (Wake 2012) and other 21st century literacy skills (e.g. research, critical and creative thinking, interpersonal; Yang, Chen, Hung 2020). For example, Huang (2023) used DST with EFL university students in Taiwan and demonstrated its efficacy in promoting students’ development of speaking proficiency, including lexical and grammar use, pronunciation, and intonation. In fact, DST offers rich opportunities to practise, and re‑record spoken words which encourage learners to refine their pronunciation and enunciation (Oskoz, Elola 2022, 264). On the other hand, Fu, Yang and Yeh’s (2022) research also demonstrates that DST can promote students’ fluency, intended as speech rate and mean length of utterances, because it involves them in authentic communicative tasks focused on specific topics. As regards writing skills, Oskoz and Elola (2014) found that university students engaged in DST in Spanish as a FL not only developed a wide range of stylistic devices, learning to differentiate between the language required in a more traditional essay and in a DS, but also experimented with grammar, to convey the desired tone and rhythm, and vocabulary, to evoke the intended emotions. Oskoz and Elola (2016a) confirmed these results by showing that DST can promote students’ writing styles and manners. As Castañeda (2013, 46) claims, when creating a DS, students are primarily engaged in the crafting, revision, and narration of the story, rather than on the use of technology which is secondary. It is this focus on writing in the DST process that helps foster students’ “personal creativity and practice with specific discourse structures” (Oskoz, Elola 2014, 197).

However, when creating a DS students are not only involved in traditional literacy practices (reading and writing) but also in new literacy practices which require a wide range of skills, including: taking advantage of technological tools and meaningfully interacting with them (technological literacy); finding, evaluating, and synthesising information on the internet (information literacy); effectively orchestrating different modes (verbal, written, visual, and auditory) for meaning making (multimodal literacy); and learning to know themselves, developing empathy, and working with others in a constructive way (socio‑emotional literacy).2

2  Vuorikari Rina, Kluzer, Punie 2022‑DigComp 2.2; Council of Europe 2018; Pegrum, Hockly, Dudeney 2022; Robin 2016; Menegale and Haring, infra.

In this context, research has shown that DST is in line with the aims and principles of multiliteracies pedagogy, which extends the notion of literacy to include “the burgeoning variety of text forms associated with information and multimedia technologies” (The New London Group 1996, 61) and addresses the different forms of knowledge and practices that characterise our multilingual and multicultural world. When creating a DS, students “select and manipulate semiotic resources like images, music, transition effects, and narration” (Kim, Yatsu, Li 2021, 2) and use different remixing strategies (Hafner 2015; see § 3.4.1) to create new meanings and express deep emotions that would otherwise remain unexpressed through only using the verbal mode (Hafner 2015). It is this process of multimodal composing which helps students bridge the gap between in‑ and out‑of‑school experiences and develop their identity (Kim, Yatsu, Li 2021, 1).

In this sense, DST is also an effective tool in the context of Global Citizenship Education (GCE; Unesco 2015). By thinking carefully about what they want to say and finding their own perspective, DST has in fact proven to promote students’ deep learning and self‑reflection (Kim, Li 2021) as well as civic participation (Jiang 2022). For example, through using DST, the lower secondary students involved in Kim and Jia’s (2020) study were not only able to develop a better understanding of mental health disorders, but also to connect with this difficult topic in a personal manner. The analysis of the stories developed by these students shows how they underwent deep interpretative and reflective processes which fostered their empathy towards people suffering from these disorders as well as the reflection on their own emotions and experiences. Finally, when DST is carried out in the context of GCE, it can also help transform students into ‘activists’ (Jiang 2022) as DS are always meant to develop different types of strategies to connect and share with an authentic audience (Hafner 2014).

Despite the affordances of DST briefly outlined above, research has also highlighted the challenges related to producing a DS in an additional language. Both students and teachers agree that DST is a time‑consuming process that requires linguistic, research, and technical skills that need time to be developed and scaffolded (see Lugossy et al., infra). Additionally, while students might be familiar with the use of technology, they often lack the multimodal literacy skills necessary to successfully and critically orchestrate different modalities for meaning making (Pegrum, Hockly, Dudeney 2022; Oskoz, Elola 2022; Fu, Yang, Yeh 2022). Finally, dealing with difficult topics and encouraging students to become civic participants require well thought‑out workshops that allow students a safe space of discussion and self‑expression (Kim, Jia 2020; Jiang 2022) as well as of language development.

2.2 Digital Storytelling as a Multimodal Reader Response to Literature

A less popular, but rising, field of research is DST as a multimodal reader response to literature. The recent introduction of the new literature scales in the CEFR (Council of Europe 2020) has certainly increased both researchers’ and teachers’ interest in how to develop students’ skills of mediating a text. These skills require language students to express different types of response to literature along an increasingly complex cognitive and linguistic continuum, which goes from being able to express the effect of a work of literature on them as individuals (e.g. engagement, interpretation) to being able to give broader and deeper interpretations of such a work (e.g. analysis, evaluation; see Menegale, infra). Multimodal composing has already been established as a powerful tool to promote these skills in the language classroom (Kress 2003). In Malta, for example, Camilleri Grima and Mantellato (2021) asked adult students of Maltese as a FL and language teachers to produce short videos in response to their engagement with a variety of creative texts, including literature. Their findings show that the use of multimodality “was definitely more effective in stimulating reflection and empathy than if they had to rely only on one mode of communication” (280). On the other hand, multimodal composing has also been proven to support learner‑centred approaches to literary texts (Bland 2018), promoting dialogue and reading motivation, fostering the sharing of different perspectives and interpretations (Ludwig 2021), and creating robust responses that extend beyond the text while remaining grounded in it (Dail, Vásquez 2018; Horne 2021).

One multimodal type of composition is DST. For example, Horne (2021) engaged A1‑A2 students of French as a FL in the reading of The Outsider (1942) by Albert Camus and found that students were able to engage more personally in the analysis and interpretation of the novel, changing their reading posture. However, when analysing the stories closely, students appeared to have only concentrated on character’s mental states, failing to incorporate post‑colonial themes. In another study, Hirsch and Macleroy (2020) harnessed the affordances of DST to engage secondary multilingual students with poetry. Their findings show that using DST as a reader response encourages students to bring together “traces from personal stories, experiences, cultures and languages to construct new meanings” (Hirsch, Macleroy 2020, 54), while also helping them to connect and be more kind to each other. However, the two authors also highlighted the challenges encountered in the first few weeks as their students “struggled to grasp the concept that they would be making their own version” of a DS (50). Indeed, despite DST being widely recognised today as a valuable tool in the FL classroom, there is insufficient research on how students orchestrate and remix different resources to create meanings that reflect the text while also extending it (Oskaz, Elola 2022; Kim, Yatsu, Li 2021; Dail, Vásquez 2018).

In light of the above, our research questions are the following:

  1. How did upper secondary EFL students orchestrate and remix across different modes and resources to creatively respond to YAL through DST?

  2. What are the benefits and challenges of using DST in response to YAL in the EFL upper secondary classroom?

3 The Study

3.1 Description of the Digital Storytelling Process

The DST project reported in this article was part of the research carried out within the Erasmus+ project DigLit.3 This project aimed at developing upper secondary students’ multiliteracy and global citizenship skills in English as a FL through the reading of Young Adult (YA) novels and follow up design of a digital story (DS) inspired by the novel they had read. Specifically, in the reading phase of the project, the students selected the novel they were most interested in and engaged in pre‑, during‑ and post‑reading activities on two digital platforms (Glose for Education and Moodle) (see Fazzi, Da Lio, Guzzon, infra). These activities (or better prompts) aimed to stimulate them to reflect on aspects of the novel, draw connections between the text and themselves and between the text and similar events taking place in the world, and become aware of the thematic, structural and formal features of the novel. Each YA novel allowed the students to explore both global issues, such as mental health, LGBTQ+ issues, racism, and other themes related to teenagerhood, such as coming of age and friendship. After the reading, the students were guided to create a reader response in the form of a digital story through a mixture of online and in‑presence sessions designed by the three authors and delivered with the support of the English teachers at the partner schools in Italy and Hungary. In the first online session, the students were introduced to digital storytelling and were encouraged to reflect on the characteristics of a digital story (Lambert 2018) through a guided multimodal analysis.4 Specifically, they were asked to watch a few examples of digital stories and identify both macro features (e.g. length, theme and structure of the story, and the multimedia elements used in the story), and micro features (e.g. the style of English and prosody, the type of music and images used, and the emotions conveyed through their combination). In the follow up in‑presence sessions, the students were asked to brainstorm ideas for their digital story by discussing the following questions about the YA novel they had read:

3 The Erasmus+ project DigLit: Lit. Up Your Phone: A Digital Toolkit for ESL/EFL Classroom to Combat Social Inequalities in Times of Covid 19 Crises was co‑funded by the Erasmus+ Programme of the European Union and the findings of this project are presented in this paper (https://diglit.narrativedidactics.org/).

4 The multimodal analysis sheet can be viewed and downloaded at this link on the DigLit project website: https://diglit.narrativedidactics.org/teacher‑resources.

  • What is the YA novel about? (themes, topics, characters, and moments);

  • Which of these themes/topics/characters/moments might be of inspiration for your digital story?

  • How can you relate them to your own experience?

As part of this brainstorming phase, the students also discussed what type of story (personal, realistic, imaginative) they wanted to create and what message they wanted to convey, how they would work (in group or individually) and what digital application they would use in creating the video. Thus, the students first wrote a script and then completed a storyboard5 which they shared with the other groups through story‑circles6 (Lambert 2018). After completing the storyboard, the students collected or created resources, recorded the voice over, and produced the first version of their DS, which they showed in class to get feedback before working on the final version. At the end of the project, a public event was organised in each of the two partner schools to allow the students to present their digital stories to the school community and their parents, before uploading them to the project website.7

5 The storyboard can be viewed and downloaded at this link on the DigLit project website: https://diglit.narrativedidactics.org/teacher‑resources/.

6 Story‑circle is an activity in which people share their stories before moving onto script writing or making a storyboard and receive feedback from their peers.

7 The digital stories produced by the students during the DigLit project can be viewed at this link: https://unitube.uni‑graz.at/portal/aufzeichnungen.html?epFrom=3b2170b7‑b1b0‑4d39‑be47‑a3456ebcc4ed.

3.2 Participants

While both the Italian and Hungarian students from the partner schools participated in the digital storytelling project, in this study we will only concentrate on the DS, DST process, and perceptions of the Italian students. These were 39 students aged 16‑17 with a B1‑B2 level in English as a FL and were new to the process of creating a DS. The students were selected based on their teachers’ availability to participate in the project and belonged to two classrooms. During the DST project, the students worked in mixed groups of three to four people based on the book they had read.

3.3 Data Collection

The data included (1) the digital stories, (2) the students’ reflective diaries, and (3) classroom observations and artefacts.

3.3.1 Digital Stories

The primary source of data for this study are the 13 digital stories produced by the Italian students in response to the reading of four YA novels:

  • We Were Liars (2014) by Emily Lockart, a psychological horror novel that tackles mental health and wellbeing, specifically portraying the symptoms of post‑traumatic stress disorder (PTSD) as well as the pressure and expectations that come with wealth and privilege (three digital stories);

  • The Maze Runner (2009) by James Dashner, a book that explores the importance of memory for both the individual and society and the difficulties of growing up (three digital stories);

  • The Hate U Give (2017) by Angie Thomas, a novel that was written following the killing of a 22‑year‑old African American by the police and that deals with police brutality and racism (four digital stories);

  • Aristotle and Dante Discover the Secrets of the Universe (2012) by Benjamin Alire Sáenz, a coming‑of‑age novel that tells the story of two young teenage boys grappling with self‑discovery, especially in relation to their ethnic identity and sexuality (three digital stories).

The videos were 3‑5 minutes long and were created using a variety of digital applications (i.e. Canva, Capcut, Clipchamp, Filmora, and Imovie) that allowed the students to combine stock images and music, video clips, and self‑created multimedia sources and effects.

3.3.2 Students’ Reflective Diary

At the end of the DST process, the students were asked to fill in an online reflective diary aimed at collecting their perceptions of the benefits and challenges of designing a DS in English as a FL as well as giving a brief account of their design choices and motives, specifically relating the DS to the YA novel they had read. The online reflective diary was administered as a Google Form consisting of an introductory section, briefly explaining the objective of the diary, a request to indicate the title of the book that had inspired the DS and of the DS itself, and ten open‑ended questions. These questions asked students to explain what they intended to communicate with their story, what aspects of the novel had inspired them and how the story related to them, the design process, the effects of DST on their language and multimodality skills, what they enjoyed, what they found challenging, and what they were most proud of.

The students were asked to complete the reflecting diary within two weeks from the end of the project. While the questions were in English, the students were given a choice to reply in whichever language (English or Italian) they felt most comfortable with.

3.3.3 Classroom Observations and Artefacts

Field notes were collected during the entire DTS process by the second and third author with the aim of recording the students’ brainstorming and discussion of the YA novel, negotiation of design choices during storyboarding and multimedia collection/creation, and issues encountered. Audio recordings, photographs and storyboards were also collected to complement the field notes further enriching the understanding of the students’ multiliteracy skills development during the DST process.

3.4 Data Analysis Procedures

3.4.1 Multimodal Discourse Analysis

Multimodal discourse analysis (MDA) has been used to analyse qualitatively the DS. Accepted modes of communication are writing, the static and dynamic image, sound, speech, gesture, gaze, and posture in embodied interaction (Jewitt 2013, 253).

These modes have been extensively theorised into analytical frameworks for multimodal discourse analysis. For the analysis of the DS produced by the students, we will refer to the model of soundscape (van Leeuwen 1999); for visual analysis, we will refer to the Visual Communication Grammar (Kress, van Leeuwen 2006).8 A soundscape is a composite semiotic system made up of speech, music, and sounds. In the soundscape these elements integrate with each other (van Leeuwen 1999, 4) to define a “meaning potential” (van Leeuwen 1999, 10; emphasis in the original), which can be investigated by means of a set of parameters (‘perspective’; ‘time and rhythm’; ‘interaction of voices’; ‘melody’; ‘voice quality and timbre’; ‘modality’). As for the visual mode, in their Visual Communication Grammar, Kress and van Leeuwen (2006) draw on Halliday’s Systemic Functional Grammar (Halliday, Matthiessen 2004) to describe visuals in terms of three types of meanings. These are ‘representational’ meaning (the way aspects of the world are represented, and representation can be ‘narrative’ or ‘conceptual’), ‘interpersonal’ meaning (the interaction between represented participants and interactive participants), and ‘compositional meaning’ (the way items are placed and composed in images).

8 The analysis of gestures will not be carried out as it is not applicable to the DS. Indeed, the students never appear in the videos as characters of the story, with one exception only (DS Learn from your mistakes: https://unitube.uni‑graz.at/portal/aufzeichnungen.html?id=de6b1302‑86bd‑45d8‑aa46‑5349663e37e9.

Also relevant to this study is ‘remix’ as discussed by Hafner (2015) and Kim, Yatsu, Li (2021). In his study, Hafner (503‑4) identifies four types of remixing techniques: layering (or ‘mixing modes’), chunking (or ‘mixing resources’), blending (or ‘mixing generic resources’), and intercultural blending (or ‘mixing cultural resources’). Layering is the integration of distinct modes of communication; chunking is the combination of students’ multimodal artefacts with those created by others; blending is the combination of different genres to create a hybrid blend; intercultural blending is the combination of elements from both local and global cultures.

As far as the methodology is concerned, the DS produced by the students are so diversified that manual, non‑software‑aided annotation was deemed the most useful method to identify how the different semiotic resources have been used and combined.

3.4.2 Qualitative Data Analysis

The reflective diaries and classroom field notes were analysed following Miles, Huberman and Saldana (2014) thematic analysis. We first compiled a start list of possible codes derived from the literature on digital storytelling (see §§ 2.1 and 2.2), focusing on the students’ design choices as related to the development of their language, global citizenship, multiliteracy, and mediation skills as well as on their perceived benefits and issues of DST. Data was first coded using this initial list of codes, which was expanded to include emergent codes, such as ‘difficulties in relating to the text’. During the second round of coding, we worked at a more abstract level, combining the codes identified during the first round into categories that represented theoretical constructs. The data collected through the students’ reflective diaries and classroom observations were analysed by the second author and triangulated (Dörnyei 2007) with the multimodal analysis of the digital stories carried out by the first author. This helped us in our understanding of the data and discuss any discrepancies in our interpretation.

4 Results

We will begin the analysis by providing an overview of the main features of the DS focusing on narration modes and organisation of contents. Then, a few selected excerpts will be analysed in detail to show how the different semiotic resources are combined for meaning making. Finally, the qualitative analysis of the students’ reflective diaries and classroom observations and artefacts will be provided.

4.1 General features of the DS

In all the DS, the narrators are the students with their voices alternating usually at turning points in the narration. In terms of remixing (Hafner 2015), all the DS are realised through layering, but a few also involve chunking and blending. The most frequent structure of the DS is the following:

  • part 1: presentation of the topic and key points;

  • part 2: development of the key points;

  • part 3: call to action/conclusion.

In part 1, the topic is introduced by means of a general statement, which is then followed by a series of reflections upon the key issues that the students decided to narrate. At this reflective stage the inclusive ‘we’ pronoun is frequently used, plausibly because, being teenagers themselves, the students feel directly involved in the narrated themes and want at the same time to involve the audience. At the visual level, this introductory part is frequently supported by static pictures that semantically match the verbal message. The pictures can be associated with the representational meaning (Kress, van Leeuwen 2006) and are mostly narrative pictures, as they often contain participants carrying out actions in groups. In terms of interpersonal level, these groups of people sometimes gaze directly at the viewer, thus increasing involvement, while in others they do not. In part 2, the development of the key points raised in part 1 takes place in different ways. In a few cases, the students embedded YouTube videos in their DS (chunking). When dynamic pictures are not included, static pictures are used with more or less direct connections with the verbal message. In part 3, the conclusion usually involves a call to action in the form of advice, rhetorical questions or proposed solutions supported by the visual component. At this stage there is a clear – and probably conscious – intention to leave the viewer with some food for thought and to convey some practical intent.

At the verbal level, other features that have been noticed include significant quotes from the book, which in some cases appear on the screen and are read by the speaker, the use of rhetorical questions addressing directly the viewer (e.g., “Have you ever thought …”), the use of ‘mental processes’ (Halliday 2004), i.e. invitations that are not aimed at material action but that rather operate at the cognitive level (e.g., “Imagine waking up and not realising…”).

As for soundscape, non‑diegetic music9 usually accompanies the narration. The preference is mainly for calm and melancholic piano music, but other genres sometimes are also involved depending on the emotions to be conveyed. Furthermore, specific sounds are used to produce emotional effects.

9 Diegetic music is used only in the DS Could this be your story, which relates to the novel The Hate U Give.

Expectedly, the DS are not flawless. The most common inaccuracies involve mispronunciation, excessive use of external material, filling the slides with the whole verbal message uttered by the speaker, music being too loud during the narration, or a non‑relevance relation between the uttered verbal message and the visuals.

4.2 Multimodal Analysis of Selected Excerpts

Due to space constraints, the analysis will be focused on a selection of fragments from the DS that were deemed most representative of the students’ multimodal response to YAL. Since we are interested in the students’ own multimodal production, the excerpts of external videos that the students embedded in their DS will not be analysed.

4.2.1 Social Pressure – the Witch in Our Life

This DS relates to the novel We were liars, which is about post‑traumatic stress disorder (see § 3.3.1).10 The main character is Cadence Sinclair, the eldest grandchild of the wealthy Harris Sinclair, owner of Beechwood Island (off the coast of Massachusetts), where the Sinclairs spend their summers in a house called Clairmont. Although her family pretends to be perfect, her parents and relatives continuously bicker over inheritance. In the summer when she was fifteen, Cadence has a mysterious accident that causes her to have gaps in memory and gives her migraines. She spends the next two years trying to solve the mystery of what really happened to her that night. She finally remembers that she had persuaded her cousins and friends (the Liars) to start a fire to destroy Clairmont, but the Liars themselves died in the fire.

With chapter 16, Cadence begins interspersing fairy tales among the chapters. In the DS here analysed, the students have basically multimodally rewritten the fairy tale about the three children and the envious witch. Social pressure is visually represented by a witch that enters teenagers’ lives and deceives them, leading them to mental health issues. Mental disruption is symbolised by a stylised dynamic picture of fire, which in turn refers to the fire set by the Liars in the book. The story is narrated by the voices of the three female students, who are non‑diegetic narrators alternating at turning points in the story and voicing the witch by means of direct speech. The plot takes place as follows:

  1. Introduction: the definition of social pressure is provided;

  2. Exposition: three beautiful children are born, and they are gifted with virtues by fairies;

  3. Rising action: the witch is introduced; she is envious of the three children and wants to take their gifts away;

  4. Climax: the witch visits the children on their 16th birthday, and persuades them to set up fire. All their gifts disappear in the fire;

  5. Falling action: call to action and advice.

Soundscape plays a key role in this DS. The sound of heartbeat is used throughout the DS and its pace changes according to the phase of the story. It is fast when the definition of ‘social pressure’ is provided – plausibly to emotionally convey the anxiety produced by pressure – and at the key moment of the climax (from 00:02:35), when the fire consumes the gifts of the children. In terms of perspective – i.e., the relative loudness of simultaneous sounds (van Leeuwen 1999, 23) – the heartbeat increases in loudness as the narrator lists the gifts that are being destroyed by the flames, thus increasing the emotional load. The moment at which the children are completely void of their gifts is marked by the sound of a flatline, which is commonly associated with clinical death.

Another interesting multimodal combination can be detected at the beginning of the climax. The static picture of the witch holding a sceptre with a raven on it – with the raven conveying an omen of death – matches the words uttered by the witch when she persuades the children to light the match. The repetition of “Go on, she said” for three times is meaningful here, as it stresses how disruptive social pressure is and conveys the feeling of anxiety that affects teenagers.

4.2.2 Friendship or Survival?

This DS relates to the novel The Maze Runner.11 The story is set in a distant future, where a group of teens are mysteriously teleported into a giant, stone maze. Among them is Thomas, the main character of the story. The teens’ memories have been erased, and they have no indication as to how to escape. To go back home, they must solve the maze, and this will involve them in difficult challenges and conflicts.

In terms of ‘remixing’ (Hafner 2015), not only layering but also chunking has been used here, as the students embedded YouTube videos containing excerpts from the book‑related movie.12 They decided to focus on the part of the story in which Ben has become dangerous for the safety of the group and a decision needs to be made about whether to banish him in the maze or save him (from 00:00:29). Dynamic pictures from the film were layered with a soundscape characterised by ‘plurality’ in interaction of voices (van Leeuwen 1999, 84): suspense‑generating, rhythmic music in the background and the students’ voices in the foreground alternate in a dialogue that effectively symbolises the ‘friendship or survival?’ conflict. The students produced the dialogue script by themselves to impersonate two characters of the story: one wanting to save Ben, the other making the safety of the group prevail over friendship‑related feelings. Thus, diegetic narration here increases emotional impact and viewer involvement.

12 Maze Runner (2014), directed by Wes Ball.

4.3 Analysis of Reflective Diaries and Classroom Observations

The analysis of the students’ reflective diaries shows that by producing digital stories, the students felt to have developed different literacy skills: language, digital, multimodal, and creative. As regards language, the students claimed to have increased their vocabulary, pronunciation, and writing, but also their confidence in speaking and engaging with media and resources in English. In fact, they stated that writing the script forced them to look for the right words to fulfil their communicative intention in the most accurate way possible, while recording their voice multiple times encouraged them to work on their pronunciation, which in turn helped them feel more confident in speaking in English. On the other hand, having to browse through resources and using tools in English also gave them the feeling they were using English authentically and not simply as a learning tool.

As regards digital and multimodal literacy skills, the students claimed to have increased their ability to use editing applications and combine different modes to construct a coherent and powerful story. Creativity was also among the positive outcomes identified by the students: many affirmed that both creating the script and producing the video required considerable experimenting to find effective communicative and technical solutions. For example, a student working on Could this be your story? claimed what follows: “We had to get creative in order to film the different clips because sometimes we messed up something little and we had to re‑do all; about the speech we had to change our way of speaking depending on the character we were playing”.

As for the positive aspects of the DST process, the students claimed to have particularly enjoyed both the collaborative and more technical aspects of the project. Working in groups and sharing their products with their peers and with the school/parent community were deemed as joyful and empowering moments. At the same time, reflecting on the topic/book and creating the videos were also seen as important positive aspects of the project from the point of view of collaboration. Here are a couple of significant comments:

  1. “I especially liked working in a group and collaborating with my classmates”;

  2. “I liked most the part when we all together saw the video”.

As for negative aspects, students claimed to have encountered difficulties in the brainstorming phase to find the right idea to convey especially considering the short length of the video, as well as in the production phase, when searching for the right images and sounds and learning how to use the editing tools. Also, most students agreed that creating the DS was time‑consuming and added to their study workload. However, the students were generally proud of their final product and of being able to tell a story “from beginning to end” with a strong emotional content delivered through the sensible orchestration of different modes and an original script. Several students, in fact, highlighted that their video was “completely” their doing, demonstrating a sense of ownership and empowerment.

The analysis of the classroom observations demonstrates that the DST process allowed the students to practise their mediation skills in line with the new scales relevant to creative texts and literature (Council of Europe 2020). In fact, in the pre‑production phase, they engaged in long discussions about the main themes, characters, and episodes of the novels, and how these related to their personal experience. An exemplary comment is the following:

  1. “Like the characters, we have to face difficult situations, we have friends, and we find ourselves in situations where we have to convince others of our opinion and step out of our comfort zone”.

Furthermore, discussing important passages and extracting key‑quotes was found to be a difficult process, too. In the field notes, the second author noticed that the students had at first a hard time remembering what the novels were about. This is due to the long break between the reading phase and the DST sessions, but also to the fact that the students did not have physical copies they could browse but only the digital versions uploaded on Glose for Education. The analysis of the classroom observations and of the DS (especially the storyboards) also shows the high level of critical reasoning behind every design choice. The student group working on Social pressure – The witch in your life, for instance, had a long discussion about the role of sound in their story and how the change in the pace of the heartbeat was important to signal the different phases in the story. As for the storyboards, while some of the first drafts were not very detailed, others appeared much richer in terms of visualising and defining the students’ production process. Finally, during the DST sessions, the students referred often to the guided multimodal analysis conducted at the beginning of the project showing an awareness of the different macro and micro features of a DS.

5 Discussion

Our first research question asked how upper secondary EFL students orchestrate and remix across different modes and resources to creatively respond to YAL through DST. Our findings demonstrate that students were able to structure their stories following Lambert’s (2018) story arc (beginning, middle, and end) and including the main characteristics of a DS (Robin 2016). Students also creatively experimented across different multimodal genres (e.g. fairy tale, book teaser, documentary, poetry), effectively using and combining different multimodal resources to reflect the thematic, structural, and formal features of the novel as well as create new personal meanings. Many of the DS included important quotes from the YA novels (e.g. Could this be your story?) or reinterpreted a scene from the novel (e.g. Friendship or survival) or enhanced/transformed an aspect of the book to communicate their chosen message (e.g. Social pressure – The witch in our life). This demonstrates the potentiality of DST as a digital mediation task (Council of Europe 2018) that unlocks students’ imagination and meaning‑making through multimodality and transmediation.13 In fact, students had to work across different sign systems to build their own interpretation of the novel, thinking carefully about the associations between the written text and the other modes, which is intensive design work (Kress 2003; Kesler, Gibson, Turansky 2016). In so doing, they also used a variety of remix strategies creating stories that (i) remain recognisable has having the assigned work of literature as its source material, (ii) alter/add to advance the story substantially by transforming the literature, (iii) take risks and push boundaries to present a creative product, (iv) integrate multiple modes of composition, and (v) communicate the message clearly (see Jenkins 2013 quoted in Dail, Vásquez 2018, 95). The use of these strategies shows how DST can provide a complex aesthetic experience, encouraging students to stay true to the text while also creatively playing with their interpretations, that integrate both personal and critical perspectives (Kesler, Gibson, Turansky 2016). In this context, working collaboratively on a DS transformed students into co‑inquirers of topics, themes, and other issues dealt with in the novels in line with the mediation scales of the Companion Volume (Council of Europe 2020). Additionally, students creatively used different discourse features (e.g. “Who said talking isn’t doing something?”) in the target language as “rhetorical hooks” (Hafner 2014, 655) showing an understanding of how to use language to appeal to the intended audience. Finally, most of the stories presented a call to action, confirming that DST can encourage students to become civic participants (Kim, Jia 2020; Jiang 2022). However, students’ DS were not exempt of problems, which highlights the need to offer them more opportunities to rehearse the script multiple times to both work on their pronunciation and enunciation skills (Oskoz, Elola 2016b), and develop digital and multimodal literacy skills to better orchestrate the different modes (Pegrum, Hockly, Dudeney 2022).

13 Transmediation is “the translation of content from one sign system into another” and it “is fundamental to meaning‑making” (Mills 2011, 56).

In response to the second research question (What are the benefits and challenges of using DST in response to YAL in the EFL upper secondary classroom?), our findings confirm previous research on the benefits of DST for language students in terms of developing language (Oskoz, Elola 2014, 2016a) and multiliteracy skills, including digital and multimodal (Wake 2012), creative and critical (Yang, Chen, Hung 2020), and socio‑emotional (Council of Europe 2018). Indeed, one of the things that students most appreciated was the opportunity to work with their peers, which scaffolded their learning (sociocultural theory; see Oskoz, Elola 2022) and allowed them to overcome problems (Hirsch, Macleroy 2020) developing a sense of belonging to a community (socio‑emotional skills; Menegale, infra). Students also appreciated the guided multimodal analysis carried out at the beginning of the DST process, confirming Pegrum, Hockly, and Dudeney’s (2022) claim according to which students need guidance to understand how to exploit the affordances of different technological tools to create efficient multimodal ensembles. Interestingly, students also felt that working on the DST gave them the chance to use English authentically through engaging with both resources and digital applications in English (Christiansen, Koelzer 2016).

Our analysis of classroom observations and artefacts also shows that students developed their global citizenship and mediation skills. Indeed, students deeply reflected on and discussed aspects related to the global issues dealt with in the novels, building connections between the novel, themselves, and the world, examining relations of power, and learning to question and act (Bland 2018). At the same time, they also articulated their different responses to the YA novels, explaining what they liked about the literary text and what they thought were the most interesting aspects, describing characters, giving personal interpretations, and commenting on the form of expression and style (Council of Europe 2020). However, our findings also highlight some challenges, such as the time‑consuming nature of DST and the difficulty to work on the digital formats of the novels. Concerning this last issue, Pegrum, Hockly, and Dudeney (2022, 13) argue that when students are engaged in long‑form reading, “it is easier to build a cognitive map of a text when turning the pages of a three‑dimensional book than when scrolling on a flat screen, in part because information becomes encoded in memory along with its spatial location”. As in the first phase students read digital copies of the novels (see Fazzi, Da Lio, Guzzon, infra), locating relevant information in the text for the design of their DS might have proven difficult. More research is needed to understand the impact of reading digital copies on the DST process.

6 Pedagogical Implications and Conclusion

Our study shows that DST can be an invaluable tool to promote both traditional and new literacy skills and to foster students’ global citizenship skills in a FL. It also highlights some pedagogical implications. First, the importance of providing students with multimodal analysis workshops in which they have the chance to analyse different multimodal compositions and discuss how different modes are used to create new meanings (Gregory‑Signes 2014). Second, the need to give students ample time to rehearse their scripts, working on their pronunciation and enunciation (Oskoz, Elola 2016b). Third, the need to offer them technical support with technology, without limiting their options in choosing the digital applications they want to work with (Christiansen, Koelzer 2016).

Our research also shows that students developed their mediation of text skills in the target language, using different remixing strategies (Hafner 2015) and genres to develop digital narratives that transmediated the source book. The use of DST as a multimodal reader response to literature is an interesting development and should be explored further especially in terms of how to guide students’ transmediation process in line with the CEFR (Camilleri Grima, Mantellato 2021).

Bibliography

Abidin, M.J.Z.; Pour‑Mohammadi, M.; Souriyavongsa, T.; Tiang, C.D.B.; Kim, N.O.L. (2011). “Improving Listening Comprehension Among Malay Preschool Children Using Digital Stories”. International Journal of Humanities and Social Science, 1(14), 159‑64.

Bland, J. (ed.) (2018). Using Literature in English Language Education: Challenging Reading for 8‑18 Year Olds. London: Bloomsbury Publishing.

Camilleri Grima, A.; Mantellato, M. (2021). “Expressing a Personal Response to a Creative Text in the Maltese as a Foreign Language Class”. Malta Review of Educational Research, 15(2), 269‑91.

Castañeda, M.E. (2013). “I Am Proud that I Did It and It’s a Piece of Me: Digital Storytelling in the Foreign Language Classroom”. CALICO journal, 30(1), 44‑62.

Christiansen, M.S.; Koelzer, M.L. (2016). “Digital Storytelling: Using Different Technologies for EFL”. MexTESOL Journal, 40(1), 1‑14.

Council of Europe (2018). Council Recommendation on Key Competences for Lifelong Learning. https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32018H0604(01)

Council of Europe (2020). Common European Framework of Reference for Languages: Learning, Teaching, Assessment. Companion Volume. https://rm.coe.int/common‑european‑framework‑of‑reference‑for‑languages‑learning‑teaching/16809ea0d4

Dail, J.S.; Vásquez, A. (2018). “Seeing the World Differently: Remixing Young Adult Literature Through Critical Lenses”. Dail, J.S.; Witte, S.; Bickmore, S.T. (eds), Toward a More Visual Literacy: Shifting the Paradigm with Digital Tools and Young Adult Literature. Lanham: Rowman & LIttlefield, 91‑100.

Dörnyei, Z. (2007). Research Methods in Applied Linguistics. Oxford: Oxford University Press.

Fu, J.S.; Yang, S.H.; Yeh, H.C. (2022). “Exploring the Impacts of Digital Storytelling on English as a Foreign Language Learners’ Speaking Competence”. Journal of Research on Technology in Education, 54(5), 679‑94.

Gregori‑Signes, C. (2014). “Digital Storytelling and Multimodal Literacy in Education”. Porta Linguarum, 22, 237-50.

Hafner, C.A. (2014). “Embedding Digital Literacies in English Language Teaching: Students’ Digital Video Projects as Multimodal Ensembles”. TESOL Quarterly, 48(4), 655‑85.

Hafner, C.A. (2015). “Remix Culture and English Language Teaching: The Expression of Learner Voice in Digital Multimodal Compositions”. TESOL Quarterly, 49(3), 486‑509.

Halliday, M.A.K.; Matthiessen, C.M.I.M. [1985] (2004). An Introduction to Functional Grammar. 3rd ed. London: Arnold.

Hava, K. (2021). “Exploring the Role of Digital Storytelling in Student Motivation and Satisfaction in EFL Education”. Computer Assisted Language Learning, 34(7), 958‑78.

Hirsch, S.; Macleroy, V. (2020). “The Art of Belonging: Exploring the Effects on the English Classroom When Poetry Meets Multilingual Digital Storytelling”. English in Education, 54(1), 41‑57.

Horne, F. (2021). “The Digital Story as a Reading Revisiting Camus’ The Outsider Response to the Literary Text in the French Foreign Language Classroom”. Carloni, G.; Fotheringham, C.; Virga, A.; Zuccala, B. (eds), Blended Learning and the Global South: Virtual Exchanges in Higher Education. Venezia: Edizioni Ca’ Foscari.

Huang, H.T.D. (2023). “Examining the Effect of Digital Storytelling on English Speaking Proficiency, Willingness to Communicate, and Group Cohesion”. TESOL Quarterly, 57(1), 242‑69.

Jewitt, C. (2013). “Multimodal Methods for Researching Digital Technologies”. Price, S.; Jewitt, C. (eds), The Sage Handbook of Digital Technology Research. London: Sage Publishing, 250‑65.

Jiang, L. (2022). “Facilitating EFL Students’ Civic Participation Through Digital Multimodal Composing”. Language, Culture and Curriculum, 35(1), 102‑17.

Kesler, T.; Gibson Jr, L.; Turansky, C. (2016). “Bringing the Book to Life: Responding to Historical Fiction Using Digital Storytelling”. Journal of Literacy Research, 48(1), 39‑79.

Kim, D.; Jia, F. (2020). “Ever Wondered What Schizophrenia Was?: Students’ Digital Storytelling About Mental Disorders”. Journal of Curriculum Studies Research, 2(2), 144‑69.

Kim, D.; Li, M. (2021). “Digital Storytelling: Facilitating Learning and Identity Development”. Journal of Computers in Education, 8(1), 33‑61.

Kim, D.; Yatsu, D.K.; Li, Y. (2021). “A Multimodal Model for Analyzing Middle School English Language Learners’ Digital Stories”. International Journal of Educational Research Open, 2, 1‑9.

Kress, G. (2003). Literacy in the New Media Age. London: Routledge.

Kress, G.; van Leeuwen, T. (2006). Reading Images: The Grammar of Visual Design. 2nd ed. London: Routledge.

Lambert, J. (2018). Digital Storytelling: Capturing Lives, Creating Community. 5th ed. Oxon: Routledge.

Ludwig, C. (2021). “Teaching Literature with Digital Media”. Lütge, C.; Merse, T. (eds), Digital Teaching and Learning: Perspectives for English Language Education. Tübingen: Narr Francke Attempto Verlag, 207‑30.

Miles, M.B.; Huberman, A.M.; Saldana, J. (2014). Qualitative Data Analysis. A Methods Sourcebook. Thousand Oaks: Sage.

Mills, K. (2011). “I’m Making It Different from the Book”: Transmediation in Young Children’s Multimodal and Digital Texts”. Australasian Journal of Early Childhood, 36(3), 56‑65.

Oskoz, A; Elola, I. (2014). “integrating Digital Stories in the Writing Class: Toward a 21st‑Century Literacy”. Guikema, J.P.; Williams, L. (eds), Digital Literacies in Foreign and Second Language Education. San Marcos (TX): Computer Assisted Language Instruction Consortium (CALICO), 179‑200.

Oskoz, A.; Elola, I. (2016a). “Digital Stories: Bringing Multimodal Texts to the Spanish Writing Classroom”. ReCALL, 28(3), 326‑42.

Oskoz, A.; Elola, I. (2016b). “Digital Stories: Overview”. CALICO Journal, 33(2), 15-173.

Oskoz, A.; Elola, I. (2022). “Digital Storytelling in SLA”. Ziegler, N.; González‑Lloret, M. (eds), The Routledge Handbook of Second Language Acquisition and Technology. Abingdon: Routledge, 258‑71.

Pegrum, M.; Hockly, N.; Dudeney, G. (2022). Digital Literacies. New York: Routledge.

Robin, B.R. (2016). “The Power of Digital Storytelling to Support Teaching and Learning”. Digital Education Review, 30, 17‑29.

Tanrıkulu, F. (2022). “Students’ Perceptions About the Effects of Collaborative Digital Storytelling on Writing Skills”. Computer Assisted Language Learning, 35(5‑6), 1090‑1105.

The New London Group (1996). “A Pedagogy of Multiliteracies: Designing Social Futures”. Harvard Educational Review, Spring, 60‑93. https://doi.org/10.17763/haer.66.1.17370n67v22j160u

UNESCO (2015). Global Citizenship Education: Topics and Learning Objectives. https://unesdoc.unesco.org/ark:/48223/pf0000232993

van Leeuwen, T. (1999). Speech, Music, Sounds. London: Macmillan.

Vuorikari Rina, R.; Kluzer, S.; Punie, Y. (2022). “DigComp 2.2: The Digital Competence Framework for Citizens‑with New Examples of Knowledge, Skills and Attitudes”. https://publications.jrc.ec.europa.eu/repository/handle/JRC128415

Wake, D.G. (2012). “Exploring Rural Contexts with Digital Storytelling”. Rural Educator, 33(3), 23‑37.

Yang, Y.T.C.; Chen, Y.C.; Hung, H.T. (2020). “Digital Storytelling as an Interdisciplinary Project to Improve Students’ English Speaking and Creative Thinking”. Computer Assisted Language Learning, 35(4), 840‑62.


Se trovi interessanti le nostre pubblicazioni e vuoi ricevere aggiornamenti sulle prossime uscite, iscriviti ora alla nostra newsletter.

x

Newsletter