Using Cartoons to Assess Interpretive Listening with Novice Learners

COUV. La Balle.indd

This week’s #langchat discussion about interpretive listening revealed that we language teachers are very diverse in the way we approach this skill, especially with novice learners. Although I reflected at length on the topic of assessing listening in an earlier post, I’d like to specifically address a few of the questions that came up during Thursday night’s discussion.

Question #1: What resources are appropriate for novice learners? While some teachers are hesitant to use authentic resources with early novices, I have found that first semester French 1 students can successfully interpret carefully selected authentic materials when given level-appropriate tasks.  My go-to resource for these students are cartoon videos for the following reasons:

  1. These videos were made for novice language learners—young children in the target culture! As a result, the vocabulary and sentence structures are relatively simple and the linguistic input is supported by strong visual cues. This is exactly what our novice learners need.
  2. The wide selection of these videos ensures that there are several choices available for any theme we’ve included in our novice curriculum. My favorites for my Level 1 and 2 students are Trotro, Petit Ours Brun and T’choupi et Doudou, because of the broad range of topics covered and the comprehensibility. I also occasionally use Peppa Pig with my level 2 students. Although originally recorded in (British) English, the French translation was clearly intended for French-speaking children, so I do consider these to be authentic resources.  However, the target culture would not, of course, be represented in these videos.
  3. Cartoons are very engaging to my students. They look forward to their turn at the computer and a few students have even mentioned that they have watched additional episodes of the series at home, “just for fun.”
  4. As authentic resources, these cartoon videos often integrate cultural products, practices and perspectives of the target culture. When Petit Ours Brun puts his shoes under the Christmas tree, his grandfather comments on the delicious turkey, and he wakes up to presents on Christmas morning, my students learn relevant cultural practices regarding Christmas celebrations in France.

Question #2: What types of tasks are appropriate for novice learners? I realized as I participated in Thursday night’s #langchat that I have interpreted ACTFL’s descriptors regarding interpretive listening differently than many of my colleagues. The Novice Mid (my goal for level 1) NCSSFL-ACTFL Can-Do Benchmark for interpretive listening reads, “I can recognize some familiar words and phrases when I hear them spoken.”  If I understood my colleagues’ responses correctly, many of us may be assessing listening by having students list the words and phrases that they hear.  Because it isn’t clear to me how this type of task would demonstrate interpretation/comprehension, I ask students to answer questions to show comprehension of the video, but phrase these questions in a way that the students can use previously-learned words/phrases (along with visual context clues) to respond.  This year I am using a multiple choice format for my formative listening assessments using our district’s recently-adopted Canvas learning management system.  Although I don’t feel that multiple choice is appropriate for many language tasks, this platform has the advantage of providing immediate feedback to my students.  In addition, since creating and assessing them requires a minimal time commitment on my part, I am able to provide more opportunities for listening than I was using other task types.  Lastly, this format provides students with additional context clues.  Their listening is more purposeful as they are listening for a specific response, as well as to eliminate distractors. While I typically use open-ended question types on my IPA’s, these multiple choice quizzes, which the students complete individually at a computer, provide the majority of my formative listening assessments.

In order to save time, I create these quizzes directly in Canvas, which unfortunately makes them very difficult to share.  For the purposes of this discussion, I’ve uploaded a Word document of screenshots from a quiz I made this morning for the video, Trotro et les cadeaux de Noel (https://www.youtube.com/watch?v=iRcv1pVaitY ). As this document shows, the questions that I’ve created enable these Novice Low-Mid students to demonstrate their ability to interpret this text using only previously-learned words and phrases and visual clues. While most of the items assess literal comprehension, I’ve included a few questions that require the students to make inferences and guess the meanings of new words using context clues. Here’s a quick explanation of my thought process for each question.

#1: While each of these questions would be appropriate to the context, my students will probably understand “pour moi” when they hear it.  They will also be able to eliminate the 2nd choice, because they know the word for Santa.  Although I’ve used the other question words in class, the students are not using them yet.  I included them in the distractors to encourage the students to start thinking about how questions are asked.

#2: This question is a “gimme.”  The students know the word for book and have visual clues as further support.  I created the question to improve the students’ confidence, enable all students to have some “correct” answers, and to provide more context for further questions.  As you can see, I write LOTS of questions, because I find the questions themselves provide important context and help the students follow along with the video.

#3: “Chouette” is a new word for these students, but it appears in a lot of children’s literature/videos and I think they’ll enjoy using it.  The context should make the meaning of this word clear.

#4/#5: The students have learned the word “jeux-video” so I think they’ll get “jeu.”  Also because Trotro also uses “jouer” I think they’ll understand it’s something to play with rather than listen to.

#6/#7 Students can answer by recognizing the previously-learned words “gros” and “belle.”

#8: Although this question does not assess listening comprehension (the word appears in written form), it does provide a contextualized way to introduce a new vocabulary word.

#9: The students can listen for the word “content” as well as eliminate the distractors based on previously-learned words.

#10: The students have heard “maintenant” repeatedly, but it hasn’t been formally introduced.  If they don’t recognize it, they should still be able to eliminate the other choices.

#11: Although the students will not understand the entire sentence in which it appears, they should be able to answer this question by identifying the word “cadeaux.”

#12: I’m curious what my students will do with this inference-based question.  They should recognize the phrase, “Moi, aussi” which should enable them to infer that Boubou got the same gift.

#13: The students should recognize the word “jouer” as well as be able to eliminate the distractors based on previously-learned vocabulary.

#14: The students should be able to use the visual context to guess the meaning of this new vocabulary.

#15: The phrase “c’est moi” should enable the students to choose the correct response for this one. As with several other items, I’ve included the transcription of the entire sentence to introduce new vocabulary—the verb “gagner.”

#16: Although my students won’t be able to use the linguistic content to answer this question, I’ve included it to encourage inference based on visual context clues.

#17: I’ll be curious how they do with this one.  “Bateau” is an unknown word and although they’ve seen “mer,” I’m not sure they’ll pick up on it.  Some might pick out “pirate” but I’ll be curious how many are able to answer this one correctly.

#18: The students have heard “rigolo” and this word even appears in Trotro’s theme song.  In addition, they should be able to eliminate the distractors based on previously-learned vocabulary.

While there’s nothing especially innovative about this assessment format, after completing many similar tasks during their first semester of language study most of my level 1 students are pretty accurate when completing this type of formative assessment.

Question #3: How should interpretive listening be assessed? I did want to make a point about grading these formative assessments.  Although I do my best to create questions that are mostly at the students’ current proficiency level, with a few items thrown in to encourage “stretch,” I rely heavily on my students’ results to determine how close I came to hitting this target.  Therefore, I do not decide how to grade these assessments until I have data on how the class scored as a whole.  In other words, this particular formative assessment will not necessarily by worth 18 points.  If, for example, the highest score is 16, I might make this the maximum score. For teachers that do not record a score on formative assessments, this isn’t an issue of course.  I only suggest that we expect and allow for student errors when assessing interpretive listening (even using objective evaluations) just as we do when assessing the other modes.

I’d love to hear from any of you who are willing to share your experiences and ideas about assessing listening with novice learners!

Image credit: www.gallimard-jeunesse.fr

11 thoughts on “Using Cartoons to Assess Interpretive Listening with Novice Learners

  1. Rebecca

    This is my dream come true, Lisa – that you write a detailed #langchat post-mortem about how YOU do it. Thank you so much! I started using Trotro last year thanks to your blog, and it’s been a hit with students and my own young French-speaking children. I’ve shied away from asking as many questions as you do, in part because I still have students who are struggling to read & write – at all! – during interpretive listening. Once we get iPads next year and kids can watch at their own pace and replay, I hope to be able to ask more. Your point about determining the grading curve only after having collected the assessment is key in my mind. I am not yet very good at anticipating which questions will be too hard for students, so this is my “out” to rejigger things as needed.

    Reply
    1. madameshepard Post author

      Hi, Rebecca. Thanks for the sweet comment! #langchat always inspires me, and I’m just too long-winded to be able to express much in 140 characters. I do think it’s pretty difficult to do much with listening as a whole group activity. I do play videos to the class, but I usually just pause and ask questions. It’s only the top kids who are able to answer and I lose a lot to a lack of attention. As for grading, once we turn are mindset to “What did they get right?” rather than “What did they get wrong?” we are setting the scene for a greater focus on proficiency. Thanks again, Lisa

      Reply
  2. Cecile

    Thank you Lisa for this clear and detailed explanation of how you assess listening (and Canvas sounds really amazing!). I completely agree with your answer to Question #1: carefully chosen authentic sources, used with lots of comprehensible input and the appropriate scaffolding is compelling for novice learners. I LOVE your point about how authentic resources expose students to culture! On Question #2, I would like to brainstorm out loud with you for a minute. I think there are three main forms of interpretive listening assessments for Novice learners: 1. Can students recognize words/phrases they know 2. Can they comprehend (some) meaning and 3. Can they make (some) language and/or cultural inferences. So, I guess a good assessment should have all three components. I think my assessments tend to heavily focus on 1. and 2. (here is an example: goo.gl/0lw9qE). It seems that your assessment focuses more on 2. and 3. So if we merge our assessments, we have a winner, right? MDR. Having said that, a novice learner gets easily overwhelmed with listening so maybe not assessing all three components each time does make more sense. A suivre… Finally, on Question #3, I am a firm believer in using standard based learning and holistic grading. Again, I completely agree with you that we should expect and allow errors. To me a student who has answered most of the questions correctly meets expectations. On an assessment like the one you shared, I do not even use points (in fact I have stopped using points completely since adopting standard based grading practices), I go with A+: all answers are correct (Stretch); AA-: Most answers are correct (Meets expectations); B+BB-: Some answers are correct (approaches expectations); C+CC-: A few answers are correct (Not yet = in my world that means a redo of the assessment). Thank you for letting me clog your blog with gigantic comments 🙂 Awaiting your reply.

    Reply
    1. madameshepard Post author

      Thank you so much for these meaningful comments–these conversations are exactly the reason why I started a blog in the first place! Thanks, too, for sharing the video and corresponding assessment. What a great resource for novices! I’d love to know more about the distinction between recognizing words they know and comprehending some meaning. I’m just having trouble understanding whether a task which assesses only recognition (w/o meaning) is actually interpretive. For example, if I had a list of Italian words and listened to an Italian recording, I could probably check the words when I heard them, without understanding what any of them meant. It seems like this type of task could be a good advance organizer, I’m just not sure it indicates comprehension/interpretation. In other words, I’m still a Novice 0 in Italian, even if I make a connection between a written and aural form of a word. As I reflect, I can certainly see value in starting with siimple identification and I might include more of these items when we go 1:1 next year. Right now, I am rotating students in and out of the 8 computers in my room, so each student only has 10-15 minutes at the computer. I try to develop tasks that don’t require several complete run throughs. You’ve given me a lot to think about!
      As for the SBG, this makes perfect sense to me. I don’t usually use multiple choice quizzes for summatives/IPA’s. Although I create similar questions, the students provide a short answer. I score these similarly to you, except that I use numbers for the gradebook. I make each task out of 10, so my grades would correspond to yours like this: 10 (A+), 9 (A/A-), 8 (B), 7 (C), etc. The main point I wanted to make about the multiple choice quizzes is that just because a teacher ends up with a numerical score, it doesn’t mean that it would be appropriate to use the maximum possible correct answers as the “A” range. We’re clearly on the same page here!
      Thank you again for taking the time to leave such detailed comments and I look forward to continuing to learn from you!

      Reply
  3. Cecile

    Lisa, I am really enjoying this exchange, especially since, as you know, I just left Ohio. I miss colleagues such as you so much (even if we still have not met)! I think you are saying: “what’s the point recognizing words/phrases without meaning?”. I agree and that’s not what I meant , I apologize. Thank you for challenging me to be precise and clear, I wish I could express myself the way you do :). Maybe an example will help us: You probably teach clothing in level 1. And since you use Trotro, here is an activity with Troto where students have to HEAR (I play the video without images at first) which clothing Trotro puts on today: goo.gl/zyLJup. I guess you could call it “making meaning” but I call it “recognizing words/phrases you know”. At this point, I am more concerned with whether they can recognize words/phrases they know in context rather than making meaning of the episode. Similarly, when we study the high frequency phrase “veux/veut”, I play the Zaz song “Je veux” and ask students how many times they hear the phrases “je veux”. Here again, they don’t really understand the meaning of the song (yet, I actually introduce the whole song later on as a song of the week), but they demonstrate that they can recognize the high frequency phrase. By making sure our Novice hear words/phrases they know in different contexts (and not just when we teacher say them), they are on their way to making meaning.

    Reply
    1. madameshepard Post author

      Thank you so much for clarifying for me! This makes so much sense to me now–I appreciate your patience and your mentorship. I look forward to continuing these conversations!

      Reply
    2. Kerri

      What a clever idea to connect to my VOULOIR lessons. I also love your song of the week idea. I tried word of the week, but it didn’t quite work out. With the music, I can have that playing as they walk in or out, even as some background music during an activity. That I can do and students are always asking me what song I play at times. I’m going to make a “playlist” on our classroom wall. Thank you!!!!

      Reply
  4. Jennifer Geroux

    Lisa, vous êtes vraiment « formidable » et je voudrais vous dire à quel point je suis ravie que vous ayez partagé toutes vos leçons et ressources ici sur votre blog. J’en utilise quelques-unes et je les trouve vraiment bien faites ! Je voulais simplement ajouter un autre dessin-animé que j’ai trouvé (car j’ai une fille de 4 ans et bien sûr qu’elle les regarde tous les jours)- cette émission s’appelle Mouk (version française se trouve sur Youtube).

    Reply
  5. Robin Noudali

    Hi Lisa,

    Thank you for the great ideas which I will be incorporate this coming school yea! I love the cartoons you found that would be accessible for novices and I love the interpretation activities on Canvas. Canvas seems to have a way to share assignments/quizzes by making them public. I wonder if we could make activities public and name them in a uniform way so that they would be easy to locate, sort of like twitter with authres and langchat. I haven’t looked too much in the public side of Canvas because it looks overwhelming and hard to find good activities. As more schools are adopting Canvas, it would be great to be able to share activities and organize them for easy accessibility.

    Reply
    1. madameshepard Post author

      Your ideas are great! However, since I wrote this post I have changed schools and am now using Schoology as a Learning Management System. Since my current school provides Chromebooks to each student, I have created Edpuzzle quizzes for many of these cartoons. I think you can search the name of the cartoon to find what various teachers, including myself, have created.

      Reply

Leave a Reply

Your email address will not be published. Required fields are marked *