A couple of weeks ago I shared my thoughts about assessing interpretive reading. In that post gave my opinion that ACTFL’s IPA Template was a generally effective way to design an assessment of reading comprehension and that, with a couple of modifications, their rubric was well-aligned with the tasks on the template. I have reservations, however, about the use of the ACTFL IPA template to assess listening. Here are a couple of my thoughts about assessing listening, please share yours!
Assessing Interpretive Listening is Important
By defining both listening and reading comprehension as Interpretive Communication, ACTFL has given us an out when writing IPA’s. We can choose to include either one, but are not required to include both. My guess is that when given the choice, most of us are choosing authentic written rather than recorded texts for the interpretive portion of our IPA’s. There are several good reasons why this may be the case.
- Authentic written texts are usually relatively easy to find. A quick Google search of our topic + Infographie will often produce a content-filled, culturally-rich text with the visual support that Novice learners need. Picture books, ads, social media posts, etc. provide additional resources. For our Intermediate students, our options are even greater as their proficiency allows them to read a wider variety of short texts, an unlimited supply of which are quickly located online.
- Written texts can be easily evaluated regarding their appropriateness for our interpretive assessment tasks. A quick skim will reveal whether a text being considered contains the targeted language and structures, culturally-relevant content, and appropriate visual support that we are looking for.
- Assessments of interpretive reading are easy to administer. We need only a Xerox machine to provide a copy of the text to each student, who can then complete the interpretive task at her own pace. When a student is absent, we can simply hand him a copy of the text and interpretive activity and he can complete the task in a corner of the room or any other area of the building where make-ups are administered.
Curating oral texts and assessing their interpretation, however, is considerably more time-consuming. While we have millions of videos available to us on YouTube (my person go-to for authentic listening texts), videos cannot be skimmed like written texts. We actually have to listen to the videos that our searches produce in order to evaluate whether they are appropriate to the proficiency of the students for whom they are intended. In some cases, we have to listen to dozens of videos before finding that gem that contains the appropriate vocabulary, cultural content and visual support that our learners need. When it comes to administering these assessments, we often face additional challenges. In my school, YouTube is blocked on student accounts. Therefore, I have to log into 30 computers in a lab (which is seldom available) or my department’s class set of IPads (sometimes available) for all of my students to individually complete a listening assessment at the same time. While many of us play and project our videos to the class as a whole, I think this places an undue burden on our Novice students who “require repetition, rephrasing, and/or a slowed rate of speech for comprehension” (ACTFL Proficiency Guidelines, 2012). A student who has her own device can pause and rewind when needed, as well as slow the rate of speech when appropriate technology is available.
In spite of these challenges to evaluating listening comprehension, I think we have a responsibility to assess our students’ ability to interpret oral texts. As Dave Burgess said at a conference I recently attended, “It’s not supposed to be easy, it’s supposed to be worth it.” Assessing interpretive listening skills IS worth it. As the adage says, “we teach what we test.” If we are not evaluating listening, we are not teaching our students what they need to comprehend and participate in verbal exchanges with members of the target culture. While technology may allow us to translate a written text in nanoseconds, no app can allow us to understand an unexpected public announcement or participate fully in a natural conversation with a native speaker. In my opinion, our assessment practices are not complete if we are not assessing listening comprehension to the same extent as reading comprehension. As a matter of fact, I include separate categories for each of these skills in my electronic gradebook. While others may separate grades according to modes of communication, I’m not sure this system provides as much information regarding student progress toward proficiency. Although both reading and listening may require interpretation of a text, they are clearly vastly different skills. Students who are good readers are not necessarily good listeners, and vice versa. In their Proficiency Guidelines, ACTFL clearly differentiates these two skills, don’t we need to do the same when evaluating our students using an IPA?
Designing Valid Interpretive Listening Assessments is Difficult
In my opinion, ACTFL has provided us very little direction in assessing interpretive listening. While we are advised to use the same IPA Interpretive template, I find that many of these tasks do not effectively assess listening comprehension. Consider the following:
Key Words. While students can quickly skim a written text to find key words, the same is not true of recorded texts. Finding isolated key words requires listening to the video multiple times and attempting to isolate a single word in a sentence. I find this task needlessly time-consuming, as I will be assessing literal comprehension in other tasks. Furthermore, this task puts some students, especially those with certain learning disabilities at a significant disadvantage. Many of these students have excellent listening comprehension, but are not able to accurately transfer what they understand aurally into written form.
Main Idea. Although this task seems fairly straightforward, I question its validity in assessing comprehension for Novice and Intermediate learners. According to the ACTFL Proficiency Guidelines, Novice-level listeners are “largely dependent on factors other than the message itself” and Intermediate listeners “require a controlled listening environment where they hear what they may expect to hear.” This means that all of my students will be highly dependent on the visual content of the videos I select to ascertain meaning. Therefore, any main idea they provide will most likely be derived from what the students see rather than what they hear. A possible solution might be for the teacher to provide a series of possible main ideas (all of which could be extrapolated from the visual information) and have the students choose the best one. However, this task would certainly be unrealistic for our novice learners who are listening at word level.
Supporting Details. I think this task on the IPA template is the most effective in providing us feedback regarding our students’ ability to understand a recorded text. By providing a set of details which may be mentioned, we provide additional context to help our students understand the text and by requiring them to fill in information we are assessing their literal comprehension of what they hear. In addition, this type of task can easily be adjusted to correspond to the proficiency level of the students. Providing information to support the detail, “Caillou’s age” for example, is a realistic expectation for a novice listener who is watching a cartoon.
Organizational Features While I see little value in this task for interpretive reading, I see even less for listening. As previously mentioned, even intermediate listeners need to be assessed using straightforward texts so that they can anticipate the information they will hear. Having my students describe the organization of a recorded text would not provide additional information about their comprehension.
Guessing meaning from context. As much as I value this task on reading assessments, I do not find it to be a valid means of assessing aural comprehension. The task requires the teacher to provide a sentence from the text and then guess at the meaning of an underlined word. As soon as I provide my students with a written sentence, the task becomes an assessment of their ability to read this sentence, rather than understand it aurally.
Inferences As with the main idea, I think Novice and Intermediate listeners will be overly dependent on visual cues to provide inferences. While I believe students should be taught to use context to help provide meaning, I prefer to assess what they are actually able to interpret verbally. ACTFL does suggest providing multiple choice inferences in the IPA template, but again the teacher would have to provide choices whose plausibility could not be derived from visual information in order to isolate listening comprehension.
Author’s Perspective. While I regularly include author’s perspective items on my assessments for my AP students, I feel this is an unrealistic task for Novice and Intermediate Low listeners. Students who are able to understand words, phrases, and sentence-length utterances will most likely be unable to identify an author’s perspective using only the verbal content of a video.
Cultural Connections. Authentic videos are one of the best tools we have for providing cultural content to our students. The content provided by the images is more meaningful, memorable, and complete than any verbal information could be. However, once again it is difficult to isolate the verbal content from the visual, creating a lack of validity for assessment purposes.
Conclusion
For now, I’m planning on using supporting detail or simple comprehension questions when formally assessing my students’ interpretive listening skills in order to ensure that I am testing what I intend to test. When practicing these interpretive skills, however, I plan on including some of the other tasks from the IPA template in order to fully exploit the wealth of information that is included in authentic videos. I’m looking forward to hearing from you about how you assess your students on interpretive listening!