Assessing with Storytelling: Listening Comprehension Quizzes

We want to assess listening comprehension because being a competent listener is essential to learning a new language. Why? If you can understand what is spoken to you, there is a wealth of “authentic” and comprehensible input available to aid with acquisition. If not, listening to a million hours of the language won’t help you or, in this case, it won’t help our students.

Case in point, I used to listen to Korean radio station on my commute. I loved listening the flow and rhythm of the language, but I can’t tell you how to say a single thing in Korean, even after many, many hours of listening. It was incomprehensible to me and, thus, not very useful for acquisition.

Okay, so we know that student listening comprehension is important. Probably didn’t need me to convince you of that. But how do we know that students are learning to listen?

The answer is, perhaps, quite obvious: test them on their listening comprehension. 

This will lead to a washback effect, and students will subconsciously try to get better at listening and participating in the story co-creation process.

Listening Quiz Specifics

The typical listening quiz in my class is “true or false”, lasts only six questions, and is student-generated and corrected. Sound like a good assessment? Let’s look at it deeper.

For this quiz I read the questions out loud, and only twice per question. I want students to show me that they can understand the language. I also don’t want this quiz to take too much time—it’s meant to be a quick assessment.

After the quiz, students quickly exchange papers with a partner and we correct it as a class. I read each question one more time and if it’s true we say “cierto”, and if it’s false we say “falso”, and correct it so that it’s true. The following is a good example:

Me: Mario is a fat man with short arms.

Class (and me confirming) False. Mario is a fat man with long arms.

The grader writes her name on the quiz, shows the taker his score, and they pass up the quizzes to the front of the class where I collect them. 

If students have been paying attention and actively negotiating meaning, it will be an easy quiz. Very easy. This is a good thing. Doing well on a quiz boosts their confidence, lets their brain give them a metaphorical pat on the back, and allows me to build reporte by giving some easily-earned (but not entirely insignificant) points.

Another benefit of these quizzes is that they are a way to “trick” students to negotiate meaning with another repetition of the same comprehensible language. It’s a quiz, so you know they are paying attention. They have to successfully negotiate meaning at least twice – once when we co-created the story and once when I read the questions during the quiz – in order to earn the easy points (easy only for those who paid attention).

Generation of the Quiz

You could make a listening quiz take place after a reading day, in which case it would be feasible for you to make the quiz ahead of time. That’s a perfectly fine route to go with it. I prefer to have a student volunteer to write the quiz based on the story we have co-created that day. This has numerous benefits.

  1. It keeps the level of difficulty to approximately the level of the class.
  2. It saves time. I have students grade a partner’s quiz and do a random spot check to make sure they are being honest. All I have to do from there is input the scores into the gradebook.
  3. Lastly, it frees up my mind to focus more intently on the story. This is huge. I can’t imagine having the mental flexibility to co-create a story, monitor comprehension, and stay “in bounds” without outsourcing this job to a student.

Final Thoughts

This is the fastest way to grade the quiz that I have found. One quality of a good assessment is that it takes a representative sample without having to spend much time to grade. Since language acquisition takes place over many, many years, grading a lower-level quiz is perhaps the biggest waste of time for language instructors.

Ideally, I would go full-hippie and not have any grades in my class. I don’t give my son a grade on his acquisition of English or Spanish, but instead give him more opportunities to communicate in the language. Of course, that is not the context in which I find myself, and so I strive to find the best assessments that fit storytelling at the college level.

I love this assessment. It’s one of the most efficient ways to help me me to see if students are really learning to listen.

Disclosure: Please note that some of the links above are affiliate links, and at no additional cost to you, I will earn a commission if you decide to make a purchase after clicking through the link. Please understand that I have experienced all of these books/products and I recommend them because they are helpful and useful, not because of the small commissions I make if you decide to buy something through my links. Please do not spend any money on these products unless you feel you need them or that they will help you achieve your goals.

Assessing with Storytelling: Chapter Quizzes (Pretests)

Chapter quizzes offer students a different kind of feedback that will help them prepare for the grammar final (If you give one).

In a previous post I talked about the grammar final I give students, and chapter quizzes go hand-in-hand with these. After each chapter I give students a one-page quiz.

I make this quiz based off of the stories that we’ve co-created, but I also ensure they draw heavily from the relevant structures covered in the chapter. Let’s say that in chapter one we cover the verb ser, to be. I will use variations of the examples found in our stories as the base for the quiz. 

Consider the following example story fragment:

There is a girl. The girl’s name is Samantha. Samantha is a tall and intelligent girl. (Instructor question to class) Are you all popular? Yes, you are all very popular. Samantha has a sister. Her sister’s name is Beth. Beth is smart too, but Beth is not tall. She is short. Samantha says to her sister, “Are you smart?” Beth says, “Of course I’m smart. I’m Beth!”

The chapter quiz (no notes) may look something like the following:

The first girl’s name ________ Samantha.

The second girl’s name ________ Beth.

Samantha and Beth ________ sisters.

________ you all very intelligent?

Of course! You all ________ very intelligent.

I ________ very intelligent too.

We ________ very intelligent people.

This is similar to a traditional chapter test, but I want these to be very brief: a maximum of one page. This is an assessment based on sampling, and is not meant to be exhaustive. As a result, I grade heavily for comprehension and minimally on accuracy. If students understand the language on the test, they are providing evidence that they are on track.

In this way, chapter quizzes are more of a pretest, even though they are given after the chapter. I often call them pretests because they are a low-risk a way for students to see where they are doing well, and where they need to focus to succeed on the grammar final, the post test.

Each chapter quiz is only worth 25 points. This helps me out in another area: if students miss one of these assessments, they miss it. No make-ups will be given unless there are extenuating circumstances, but it’s not that big of a deal. It won’t penalize them too much to miss one of these quizzes.

A lot of students feel the need to study for this, and as a supplement I give students an exhaustive study guide for all the material covered in the text. Between storytelling and the study guide, most students are in really good shape.

If I didn’t give a grammar final, I don’t think I would give these quizzes either. But I like to afford students the opportunity to prepare for all the different assessments that happen at the end of the quarter.

Assessing with Storytelling: Final Grammar Test

People who excel at math also tend to excel at grammar tests. Pattern recognition is fine and even useful, but we testing for proficiency?

On the second to last day of instruction my students take a traditional grammar final. This exam covers all the relevant grammar covered in the course. You know what this looks like if you’ve ever given or taken a grammar test, so I won’t explain it further here.

To be completely honest, I’m not a fan of this kind of assessment. I don’t think it assesses proficiency in any meaningful way. In his book Assessing Proficiency in the Classroom, second language acquisition expert Eric Herman agrees. “New approaches require new testing methods,” posits Herman.

Note: Herman proposes some very interesting ways of evaluating students for proficiency, which I will discuss in greater detail in a another post.

I totally agree with this sentiment, and it’s why I put so much emphasis on the Oral Story Test and Final Writing Project. I think these assessments do better at evaluating proficiency than their traditional counterparts. This begs the question of why do I still give a grammar test at all?

I’ve experimented with eliminating grammar tests entirely in the past, and it made me a little nervous. Frankly, the main reasons I still give grammar tests are to prepare students who go on to take Spanish from a more traditional instructor, and to have proof that my students can still pass a traditional test. That’s it.

If everyone jumped on the storytelling/teaching with comprehensible input/feed the acquisition monster train tomorrow, I would abandon the grammar test entirely. Until then (or until I become a department head somewhere), I will keep the grammar test, despite my conviction that it doesn’t assess what I want it to assess.

With that said, you’ll notice that the grammar final (and pretests) are worth only ~18 percent of the final grade. This is by design. It minimizes student exposure to the assessment of discrete grammar items, and gives me the proof that they can still pass this kind of exam.

For now, I think this is the best solution in my context. Perhaps you have more freedom that I perceive to have. In that case, I would encourage you to think about creating some exams/assignments that better align with assessing proficiency. 

Disclosure: Please note that some of the links above are affiliate links, and at no additional cost to you, I will earn a commission if you decide to make a purchase after clicking through the link. Please understand that I have experienced all of these books/products and I recommend them because they are helpful and useful, not because of the small commissions I make if you decide to buy something through my links. Please do not spend any money on these products unless you feel you need them or that they will help you achieve your goals.

Assessing with Storytelling: Final Vocabulary Test

Gestures help establish meaning in L2, short-circuit English from the acquisition process, and can be effective for assessing knowledge of vocabulary.

Most college language textbooks include too much irrelevant vocabulary. Instead of focusing on essential vocabulary from the beginning, they want students to memorize personal pronouns out of context and every form of the verb “to be”. Unsurprisingly, this is not how we acquired our first language(s). Instead, we learn new words in context when the communicative need arrises. The thing is, I want my classroom to mimic first language acquisition as much as possible (I know L1 and L2 acquisition have differences, but I’m sure the processes are more similar than different).

How, then, do we create a storytelling system that lines up with the textbook?

I find myself in this position: my department uses a textbook and I’m the only storytelling instructor. I need to ensure that my students can leave my classroom and be successful under the tutelage of a traditional immersion-style” teacher (I would argue that an instructor who uses storytelling is also an immersion-style teacher).

The key to pulling this off is to focus the high-frequency vocabulary and hammer it home every day. I’d much rather they be able to use the most important verbs to be, to want, to have, to go, etc. than to know how to correctly conjugate every single stem-changing verb. Students are still responsible for all the vocabulary in the text on chapter pretests and the final grammar exam, but I will use our precious class time on the vocabulary that will help students learn to teach themselves additional vocabulary (i.e. teach them the words that will help them be competent readers in the language).

Assessing Vocabulary

This quarter I’m trying something new to teach and assess essential vocabulary. Throughout the quarter I am creating a running list of high-frequency vocabulary for which we will create TPR gestures (Total Physical Response — See James Asher’s book for more information) in class.

Each day we will learn five words/phrases, assign gestures, practice them a bit, and use them in context for as long as student interest remains high. The next day we will review the previous day’s gestures, learn five more, and repeat the process. This is a powerful way to build essential vocabulary fast as it does the following:

Day 1

  1. Present Word in Spanish
  2. Translate Word to English
  3. Invent Gesture (repeat 1-3 with all target vocabulary for the day)
  4. Discuss the gestures in context using L2 via Personalized Questions and Answers (PQA)

Day 2

  1. Review TPR gestures from the previous day (present word in Spanish, show comprehension via the gesture we invented).
  2. Present New Vocabulary in Spanish
  3. Translate New Vocabulary to English
  4. Invent Gesture (repeat 2-4 with all target vocabulary for the day)

Day 3

  1. Review TPR gestures from the previous day (present word in Spanish, show comprehension via the gesture we invented)*.
  2. Present New Vocabulary in Spanish
  3. Translate New Vocabulary to English
  4. Invent Gesture (repeat 2-4 with all target vocabulary for the day)

Repeat the process everyday, or as often as you’d like. The more frequent the better.

*Reviewing all the TPR gestures will get lengthy, so perhaps you do all the gestures for a few weeks, but then only do a sampling of gestures before adding that days new vocabulary.

Students are asked to recall the meaning of the Spanish word using a gesture. Notice how the English translation is removed from the process? TPR short-circuits English and establishes meaning in the TL by the second day. From that point on they are recalling the meaning via a gesture and not via English like they would be with flashcards or some other method. Powerful stuff.

Note: I learned Spanish in high school and beyond. TPR was instrumental to my acquisition since it gave me chunks of language that I used to read stories in Spanish, which recycled this vocabulary enough to make the words stick in my brain. I can still clearly remember the TPR gestures we used for many words. I think that letting students pick the gesture is a way for them to take ownership and gives gestures a more personal meaning.

Another benefit of TPR is that students can self-assess whether or not they know this vocabulary as we review as a class. Furthermore, they can ask for clarification, and they can quickly study and learn this vocabulary in class.

We will go through this routine at the beginning of class all quarter, except for a few days when we have a chapter pretest or are doing some other activity that takes a while.

There are roughly 50 class days in my quarter. At five words per day X 50 days, students are responsible for approximately 250 TPR words and phrases. At the end of the quarter they will have a Final Vocabulary Test where I show the class 50 random gestures in a video, and they will have to write down the correct word on a piece of paper.

Since this is my first time doing this assessment, I will report back the results. But I suspect this to be a large enough sample to accurately assess how much vocabulary students actually possess.

Disclosure: Please note that some of the links above are affiliate links, and at no additional cost to you, I will earn a commission if you decide to make a purchase after clicking through the link. Please understand that I have experienced all of these books/products and I recommend them because they are helpful and useful, not because of the small commissions I make if you decide to buy something through my links. Please do not spend any money on these products unless you feel you need them or that they will help you achieve your goals.

Assessing with Storytelling: Homework

 Online grammar workbooks are the trend in higher education, but they aren’t the most efficient use of time.

When I learn languages on my own I read, read, and read some more. This is perhaps the best way to acquire vocabulary and, therefore, grammar. As linguist John Pasden puts it in a post entitled Flashcards: That’s Not How It Works!, “reading is the original spaced repetition.”

I am convinced that the ideal homework for language learners should be to listen to and read level-appropriate language that they find compelling. I define level-appropriate input as i + 1, not i + 100. This means that Language 101 students would be better served to read and listen to language that shelters vocabulary. Reading the newspaper in Spanish won’t hurt beginners, but it will be highly incomprehensible and, thus, an inefficient use of time.

With this in mind, I am piloting* a digital workbook in my 101 classes that will help students acquire 50 of the most important words in Spanish through reading. Each night, students will have reading homework that puts the vocabulary and grammar I want them to acquire in context. The readings build off each other and at the end of the workbook I will test to see what they have acquired.

I will detail this pilot homework program in another post, but this nightly reading homework will give students another chance to negotiate meaning and mentally process the language.

The ultimate goal of this homework program is to prepare students to read one of the many CI readers available for beginners. The earlier learners start reading for fun in the language, the faster they will acquire, the more they will acquire, and the easier our jobs will be as educators.

This kind of homework fits nicely with storytelling. We tell and co-create stories so our students can seek out other stories that interest them. In this way, we create lifelong, independent language learners.

*Unofficially. This is just an experiment my classroom.

Online Textbook Homework

My department adopted Panorama from Vista Higher Learning. This comes with an online workbook that students use to practice the language. It’s better than some online workbooks, but it has a definite grammar focus with videos explaining grammar concepts in English. These videos are dreadfully boring, and they don’t help students build mental representation of the language (Further reading: an excellent post by Chris Stolz that summarizes Dr. Bill VanPatten’s idea of mental representation).

In some cases we can’t escape using the textbook entirely. In my context, for example, I am required to use the online workbook for homework. And since students have to use it, I want to ensure they get the most out of their effort. I do a few things to accomplish this:

  1. I assign mostly listening comprehension, multiple choice fill-in-the-blank, and fill-in-the-blank translation activities (i.e. Pablo ________ [to eat] una pizza). I do assign a few of the paragraphs with random blanks, but they have diminishing returns.
  2. I spend some time in class going over the assigned activities and “PQA”*-ing some of the unfamiliar vocabulary. This helps keep this homework comprehensible and makes it somewhat worthwhile for them to do.
  3. I limit student exposure to this homework in terms of their final grade. In fact, some of my students choose to never complete these assignments, but can still pass the class. The most skipping all the online workbook homework can do is drop their grade by a little less than 10 percent, or about a full letter grade. It’s there to give students a chance to improve their skills, not to as an assessment tool.
  4. I tell students about other things they can do to increase their proficiency (e.g. reading, listening, seeking out other forms of compelling input).

*PQA – Personalized Questions and Answers

I should also note that students always complain about the online homework as being too hard, and I agree with them. It is hard because we don’t learn language by practicing but rather through the mental processing of input. To quote the great Carl Sagan:

“The brain does much more than just recollect. It inter-compares, it synthesizes, it analyzes, it generates abstractions. The simplest thought, like the concept of the number one, has an elaborate logical underpinning. The brain has its own language for testing the structure and consistency of the world.”

Indeed, the brain has its own language for grammar too. Textbook rules only attempt to explain the phenomenon of language as we observe it. They are not what our brains use to produce language.

One day I would like to see us move away from these language “practicing” systems and over to ones that provide more and varied forms of compelling, highly comprehensible input. In this way, students will have the opportunity to acquire language in a more natural, efficient, and self-sustaining manner.

Disclosure: Please note that some of the links above are affiliate links, and at no additional cost to you, I will earn a commission if you decide to make a purchase after clicking through the link. Please understand that I have experienced all of these books/products and I recommend them because they are helpful and useful, not because of the small commissions I make if you decide to buy something through my links. Please do not spend any money on these products unless you feel you need them or that they will help you achieve your goals.

Assessing with Storytelling: Oral Story Test

Picture notes are a personalized way to help students stay in the TL while retelling a story.

This is my favorite assessment of the quarter. Students pick any story from the quarter that we have read and they retell it to me as if I have never heard it before. Each story is about the same level of difficulty, so I don’t mind letting them pick one from the beginning of the quarter. The last story we co-create might even be easier for them since it’s fresher in their mind (In the past I have also just assigned the last story we co-created. It really doesn’t matter).

After students have completed the test, I ask questions about them that are related to the story they chose. For example, let’s say that in the story a character goes to a café and drinks a coffee (mmm, coffee). I may ask a student something like, “Do you drink coffee?”, “Do you prefer coffee or tea?”, or “What do you drink when you are thirsty?”. This provides a way to assess their conversational ability during a spontaneous interaction.

In a way, I suppose that I do prepare these questions. I make sure to include specific vocabulary and phrases in the written version of the stories, and this lends itself to asking certain questions. But I don’t write out a list of questions ahead of time, and instead take them from the text itself. 

(Note: I will do a separate post about how I introduce and structure stories throughout the quarter after I finish this current Assessing with Storytelling series.)

Notes

I allow students to use picture notes on this assessment with as many hand-drawn pictures as they like — one picture for each word or even each syllable if they want. The only caveat to these notes is that there can be no words on the paper or they can’t use it. In this way, students can focus on how to express themselves instead of being pressured into remembering every detail of the story.

The picture notes are not required, but maybe the should be. Students who take the time to draw out the story tend to perform way better than those who just try to memorize or remember everything without notes. If they draw enough pictures they always remember how to say something, even if it’s not perfectly accurate.

I use this assessment as my final exam, which is usually a 2-hour block for me. When I’m done with the block of tests I’m mentally exhausted from listening to all these students, but I’m done grading. I’d argue that this is the mark of a great assessment.

In Groups or Individually?

I keep going back and forth on this one. Some quarters I have had students go individually. This provides the best feedback for students, but takes way longer and gives me the impression that I may be intimidating some students (I’ve had some students cry one-on-one due to the test anxiety—Granted, they also could have cried in a group).

Other quarters I’ve put students in groups of two and had them each tell half the story. This way they have to listen to each other and pick up where the other left off. If you let them pick the story

I pick the groups, and they don’t know which half of the story they’re going to retell. That way they have to prepare the whole thing. Really the whole thing is to get them to do a deep dive on a story and get more comprehensible input. My son does this naturally by picking the same story to read over and over, but adults need a little prodding to do this behavior. This is the perfect way to get them to reread a story from class.

Note: If you want then to tell the story in groups and let students pick which story they want to retell, you need to know ahead of time which students studied which story. That way students who studied Story A can go with other students who also studied Story A. 

In Front of the Class or in Private?

I let the small groups go privately at an assigned time. The rest of the students wait out in the hall/lobby until they are called into the classroom. The idea behind this assessment is to see what students can produce, and talking in front of the class on a test doesn’t do anything but raise the affective filter. The test itself is enough anxiety, even though I aim for it to be as low-key as possible.

How I Grade the Oral Story Test

Just like the Final Writing Project, I have a rubric for grading this assessment. It helps me to be more objective, and it lets me finish grading as soon as they’re done taking the test.

Note: While students are talking, I scribble notes on a blank piece of paper so I can remember what students said on the test. This practice has come in handy on more than one occasion when a student has come seen me for additional feedback or the time I forgot to circle the scores on my rubric. Let’s just say I’m glad I wrote down what they said.

Communication and Flow

10 203040
Student could not communicate in Spanish. Student could not respond freely to instructor’s questions using emerging output in Spanish.
Use of English.
Student demonstrated limited communication in Spanish. Student responded to some of the instructor’s questions freely using emerging  output in Spanish that approached the expected level.Student demonstrated level-appropriate communication of a message (i.e. the events of a story) in Spanish. Student responded to most questions using level-appropriate emerging output in Spanish.Student demonstrated above-average Successful communication of a specific message in Spanish. Student responded fluently to all the instructor’s questions using above-average emerging output in Spanish.

Like it’s written counterpart, communication is the most important factor that I look for when grading this assessment. I want to hear students being able to communicate the events of a story using only their emerging output in L2.

Grammar and Vocabulary

10203040
Student did not demonstrate understanding of relevant vocabulary. Student did not demonstrate natural or accurate production of grammatical forms, especially those covered in class.Student demonstrated some understanding of relevant vocabulary. Student demonstrated accurate production of  relevant grammatical forms below the expected level.Student demonstrated level-appropriate understanding of relevant vocabulary. Student demonstrated accurate production of vocabulary and grammatical forms, especially those covered in class.Student demonstrated above-average understanding of relevant vocabulary. Student demonstrated above-average production of  grammatical forms, especially those covered in class.

This is another one of the few times in my course that I evaluate grammar in context. Students have been afforded the opportunity to select and study the story and bring in picture notes. I’m looking for their grammar and vocabulary to be accurate.

Despite the generous design of this assessment, it’s still incredibly difficult for students to remember everything.  It is easy, however, to identify the forms and vocabulary students have acquired successfully.

Natural Progression of Fluency

5101520
Student demonstrated a lack confidence while speaking and interacting, which was highlighted by frequent and long hesitations in Spanish (e.g. long pauses. Many “ums,” “uhs,” and/or other English filler words).Student demonstrated a lack confidence while speaking and interacting, which was highlighted by some hesitation in Spanish (e.g. some long pauses. Many “ums,” “uhs,” and/or other English filler words).Student demonstrated adequate confidence while speaking and interacting, which was highlighted by  a lack of hesitation in Spanish (e.g. few pauses and very infrequent use of “ums,” “uhs,” and/or other English filler words).Student demonstrated a high degree of confidence while speaking and interacting in Spanish, which was highlighted by few (if any) pauses. Use of Spanish filler words such as  “bueno” “pues”, “este”, etc.

Last summer I attended a second TPRS© workshop, this one in Seattle. The presenter was Mike Coxon, and he defined fluency as students demonstrating “confidence, accuracy, and a lack of hesitation”. I like this definition of fluency. As a long-term goal, we want students to speak confidently and accurately, not with a bunch of errors in their speech. we also wan them to speak without hesitation. We don’t want them to think about the language; we want them to think in the language.

I’m sure this isn’t the perfect rubric, but I also believe it gives me a fairly accurate picture of my students’ ability to speak the language.

I look forward to this assessment every quarter. It’s the easiest thing in the world to grade, and it lets me see how far they have come during the short time we spent together.

Assessing with Storytelling: Final Writing Project

Doing writing assignments in class is key to avoid the dreaded Google Translate mosnter.

In another post I talked about using timed-writes as an assessment tool. This is great for people who over-monitor their writing, and it affords me the opportunity to see a snapshot of my students’ true level of spontaneous written production.

When I learn languages on my own, I will often use timed-writes to track my progress. The benefit of having someone correct my writing is limited because my interlanguage is constantly evolving as I get more comprehensible input.

The same is true for students. As long as they keep getting comprehensible input, their interlanguage more closely approximates native level L2.

While I believe timed-writes good enough evaluate student progress on their own, I also like to afford students the opportunity to demonstrate what they can produce with their monitor activated.

On the last day of instruction I have students write an original story in class using the language they have learned during the quarter. While the timed-writes show much how fluently they can write, the original story lets me see their best writing.

The Final Writing Project

I give students two weeks notice of the content of this assessment (although they have access to the rubric all quarter), and they can prepare as they see fit. Some plan out a well-structured story and other wing it the day of. Students get a whole class period to write the story (not to exceed 300 words, I don’t want to read these things for more than a couple of days), and they can leave when they finish. Some leave in 20 minutes, others take the whole class period.

Note: To avoid the headaches of Google Translate, this assignment only works if it’s done in class.

The structure of the assessment asks students to use all the big verbs: to be, to want, to have, to go, to like, etc., and I have a rubric that makes grading a piece of cake, pan comido. Below are the instructions students will see on the day of the assessment (or in the course documents from the beginning of the quarter if they bother to look there):

You will have the approximately 45 minutes to write an original story in Spanish. Your story should have a minimum of 200 words and a maximum of 300 words. Previous writing assignments have focused on word count under a time constraint, but this exercise is different. Here I am looking for your best writing. Please take your time and edit your sentences carefully.

The events in your story are entirely up to you, but your writing should flow nicely and reach a logical conclusion. Below is a sample story structure (similar to the format we generally use in class to co-create stories) that may help you write more:

  • Introduce and describe your main character(s)
    • Where are they from? How old are they? What are they like? How are they feeling? Etc.
  • Define the problem
    • Your character(s) should want something but be unable to get it at the beginning.
  • Movement 1
    • Your character(s) should go somewhere and ask for help to get what they want (dialogue).
    • Your character(s) should not solve the problem in this location.
  • Movement 2
    • Your character(s) should go to a new location and ask for help (dialogue).
    • Your character(s) should solve the problem in this location.
  • Tie up loose ends
    •  -How does the story resolve? Are there psychological or moral changes in your character(s) (i.e. How do(es) your character(s) grow?)
    • Use the falling action to close any loops you may have opened.

How I Grade the Final Writing Project

Naturally, I have a rubric for grading this assignment. It makes it easy to evaluate and allows me to be more objective. You will find it below:

Communication and Flow

10 203040
Student failed to demonstrate successful  communication in Spanish. The message was difficult to discern and was difficult to follow.  Many choppy sentences.Student demonstrated limited communication in Spanish. The message was somewhat difficult to discern and was often difficult to follow. Many choppy sentences.Student demonstrated level-appropriate communication in Spanish. The message was somewhat clear and was mostly easy to follow. Few choppy sentences.Student demonstrated successful communication in Spanish of the events of a story Spanish. The message was clear and was easy to follow. No choppy sentences. 

Communication is the most important factor that I look for when grading this assessment. I want to see students express themselves clearly and with good flow. This part of the rubric shows the four main categories student work may fall into, but I can break it down more if need be. Perhaps a student has too many choppy sentences to earn all 40 points, for instance. I could give the student a 35 and call it good.

Grammar and Vocabulary

10203040
Student demonstrated poor accuracy in grammar forms covered in class and understanding of relevant vocabulary. Many spelling and accentuation errors.Student demonstrated below-average accuracy in grammar forms covered in class and understanding of relevant vocabulary. Some spelling and accentuation errors.Student demonstrated acceptable accuracy in grammar forms covered in class and understanding of relevant vocabulary. Few spelling and accentuation errors.Student demonstrated above-average accuracy in grammar forms covered in class and understanding of relevant vocabulary. Very few or no spelling or accent errors.

This is one of the few times in my course that I evaluate grammar in context. Since students have time to think about what they are writing, I feel it important to evaluate them on this point. If they mess a few things up it will not destroy their grade, nor should it.

Spelling and accentuation also count here, but not that much. In the process of acquisition it’s okay to make spelling mistakes. I still misspell things in English and Spanish all the time (especially in English), and those are the languages I speak best. I would dread it if someone evaluated my spelling in my beginner French (Je ne parle pas très bien le français… yet).

Creativity 

5101520
Student’s work was not original (e.g. wrote a summary of a story we co-created in class). Student failed to include details from the outline above.Student’s work was minimally original. (e.g. Student copied the details from a previous class story, but changed a few details). Student included some details from the outline above.Student’s work was original and included many details from the outline above. Student included some creative dialogue.Student’s work was original and included all the requested details from the outline above. Student included creative dialogue and went above and beyond by showing how their character(s) changed at the end of the story.

The last part of this rubric is the one that I struggle to justify with 100 percent confidence. This quarter I have included it because I need a way to hold students accountable for preparing for the assessment. In the past I didn’t include this section and some students used a story we had co-created in class and just changed the names—not exactly the spirit of this assignment.

I also want to reward students for going through the process of writing a story in L2. It’s difficult, but by the end of the quarter even the lower students should be able to approximate the stories we co-created in class and make enough modifications to be original.

One Last Note on Rubrics

No rubric is perfect and, the way I see it, most of them are too crude to give a highly precise assessment. For example, is this rubric really sensitive enough to give an 83.5%? I don’t think that it is. For this reason I round all scores to the nearest 5%. It makes grading easier because it gives larger error bars to help assess the grade, even though the assessment tool itself is imperfect.

Assessing with Storytelling: In-Class Essays

Timed-writes are an effective way to evaluate student progress throughout the term.

My class is 12 weeks long, ten if you don’t count Thanksgiving and finals week. The first week I don’t have students do much writing, especially in Spanish I. Instead I want them to listen and read. They need to get used to hearing the language, seeing it written out, and using it to communicate. I do, however, want to start to get a baseline for my students’ abilities, and so I have them begin to write about topics they are familiar with by the end of week 2.

In my Spanish I classes, this takes the form of a story summary under a constraint. During the week we learn, read, and discuss a story in the target language. On the last day of the week, students get five minutes to handwrite as much as possible in Spanish about that week’s story. In that time they write as much as they can without pausing to think about “correctness.” I see the same grammar mistakes you can expect to see on grammar tests, but I also see a more complete picture of their emerging out, even from the low students.

The idea behind this is twofold. First, the time constraint forces students to turn off their conscious mind and let the words bubble up naturally from their subconscious. This is what I’m most interested in seeing, since this is the language they are able to successfully recall and access, that is, the language they have acquired.

The second benefit of the timer is that of Parkinson’s Law: work expands so as to fill the time available for its completion. If we give students 20 minutes to do a summary, they will take the whole twenty minutes.

I grade these timed writes on word count because I’m not interested in the improving grammar from week to week, although I do see that progress. What I most care to assess here is that students make the expected progress in fluency (i.e. adjective agreement, subject-verb agreement, verb morphology, etc.) by the end of the quarter. I don’t want them to ignore grammar entirely, but I tell them to do their best with grammar and spelling. Not only do students write more this way, but they also write better when I tell them I’m not grading for the things that most concern them: grammar and spelling.

The word count is a moving target. At the beginning of the quarter a student that writes 50+ words in 5 minutes earns a 25/25, 40-49 words earns a 22.5/25, 30-39 earns a 20/25, 25-29 earns a 17.5, and < 25 earns a 15/25.

Depending on the results, I challenge students to push themselves. The next week I might make 60+ words a 25/25, 50-59 a 22.5/25, and so on. By the end of the quarter I expect a good number of students to write 90+ words in 5 minutes. Many students end up writing 150+ words in five minutes. That’s more than I can write in that amount of time.

I play the timeline of this sliding scale by ear. That means that at any given time I have some sections that need 75+ words for a 25/25, and others that 60+. This makes total sense to me because it allows for differentiated instruction. Not every class will progress uniformly, and this is especially true for a language class.

I love this assessment because I can track student progress throughout the quarter. I can see some students make leaps in their abilities and I can see the steady progress of others.

Another reason I love this assessment is that students find it hard to cheat. They only have five minutes and are graded on word count. If they stop to Google Translate a word, they’re not going to get a good grade. This is, of course, by design. The time constraint takes the temptation of translation away from students and allows me to see their true ability.

Yet another reason I love this assessment is that I can put students to work self-grading their work. I put the grading scale up on the board, and they count up their words and write their number and grade on the top of the page. Chalk this up as another win for systematizing the grading process.

I still read their work, but now I get to read it for entertainment, not for grading purposes. Naturally, I still offer feedback to anyone who asks for it, but in my experience this rarely ever happens. Occasionally someone will come up and ask for feedback, and I am more than happy to go over their essay with them.

If it’s possible, I love this assessment for yet another reason: students re-read the story we made up in class, sometimes a few more times. As they prepare for the timed-write, they seek out comprehensible input. Not only that, but it sets the expectation that they will need to recall the information in the story at a later date, which washes back to the next time we co-create a story together. It’s a positive feedback loop that leads to increased engagement on the next story, which leads to improved recall on the next timed-write, which leads to the endorphin rough of successfully learning something new, ad infinitum.

This is a phenomenal assessment tool for the storytelling classroom.

Assessing with Storytelling: Interpersonal Communication

Communication is king/queen/the democratically elected executive officer of the week.

This is the daily grade that has replaced the “participation” grade in my syllabus. In reality, this is the way we norm students to set themselves up for success — These are the habits we want them to have so their affective filter is lowered and their focus is on communication in the target language.

This is a daily grade, but it goes into the gradebook only once per week for my own sanity (5pts per day x 5 days = 25pts per week). Systematizing your gradebook like this will save you hours of data entry over the course of the term. I highly recommend you figure out this kind of trick sooner rather than later.

The Interpersonal Communication Rubric

Good artists copy. Great artists steal. – Pablo Picasso

Another way to systematize your course is to use rubrics — really good rubrics. The rubrics that I use for Interpersonal Communication is a modified version of jGR (Jen’s Great Rubric – I’m sorry, but I cannot for the life of me remember who the Jen is from this as I first read about it many years ago… Jen, if you’re reading this, I’m sorry!).

My version of this rubric also borrows from Mike Peto, whose work you can find at My Generation of Polyglots. It is outlined below:

5/5—Pays attention and Contributes at the 4.5 level and Goes Beyond by:

  • Adding interesting and useful comments that are appropriate and in the target language
  • Speaking aloud voluntarily with longer spontaneous answers in the target language
  • Helping build a positive classroom community by volunteering for jobs (e.g. acting, which lets us use more and different language naturally)

4-4.5—Pays attention and Contributes by:

  • Using English only with permission
  • Regularly using the “I do not understand” signal or the “slow down” signal to help keep comprehensibility high
  • Playing the game by answering with one-word answers or short responses, participating enthusiastically, and particularly by showing that you get the idea that, “Anything is possible in this class.”

3-3.5—Pays attention by:

  • Showing your intent to understand with body language and responses
  • Sitting up; Maintaining eye contact with the instructor, peers who are speaking, and media
  • Having nothing on lap or desk—particularly cell phones—in order to more completely focus on communication in Spanish
  • Observably listening when others speak
  • Not distracting or disrupting
  • Not blurting out in English and not having side conversations.
  • Late to class (especially if more than twice per week)

2.5—Does not pay attention regularly as evidenced by:

  • Having something on lap or desk (such as a cell phone)
  • Slumping in chair; Showing limited effort and/or eye contact
  • Using English more than once without permission

0—Absent

  • Why? It is not possible to contribute or pay attention if you are not in class—this includes excused absences. You cannot demonstrate interpersonal communication if you are not here! – Note: You get three free absences without affecting your Interpersonal Communication Grade. Refer to the syllabus for more details.

You’ll notice that students cannot earn a lower score than a 2.5/5 per day if they are present. Interpersonal Communication is a formative assessment, and I see no reason to penalize students more than 50 percent, even if they are really not living up to expectations. Fifty percent is already an F.

This rubric is powerful. It norms the class for storytelling, keeps the class  90%+ target language, builds classroom community, is easy to keep track of (you know instantly who is doing all the right things to succeed), and can be systematized into a once-per-week data entry task.

This is by far one of the best tools for getting storytelling to work at the college level.

Assessment with Storytelling Part 1

A college language student studying for a big test or trying to find Pepe Silvia or something.

Course Content and Outcomes

Everything starts with the outcomes we want for our students. What do we want students to get out of the course? What do we want them to be able to do by the end of the term? If you can answer this question, you can begin to plan accordingly.

Here is the excerpt from my syllabus that outlines content and outcomes for Spanish 101 (Note: I don’t know who wrote these. They’ve been the outcomes in my department since before I started teaching at this college in summer 2013):

 

Course Content

A. greetings and farewell expression
B. weather descriptions
C. direct object nouns & pronouns
D. daily routines (present tense & present progressive)
E. vocabulary related to academic life, family & pastimes
F. numbers, time, and date expressions

 

 

Student Outcomes

1. Comprehend speech samples providing information pertaining to course content
2. Comprehend written language samples providing information pertaining to course content.
3. Construct basic sentences and paragraphs appropriately.
4. Vary basic information appropriately with modifiers (adjectival and adverbial).
5. Request and provide information pertaining to course content in writing.
6. Request and provide information pertaining to course content orally.
7. Distinguish and demonstrate appropriate forms of address.
8. Apply and comprehend case, person, number, gender, and tense appropriately within the scope of the content of this course as indicated in other outcomes.

I’m not going to say these are the perfect outcomes or that they are inherently flawed. That’s irrelevant for this discussion. These are the outcomes that my department defined and, therefore, they are the target I’m going to aim for.

What to Assess

My goal for assessment is to evaluate students on the course outcomes. In this way they will be prepared for the next course in the sequence regardless of whether or not their next professor uses storytelling.

Your course outcomes will depend on the context in which you find yourself, and you should adjust your assessment accordingly. Perhaps you have a common grammar test students need to pass at the end of the course. On the other hand, maybe you have some flexibility in terms of what you teach and where you want your students to end up.

Storytelling can get you to your course outcomes, regardless of what they are, due to its inherent flexibility. That is, you can make stories use whatever grammar, vocabulary, syntax, etc. you need to present to students.

In my ideal situation, I wouldn’t assess students on grammar in any introductory course. That may sound controversial, but it shouldn’t be. I don’t test my son on whether or not he uses each irregular verb properly (He doesn’t, by the way. But that doesn’t stop him from getting better at English and Spanish every single day).

However, some of us (myself included) are in a context that doesn’t allow us to shun grammar entirely. That’s okay, though. The storyteller doesn’t relegate grammar to the textbook. Instead we help learners build a functional “mental representation” of the language through comprehensible input and ample opportunities for students to negotiate meaning through communicative acts.

One of my course outcomes is for students to be able to understand and use the present tense in class. That means that in Spanish I we will be using almost exclusively present tense, and students will only be expected to produce in the present tense. That means I will select stories and other activities that focus primarily on the present tense.

Knowing where I’m going (i.e. knowing the outcomes I want for students) lets me make a roadmap to get there.

Backwards Planning

Backwards planning is the key to making assessment of storytelling-taught-classes work properly.

When we plan an assessment in advance, we know what’s on it. We know that students must complete certain tasks on the assessment by a certain date. This isn’t a surprise—we put it on the calendar. Backwards planning begs the question “What tools do our students need to complete the tasks on the assessment successfully?”

For instance, let’s say that students must complete a grammar test by the end of the quarter. Now that we know what the goal is, we can start to plan how arm students with the necessary tools to complete the task at hand. What are the tools they need in this scenario?

  1. They need to acquire the vocabulary that will appear on the test.
  2. They need to acquire the forms that appear on the test.
  3. They need to be able to read and comprehend the questions asked of them.
  4. They need to be able to respond to those questions appropriately.
  5. They need to be able to connect their ideas in fluidly.
  6. Maybe some other things I haven’t thought of yet.

How does the storyteller make this knowledge accessible to students? We plan it out, way in advance. Each story I select builds on the previous one so that we recycle all the elements on the test in different communicative contexts.

We’ve seen the vocabulary and the grammar, we’ve discussed the questions, and we’ve heard numerous people express their ideas in a logical, coherent way in the language.

By encapsulating vocabulary, grammar, etc. in a story, students hear and see the language in a natural context that we humans have been using for thousands of years. In my experience, this leads most students to perform better on both formal and informal assessments.

My Assessments for Fall 2018

The first thing I do when designing my courses is to write out a list of all the assignments and assessments that my students will complete. Instead of a percentage-based grade, I tend to favor an overall point system. It’s still broken down by percentage, but I find points easier for students to understand and I field fewer questions about grades as a result. Below is a list of the assignments and assessments for my Spanish I courses:

Interpersonal Communication – 250pts

Online Homework (VHL Central) – 100pts

Listening Quizzes – 8 quizzes x 6pts each – lowest score = 42pts

Timed Writes – 8 timed writes x 25pts – lowest score = 175pts

Pretests – 4 pretests x 25pts = 100 pts

Final Grammar Test – 100pts

Final Vocabulary Test – 50pts

Written Story/Essay Assignment – 100pts

Oral Story Test – 100pts

Total Points: 1017

The first three items on the list (Interpersonal Communication, Online Homework, and Listening Quizzes) are what I consider to be “formative assessments”, and they make up roughly 40% of the total points in the course.

The rest of the assignments on the list (timed writes, pretests, final grammar test, final vocabulary test, written story/essay assignment, oral story test) are what I consider to be “summative assessments”, and they make up roughly 60% of the total points available in the course.

I have a lot to talk about here, so I have decided to break each one of these down in greater detail in separate posts.

When do we need students to demonstrate results?

It would be great for students to show me that they’ve mastered all the vocabulary and grammar from chapter 2 at the time of the test, but we humans acquire languages asynchronously. That is, different people pick up languages at different rates.

Furthermore, some people acquire a certain vocabulary and structures while other people acquire different vocabulary and structures. Yes, we all follow the general order of acquisition, but every learner does so at a pace unique to the individual.

What does this mean for the purpose of assessment? Personally, this means that I care most about what students can produce at the end of the quarter. The mid-term results are a nice barometer for me, but they are useless if the student can’t demonstrate the expected degree of fluency at the end of the quarter.

Backloading Assessments

Putting a Consider the last four assessments in my Spanish I course:

 

Final Grammar Test – 100pts

Final Vocabulary Test – 50pts

Written Story/Essay Assignment – 100pts

Oral Story Test – 100pts

These four assessments take place during the last five days of my course, and they constitute roughly one-third of all the points. After a quarter of delivering high-quality CI via highly personalized and engaging stories, it’s time for the students to show me that they’ve reached the expected level of fluency.

To me, this is the perfect level of balance. Two-thirds of the total points are spread out over the rest of the quarter, which allows me to reward the daily effort required to learn a new language. That leaves one third of the total points at the backend of the course. This allows me to assess proficiency when it matters most: at the end of the course.

Spreading 350 points over four assessments takes the pressure off students and, hopefully, lowers their affective filter.

Q: What if I bomb the Oral Story test?
A: Well that wouldn’t exactly be good for your grade, but it also wouldn’t be the end of the world.

Students always perform better when their affective filter is lowered.

What Do My Assessments Look Like?

In the coming days and weeks I will be delineating all the assignments and assessments in my syllabus. I will make a post about each, the rationale behind them, and how they fit in with storytelling.