During my internship year of teaching my primary of way of eliciting evidence of student learning was through discussion in class. The format of the class was short review of topics related to the Math SAT, then a sample Math section of the SAT, then a review of the questions covered. I could provide evidence of student learning if I saw students doing better on geometry, algebra, numeracy, or chart and graph questions that we worked in class.
With the information of students either getting problems right during sample tests or being able to describe where they went wrong, I was able to suggest other homework or exercises that students might complete.
Since there was not grade in the class, the students were all preparing to take the SAT on June 3rd. That was the final summative assessment, if you will. What I think I could have improved on was the gathering of concrete data that students were getting more answers correct in various subtopics of the Math SAT. I have spoken with my mentor teacher at length about a more effective use of homework, but I think it is fair to say that homework is not something that students are motivated to do at my school, and the value of pressing the issue was unclear.
I have to admit, too, that I was way more focused on the presentation of material and of the simulating of the testing environment and getting the students to see questions, than whether the students were improving their performance on the test. An embedded formative assessment model would enable me to tailor my lesson delivery based on student ability to complete lessons or problems. Also I would enable students to help each other learn and push them to be in charge of their own learning.
I was extremely happy to see students engaging each other on explanations of problems they got right, i.e. to help other students who didn’t get the answers. However I didn’t harness that information to feed back into what the instruction part of the course was doing. Also for all the test sample questions we took and talked about, I never kept a running tally of who was getting which questions right or wrong. That would have enabled me to tailor some instruction appropriately. 
 Finding Out What Students Know
 TIMSS example questions that “seem” the same many times aren’t.
 Opinions vary on why these two sample questions are so different in success rate.
 In answering the second question, 39 percent of the students chose B. Since this question was answered correctly by 46 percent of the students, it was answered incorrectly by 54 percent, but 39 percent chose the same incorrect answer.
 This suggests that students’ errors on this item are not random but systematic.
 Students seem to misapplying a rule from first experiences in fractions.
 Smallest fraction=largest denominator (OK). Largest fraction=smallest denominator (NOT OK).
 In other words, there is strong evidence that many students who got the first question right got it right for the wrong reason.
 Teachers need to ask questions better aimed at uncovering misconceptions.
 Where Do Students’ Ideas Come From?
 However, some students describe this shape as an upsidedown triangle even though they know that the orientation does not matter for the naming of the shape, because they are using vernacular, rather than mathematical, language.
 In the world outside the mathematics classroom, however, the word square is often used to describe orientation rather than shape…
 What seems like a misconception is often, and perhaps usually, a perfectly good conception in the wrong place.
 When a child says, “I spended all my money,” this could be regarded as a misconception, but it makes more sense to regard this as overuse of a general rule.
 Some people have argued that these unintended conceptions are the result of poor teaching.
 The key insight here is that children are active in the construction of their own knowledge. correlation <> causation, trees and wind.
 The second point is that even if we wanted to, we are unable to control the students’ environments to the extent necessary for unintended conceptions not to arise.
 2.3 * 10 <> 2.30. But 7 * 10 =70! We could make such a “misconception” less likely to arise by introducing decimals before teaching multiplying singledigit numbers by ten, but that would be ridiculous.
 Thus, it is essential that teachers explore students’ thinking before assuming that students have understood something.
 Pair of equations
 When asked what a and b are, many students respond that the equations can’t be solved. Skills versus beliefs.
 The point here is that had the sixteen in the second equation been any other number at all, provided they had the necessary arithmetical skills, students would have solved these equations, and the teacher would, in all likelihood, assume that the class’s learning was on track.
 Questions that give us insight into student learning are not easy to generate and often do not look like traditional test questions. Indeed, to some teachers, they appear unfair. Example
 The question is perceived as unfair because students “know” that in answering test questions, you have to do some work, so it must be possible to simplify this expression; otherwise, the teacher wouldn’t have asked the question—after all, you don’t get points in a test for doing nothing.
 Example, “which is larger”
 The fact that this item is seen as a trick question shows how deeply ingrained into our practice is the idea that assessment should allow us to sort, rank, and grade students, rather than inform the teacher what needs to be done next.
 Example, what is between water molecules? “water”.
 Questions that provide a window into students’ thinking are not easy to generate, but they are crucially important if we are to improve the quality of students’ learning.
 So the important issue is this: does the teacher find out whether students have understood something when they are still in the class, when there is time to do something about it, or does the teacher only discover this once he looks at the students’ notebooks?
 As noted previously, questions that give us this window into students’ thinking are hard to generate, and teacher collaboration will help to build a stock of good questions.
 Practical Techniques
 Teacherled classroom discussion is one of the most universal instructional practices.
 So although many people assume that American teachers talk too much, they actually talk less than teachers in countries with higher performance. It would appear that how much students learn depends more on the quality than the quantity of talk.
 Less than 10 percent of the questions that were asked by teachers in these [Brown & Wragg, 1993] classrooms actually caused any new learning.
 I suggest there are only two good reasons to ask questions in class: to cause thinking and to provide information for the teacher about what to do next. Example, triangle with 2 right angles.
 The other reason to ask questions is to collect information to inform teaching…
 …American classrooms were characterized by relatively low degrees of student engagement.
 Student Engagement
 Malcolm Gladwell, Outliers, ages of professional hockey players
 Other sports, as well.
 In almost any classroom, some students nearly dislocate their shoulders in their eagerness to show the teacher that they have an answer to the question that the teacher has just asked.
 Highengagement=students working together and using language as a tool, show higher achievement.
 …[Students] who are participating are getting smarter, while those avoiding engagement are forgoing the opportunities to increase their ability.
 This is why many teachers now employ a rule of “no hands up except to ask a question” in their classrooms (Leahy, Lyon, Thompson, & Wiliam, 2005). The teacher poses a question and then picks on a student at random.
 Some teachers claim to be able to choose students at random without any help, but most teachers realize that when they are in a hurry to wrap up a discussion so that the class can move on, they are often drawn to one of the usual suspects for a good answer. Popsicle sticks.
 The major advantage of Popsicle sticks can also be a disadvantage. It is essential to replace the sticks to ensure that students who have recently answered know they need to stay on task, but then the teacher cannot guarantee that all students will get a turn to answer.
 Most teachers realize that being called upon at random will be a shock for students unused to participation in classrooms. However, moving to random selection can also be unpopular with students who participate regularly.
 For other students, random questioning is unwelcome because they are unable to control when they are asked questions.
 Lemov (2010). Coldcalling and No Opt Outs.
 Often, students will say, “I don’t know,” not because they do not know, but because they cannot be bothered to think.
 “phone a friend”, “ask the audience”, “fiftyfifty”…All these strategies derive their power from the fact that classroom participation is not optional…
 Wait Time
 However, the amount of time between the student’s answer and the teacher’s evaluation of that answer is just as, if not more, important.
 Alternatives to Questions
 Asking questions may not be the best way to generate good classroom discussions. Try statements.
 The quality of discussion is usually enhanced further when students are given the opportunity to discuss their responses in pairs or small groups before responding (a technique often called “thinkpairshare”).
 Evaluative and Interpretive Listening
 [John Wooden] was once asked why other coaches were not as successful, and he said, “They don’t listen.
 When teachers listen to student responses, many focus more on the correctness [Evaluative listening] of the answers than what they can learn about the student’s understanding (Even & Tirosh, 1995; Heid, Blume, Zbiek, & Edwards, 1999).
 However, when teachers realize that there is often information about how to teach something better in what students say—and thus how to adjust the instruction to better meet students’ needs—they listen interpretively.
 Question Shells
 There are a number of general structures that can help frame questions in ways that are more likely to reveal students’ thinking. Example “Why is ___ an example of ___.”
 Another technique is to present students with a contrast and then ask them to explain the contrast, as shown in table 4.2.
 HotSeat Questioning
 In hotseat questioning, the teacher asks a student a question and then a series of followup questions to probe the student’s ideas in depth.
 If teachers are to harness the power of highquality questioning to inform their instructional decisions, they need to use allstudent response systems routinely.
 AllStudent Response Systems
 The problem with such techniques [“thinking thumbs” or “fist to five”] is that they are selfreports, and, as we know from literally thousands of research studies, selfreports are unreliable.
 However, a very small change [asking cognitive not affective questions] can transform useless selfreports into a very powerful tool.
 Example students that signal correct or incorrect know they will be followed up with.
 Mercury phosphate equation example.
 Example about the length of a line on a grid.
 Example about incorrect classification of levers.
 In each of these four examples, the teacher was able to ensure both student engagement—after all, it is very easy to tell if a student has not voted—and highquality evidence to help decide what to do next.
 ABCD Cards
 Each student has a set of cards with letters. Example.
 Most students should recognize that A and B represent onefourth, and hopefully, most students will also realize that in diagram C, the fraction shaded is not onefourth.
 The use of multiple correct answers allows the teacher to incorporate items that support differentiation, by including some responses that all students should be able to identify correctly but also others that only the ablest students will be able to answer correctly. Such differentiation also helps keep the highestachieving students challenged and, therefore, engaged.
 Example Heysel Stadium disaster 1985. …she was pleased to see that the class now had a much more complex view of the tragedy…
 One elementary school teacher takes the idea of cards one step further with what she calls “letter corners.”
 ABCD cards can also be used to bridge two lessons. Teacher verified that content covered that day was understood and content for the next day wasn’t understood yet.
 A major difficulty with ABCD cards is that they generally require teachers to have planned questions carefully ahead of time, and so they are less useful for spontaneous discussion….
 Mini Whiteboards
 Whiteboards are powerful tools in that the teacher can quickly frame a question and get an answer from the whole class….
 One teacher wanted to use whiteboards, but there was insufficient money in the school’s budget to acquire these, so instead, she placed sheets of lettersized white card stock inside page protectors to provide a lowcost alternative.
 Exit Passes
 When questions require longer responses, teachers can use the exit passes….
 Exit pass questions work best when there is a natural break in the instruction; the teacher then has time to read through the students’ responses and decide what to do next.
 All these techniques create student engagement while providing the teacher with evidence about the extent of each student’s learning so that the teacher is able to adjust the instruction to better meet the students’ learning needs.
 Discussion Questions and Diagnostic Questions
 Students who say A or B have some understanding but not of the value of the index of a sequence.
 This question can lead to a valuable discussion in the mathematics classroom, since it allows the teacher to challenge the idea that mathematics is a rightorwrong subject.
 The teacher learns little just by seeing which of these alternatives students choose. She has to hear the reasons for the choices. That is why this question is a valuable discussion question, but it is not a good diagnostic question.
 Example
 In some ways, this is a sneaky question, because there are two correct—and four incorrect—responses.
 The crucial feature of such diagnostic questions is based on a fundamental asymmetry in teaching; in general, it is better to assume that students do not know something when they do than it is to assume they do know something when they don’t. What makes a question useful as a diagnostic question, therefore, is that it must be very unlikely that the student gets the correct answer for the wrong reason.
 Science Example
 A teacher who has been focusing on Archimedes’ principle hopes that the students choose B, but there are valid reasons for choosing alternatives.
 Example: physics/forces.
 A seems reasonable, but is naïve.
 B has also an element of truth.
 C was what the teacher had been hoping for, but is a little imprecise.
 D is similar to B
 E is clearly mystical and wrong.
 The first and last responses, therefore, are obviously incorrect and related to wellknown naïve conceptions that students have about the physical world.
 Physics is an unnatural way of thinking about the world—if it were natural, it wouldn’t be so hard for students to learn.
 This question is not really asking, “What’s happening here?” It is asking, “Can you think about this situation like a physicist?”
 History question example. World War II started 1937, 1938, 1939, 1940, or 1941?
 Again, the point is that the teacher learns little about the quality of student thinking from hearing which answer a student chooses; the teacher needs to hear reasons for the choice, and that means hearing from every student in the classroom.
 Example: Diagnostic question in history.
 It is the quality of the distractors that is crucial here—it is only because the distractors are so plausible that the teacher can reasonably conclude that the students who choose D have done so for the right reason.
 Example: Thesis statement question.
 Example: Modern foreign language question.
 Diagnostic questions can be used in a variety of ways.
 They can be used as “rangefinding” questions to find out what students already know about a topic before beginning the instruction.
 [D]iagnostic questions are most useful in the middle of an instructional sequence to check whether students have understood something before moving on. [Hingepoint questions.]
 Hingepoint question is a craft. Two guidelines/principles.
 First, it should take no longer than two minutes, and ideally less than one minute, for all students to respond to the question
 Second, it must be possible for the teacher to view and interpret the responses from the class in thirty seconds (and ideally half that time).
 Example NAEP flights from Newton to Oldtown.
 [S]tudents who think there are a hundred minutes in an hour and those who know there are just sixty get the same answer.
 Earlier in this chapter, I proposed that there are two good reasons to ask questions in classrooms: to cause thinking and to provide the teacher with information that assists instructional decision making.
 Or, to put it another way, ideally it would be impossible for students to get the right answer for the wrong reason
 One way to improve hingepoint questions is to involve groups of teachers. See also Larry Cuban on this point.
 No question will ever be perfect, but by constantly seeking to understand the meaning behind students’ responses to our questions, we can continue to refine and polish our questions and prompts
 A second requirement of questions to assist instructional decision making—much less important than the first but still useful to bear in mind when designing questions—is that the incorrect answers should be interpretable. That is, if students choose a particular incorrect response, the teacher knows (or at least has a pretty good guess) why they have done so.
 Multiplechoice questions are often criticized because they assess only loworder skills such as factual recall or application of a standard algorithm, although as the previous examples show, they can, if carefully designed, address higherorder skills.
 In the classroom, these are much less important considerations, and multiplechoice questions have one great feature in their favor: the number of possible student responses is limited.
 Sometimes it makes sense to administer the question as a series of simple questions.
 Example lines of symmetry. How many does each shape have?
 She doesn’t try to remember how each student responds. Instead, she focuses on just two things: 1. Are there any items that a significant proportion of the class answers incorrectly and will need to be retaught with the whole class? 2. Which two or three students would benefit from individualized instruction?
 She realizes that the incorrect answers may not necessarily indicate poor mathematical understanding.
 In this episode, the teacher administered, graded, and took remedial action regarding a wholeclass quiz in a matter of minutes, without giving herself a pile of grading to do. She does not have a grade for each student to put in a gradebook, but this is a small price to pay for such an agile piece of teaching.
 However, highquality questions seem to work across different schools, districts, states, cultures, and even languages [portability]. Indeed, sharing highquality questions may be the most significant thing we can do to improve the quality of student learning.
