EDU6978: Week 01: Due 2012-07-01


Since NRC (2009) describes formative assessment as a key component in classroom environments that produce effective learning and since Wiliam (2011) describes the five key strategies of formative assessment, lets see how I did in the SAT Math Review course that I conducted this past year. Note that since the rest of Wiliam (2011) is comprised of deep dives on each strategy, my self-assessment below lacks an in-depth rubric.

Strategy Grade Comments.
Learning Intentions and Criteria for Success are Clear A- I think it was implicitly clear from the nature of the course what our learning intentions were, i.e. “to cover math that is going to be on the SAT.”  Criteria for success was also clear, i.e. “students need to answer questions of similar difficulty in order to get a good score on the SAT.”  I ding myself here only because I believe that I never made that explicit, or probed to see if that was explicit.
Discussions, Activities and Learning Tasks Elicit Evidence of Learning C

There was little variety in the activities used in the class. We mostly did lecture and group discussion, with some pairwise discussion.  I ding myself further since the activities we did weren’t eliciting individually any evidence of learning.  Since students alone knew their answer and whether it was correct or incorrect, I couldn’t adjust the tempo or flow of the class relative to mass mis/understanding in the class.

Note that NRC (2009) derives a teaching implication of their key finding that teachers need to cover concepts and examples in depth.  However, due to lack of time, I was not able to do this consistently.

Feedback that Moves Learning Forward B+ When covering questions and answers in class, feedback was immediate and showed students where conceptual understanding or process was flawed. However, I am at a loss to prove that learning was moved forward. For example if a student got a problem wrong, what were the steps they could take to do better next time. That was unclear.
Activating Learners as Instructional Resources for One Another A The class was small, and the students were familiar with one another.  Thus student participation in discussions was good.  Students were regularly called upon to describe their thinking to other students who were having trouble.
Activating Learners as the Owners of Their Own Learning C+ Students were owners of their own learning, since they all knew the SAT was coming, and they knew that this class was not graded.  However, were they activated in that role?  It was suggested that the regular homework was not stressed enough as a tool to help students take control of their learning.  Homework also had answers available so students could think they are owning their learning but perhaps that gave opportunity to not be completely honest with themselves.

I believe I am not alone relative to my other classmates in this class that I will be challenged in implementing formative assessment equitably across the whole class in the time allotted..  Take students that may not have made uniform progress in prior grades relative to the Math Practices (CCSSI, 2009) and getting the whole class to a uniform place for assessment and instruction may be difficult.  Compound that with Meyer’s (2010) recommendation that math and science classes need to practice and model patient problem solving on real-world problem solving, and it sounds like the main challenge for me will be finding the time.  However, for the sake of real learning there is no option to not do formative assessment.


How We Learn (NRC 1999)

In chapter 2 of the first reading (National Research Council [NRC], 1999), three key findings are elucidated as having “strong implications for how we teach (pg 10).”

Three Core Learning Principles (NRC, p. 10)


No reason to think this has changed from 1999 to present. Learners come to grips with the world long before they darken the door of a classroom. Even in the classroom, unless the learning supplants their hard-won intuition from the real-world, then true learning will not occur.


An expert doesn’t just know more, they also have frameworks that help them reconstruct/re-derive relationships/laws that they may have forgotten, and a schema to interpret new data and see new patterns.


That’s what we are talking about, metacognitive approach to instruction is assessment for learning, is formative assessment, where the student has all the information and tools to take control of their own learning.

Three Implications for Teaching (NRC, p. 15)

  1. Teachers must draw out and work with the preexisting understandings that their students bring with them.
    1. Child is not an empty vessel, teacher must reveal student initial conceptions.
    2. Assessments must be frequent, and provide feedback, so that student thinking is modified, refined, and deepened.
    3. Schools of education should teach teachers
      1. how to recognize common preconceptions
      2. how to uncover less common preconceptions
      3. challenge and replace preconceptions
  2. Teachers must teach some subject matter in depth, providing many examples in which the same concept is at work and providing a firm foundation of factual knowledge.
    1. teach in-depth, even at the cost of coverage, but get coverage by coordinating across school years
    2. teachers must know the subject: progress of inquiry, terms of discourse, organization of information, growth of student thinking
    3. teachers need to balance the tradeoff between assessing depth and assessing objectively
  3. The teaching of metacognitive skills should be integrated into the curriculum in a variety of subject areas. Foster a student’s internal dialogue.
    1. Teachers should integrate metacognitive instruction and discipline-based learning.
    2. Schools of education should help develop strong metacognitive strategies for instruction and classroom use.

Bringing Order to Chaos (NRC, p. 18)

Proliferation of teaching strategies can be bewildering. The truth is you need them *all* because you are trying to uncover preconceptions, and there is no universal best teaching practice. Teachers should teach basic skills in a way that students can learn and then apply to problems.


Four Ways of Designing Classroom Environments (NRC, p. 19)

  1. Schools and classrooms must be learner centered
    1. cultural differences affect student learning.
    2. student theories of intelligence (fixed/malleable) can affect effort and motivation (Dweck, 1989).
  2. To provide a knowledge-centered classroom environment, attention must be given to what is taught (information, subject matter), why it is taught (understanding) and what competency or mastery looks like.
    1. Expertise is well-organized knowledge that supports understanding.
    2. Learning with understanding is not memorizing or disconnected facts.
    3. Knowledge-centered environments are not just about activities, it is about doing with understanding.
  3. Formative assessments—ongoing assessments designed to make students’ thinking visible to both teachers and students—are essential. They permit the teacher to grasp the students’ preconceptions, understand where the students are in the “developmental corridor” from informal to formal thinking, and design instruction accordingly. In the assessment-centered classroom environment, formative assessments help both teachers and students monitor progress.
    1. Think and do, not memorize and regurgitate.
  4. Learning is influenced in fundamental ways by the context in which it takes place. A community centered approach requires the development of norms for the classroom and school, as well as connections to the outside world, that support core learning values.
    1. Teachers should build a culture where it is ok to take risks, make mistakes, obtain feedback, and revise.
    2. Teachers need to build a community where students can learn together, and help each other.
    3. Teachers need to create communities that can encourage themselves and their peers.
    4. Schools need to connect learning to other aspects of students’ lives.

Applying the Design Framework to Adult Learning (NRC, p. 23)

Even adults could benefit from using the principles in How People Learn, e.g. professional development programs for teachers:

  1. Are not learner centered
  2. Are not knowledge centered
  3. Are not assessment centered
  4. Are not community centered

Dan Meyer: TEDxNYED (Meyer, 2010)

Five signs that you are doing math reasoning wrong in the classroom:


Goal: Patient problem solving. Why are students impatient?

  1. TV situation comedies (problems resolve in 22 minutes).
  2. Books have problems that are just re-hashes of sample problems.
  3. Books have illustrations/pictures that overlay too much information at once.
  4. Formulation of the problem is key, but we just give problems to students.

Goal: Real world problems in our curriculum.

  1. Take out the extraneous information, make students decide.
  2. Take a video of the problem, “bait the hook”
  3. Get students on a level playing field.
  4. Conversations about error have been great (why does theory not match experiment).

Things you can do to improve your math reasoning in the classroom:


Standards for Math Practice (CCSSI, 2009)

  1. Make sense of problems and persevere in solving them.
  2. Reason abstractly and quantitatively.
  3. Construct viable arguments and critique the reasoning of others.
  4. Model with mathematics.
  5. Use appropriate tools strategically.
  6. Attend to precision.
  7. Look for and make use of structure.
  8. Look for and express regularity in repeated reasoning.

Connecting the Standards for Mathematical Practice to the Standards for
Mathematical Content
The Standards for Mathematical Practice describe ways in which developing student
practitioners of the discipline of mathematics increasingly ought to engage with
the subject matter as they grow in mathematical maturity and expertise throughout
the elementary, middle and high school years. Designers of curricula, assessments,
and professional development should all attend to the need to connect the
mathematical practices to mathematical content in mathematics instruction.

The Standards for Mathematical Content are a balanced combination of procedure
and understanding. Expectations that begin with the word “understand” are often
especially good opportunities to connect the practices to the content. Students
who lack understanding of a topic may rely on procedures too heavily. Without
a flexible base from which to work, they may be less likely to consider analogous
problems, represent problems coherently, justify conclusions, apply the mathematics
to practical situations, use technology mindfully to work with the mathematics,
explain the mathematics accurately to other students, step back for an overview, or
deviate from a known procedure to find a shortcut. In short, a lack of understanding
effectively prevents a student from engaging in the mathematical practices.

In this respect, those content standards which set an expectation of understanding
are potential “points of intersection” between the Standards for Mathematical
Content and the Standards for Mathematical Practice. These points of intersection
are intended to be weighted toward central and generative concepts in the
school mathematics curriculum that most merit the time, resources, innovative
energies, and focus necessary to qualitatively improve the curriculum, instruction,
assessment, professional development, and student achievement in mathematics. (p. 8)

What a lot of jargon, bordering on gibberish!  I would translate thus: “look for the word understand in the standards and use the practices to help measure extent of understanding for that content.”

Framework for K-12 Science Education (NRC, 2012)

On pages 50-53 of this source, the differences between Science and Engineering are described in 6 categories, entitled “Distinguishing practices in science from those in engineering.”

  1. Asking Questions and Defining Problems
  2. Developing and Using Models
  3. Planning and Carrying Out Investigations
  4. Analyzing and Interpreting Data
  5. Using Mathematics and Computational Thinking
  6. Constructing Explanations and Designing Solutions
  7. Engaging in Argument from Evidence
  8. Obtaining, Evaluating and Communicating Information

Anyone who has seen the TV Show “Big Bang Theory” might be familiar with the animosity between 3 of the main characters and Howard, the lowly engineer, who merely implements solutions, or solves experimental problems and doesn’t do original science.  Of all these distinguishing practices, I think that is the defining one the crux of the matter, I actually don’t think these are distinguishing practices at all, if they are worded identically and for the engineer it says:  and then builds something that average people can use, and for the scientist it doesn’t say that.

My question is:  where are the standards for engineering education, the engineering standards?  It seems as though there have been some working groups and draft proposed but I couldn’t find a definitive engineering standards link.

Of course, that begs the question of what the technology education standards are.  Those standards do exist, and have existed since 2000.  I found Technology Education standards downloadable here:

Embedded Formative Assessment (Wiliam, 2011)

  • Chapter 1 Outline (verbatim from author unless italic)
    • Why Educational Achievement Matters
      • Wiliam basically rehashes the arguments that dropouts are worth less to society (as measured by lifetime earnings) and college-educated are worth much more.  My question:  is that all adjusted for the debt that the college educated accrue?
      • “Higher levels of education man lower health care costs, lower criminal justice costs, and increased economic growth
    • The Increasing Importance of Educational Achievement
      • Hourly pay has changed depending on level of educational achievement
      • Dropouts earn less.
      • Higher levels of education are associated with better health…
      • People with more education live longer… (the converse is not necessarily true!)
      • Educational achievement matters for society (taxes, health care costs, criminal justice costs)
      • Higher levels of education are needed in the workplace.  In fact, …young people today are substantially more skilled than their parents and grandparents.
      • Scores on standardized tests…have been rising steadily
      • Only 6 percent of children in 1947 performed as well as the average child of 2001 on the Similarities test.
      • However, the quality of teaching in public schools is, on average, higher than in private schools in the United States…[emphasis mine]
      • …[T]he scores of students attending private schools in the United States were higher than those attending public schools.
      • This, however, does not show that private schools are better than public schools, because the students who go to private schools are not the same as those who go to public schools.
      • Schools have improved dramatically, but the changes in the workplace have been even more extraordinary.
      • In the 1960s and ‘70s, the average workingman (or woman) needed neither to read nor to write during the course of a workday; there were many jobs available to those with limited numeracy and literacy skills, but now those jobs are disappearing.
      • But the speed of the down escalator has been increasing—technology and outsourcing are removing jobs from the economy—and if we cannot increase the rate at which our schools are improving, then, quite simply, we will go backward.
      • Changes in technology actually affect white-collar jobs more than they affect blue-collar jobs.
      • Routine data entry work and jobs as call-center operators were among the first to be outsourced, but now high-skilled jobs are outsourced, too.
      • The one really competitive skill is the skill of being able to learn. It is the skill of being able not to give the right answer to questions about what you were taught in school, but to make the right response to situations that are outside the scope of what you were taught in school. We need to produce people who know how to act when they’re faced with situations for which they were not specifically prepared. (Papert, 1998).
      • Getting every student’s achievement up to 400 (the OECD’s definition of minimal proficiency) would be worth $70 trillion, and matching the performance of the world’s highest-performing countries (such as Finland) would be worth over $100 trillion.  But what would it cost to get there?  Social Cost?  Political Cost?
    • Why is Raising Student Achievement So Hard?
      • …[T]he depressing reality is that the net effect of the vast majority of these measures [to raise standards] on student achievement has been close to, if not actually, zero.
      • [Structural changes:  size]  However, the promise of such smaller high schools was not always realized.
      • In other cases, the potential benefits of small high schools were not realized because the creation of small high schools was assumed to be an end in itself, rather than a change in structure that would make other needed reforms easier to achieve.  [like smaller student to teacher = engagement, =relationships, e.g. looping]
      • Other countries are going in the opposite direction. …[B]ut as yet, there is no evidence that this has led to improvement.
      • [Structural changes:  governance].    As the characteristics of successful charter schools become better understood, it will, no doubt, be possible to ensure that charter schools are more successful, but at the moment, the creation of charter schools cannot be guaranteed to increase student achievement (Carnoy, Jacobsen, Mishel, & Rothstein, 2005).
      • In England, a number of low-performing schools have been reconstituted as “academies”, [but] a comparison with similarly low-performing schools that were not reconstituted as academies shows that they improve at the same rate (Machin & Wilson, 2009).
      • For-profit schools in Sweden, also don’t show a significant improvement.
      • Specialist schools in England (similar to US magnet schools) get more money per student, and thus get better scores, but so do public schools.
      • [Curriculum reform].  Trying to change students’ classroom experience through changes in curriculum is very difficult. A bad curriculum well taught is invariably a better experience for students than a good curriculum badly taught: pedagogy trumps curriculum. Or more precisely, pedagogy is curriculum, because what matters is how things are taught, rather than what is taught.  [emphasis mine]
      • Three levels of curriculum:  intended, implemented and achieved.  The greatest impact on learning is the daily lived experiences of students in classrooms, and that is determined much more by how teachers teach than by what they teach.
      • [Textbooks]  Reviews of random-allocation trials of programs for reading in the early grades and for programs in elementary, middle, and high school math concluded that there was little evidence that changes in textbooks alone had much impact on student achievement.
      • [Scale] Challenges of scale ensures that small pilots almost never perform comparably when rolled out more widely.
      • [Techology]   While there is no shortage of boosters for the potential of computers to transform education, reliable evidence of their impact on student achievement is rather harder to find.
      • [Computers] With the exception of Cognitive Tutor ® Algebra I for 9th graders, there is little evidence that computers as teaching tools really work.  See What Works Clearinghouse page:
      • [Interactive White Boards]  These tools are prohibitively expensive, and need a lot of training to use effectively.
      • Teacher’s aides also don’t help student achievement.
    • Three Generations of School Effectiveness Research
      • The fad of “school effectiveness” was a result of economic thinking applied to schools.
      • You can’t emulate characteristics of an effective school to reproduce that schools effectiveness elsewhere.  Unless you:
        • First, get rid of the boys.
        • Second, become a parochial school.
        • Third, and most important, move your school into a nice, leafy, suburban area.
      • But seriously, all we learned from school effectiveness studies is that differences in school scores is the result of differences in students.
      • This, in turn, means that only 8 percent of the variability in student achievement is attributable to the school, so that 92 percent of the variability in achievement is not attributable to the school.
      • It turns out that as long as you go to school (and that’s important), then it doesn’t matter very much which school you go to, but it matters very much which classrooms you’re in.  [emphasis mine]
      • It turns out that these substantial differences between how much students learn in different classes have little to do with class size, how the teacher groups the students for instruction, or even the presence of between-class grouping practices (for example, tracking). The most critical difference is simply the quality of the teacher.
    • The Impact of Teacher Quality
      • It was never considered that teacher quality could differ so markedly.
      • Teachers have long been treated as a commodity, identical, fungible, low value.
      • Performance related pay for teachers when calculated from test scores is fundamentally flawed because teachers hand off each year, who should get the credit?
      • Paying bonuses to teachers will not improve student test scores.
      • Which surprises economists, by the way.
      • [You must read Sanders & Rivers (1996), it shows that certain teachers had amazing impact relative to other teachers]
      • More recent studies (for example, Rockoff, 2004; Rivkin, Hanushek, & Kain, 2005) have confirmed the link between teacher quality and student progress on standardized tests, and it appears that the correlation between teacher quality and student progress is at least 0.2, and may be even larger (Nye, Konstantopoulos, & Hedges, 2004).
      • In other words, the most effective teachers generate learning in their students at four times the rate of the least effective teachers.
      • Excellence in classrooms is infectious, standard deviation of scores decreases as scores increase.
      • In their work in kindergarten and first-grade classrooms, Bridget Hamre and Robert Pianta (2005) found that in the classrooms of the teachers whose students made the most progress in reading, students from disadvantaged backgrounds made as much progress as those from advantaged backgrounds, and those with behavioral difficulties progressed as well as those without.
      • This last finding is particularly important because it shows that Basil Bernstein was wrong—education can compensate for society provided it is of high quality.
      • Equitable outcomes will only be secured by ensuring that the lowest-achieving students get the best teachers, which, in the short term, means taking them away from the high-achieving students; this is politically challenging to say the least.  [empasis mine]
      • Rather than thinking about narrowing the gap, we should set a goal of proficiency for all, excellence for many, with all student groups fairly represented in the excellent.
    • How Do We Increase Teacher Quality
      • Two options
        • The first is to attempt to replace existing teachers with better ones, which includes both deselecting serving teachers and improving the quality of entrants to the profession.
        • The second is to improve the quality of teachers already in the profession.
      • Deselection is the only option that we really haven’t tried yet, and thus must be the way we should go forward.
      • [Deselection] may not work because, to be effective, you have to be able to replace the teachers you deselect with better ones, and that depends on whether there are better potential teachers not currently teaching.
      • The third problem with deselection is that it is very slow.
      • We [the US versus Finland] can’t afford to turn away anyone who might be a good teacher, so we need to have better ways of identifying in advance who will be good teachers, and it turns out to be surprisingly difficult, because many of the things that people assume make a good teacher don’t.
      • It has been well known for quite a while that teachers’ college grades have no consistent relationship with how much their students will learn, and some have gone so far as to claim that the only teacher variable that consistently predicts how much students will learn is teacher IQ (Hanushek & Rivkin, 2006).
      • Some progress has been made in determining what kinds of teacher knowledge do contribute to student progress.
        • The MKT (mathematical knoweldge for teaching) for elementary school teachers.
        • This suggests that subject knowledge accounts for less than 10 percent of the variability in teacher quality.
      • A study of over 13,000 teachers, involving almost 1 million items of data on over 300,000 students in the Los Angeles Unified School District (LAUSD), found that student progress was
        • unrelated to their teachers’ scores on licensure examinations,
        • nor were teachers with advanced degrees more effective (Buddin & Zamarro, 2009).
        • Most surprisingly, there was no relationship between the scores achieved by LAUSD teachers on the Reading Instruction Competence Assessment (which all elementary school teachers are required to pass) and their students’ scores in reading.
        • So did we discourage some people from teaching that would have on the contrary made excellent teachers?
      • [Extended analogy to finding a good quarterback in the NFL via New Yorker article from Malcolm Gladwell]
      • Although efforts continue to try to predict who will do well and who will not within the NFL, Gladwell suggests that there is increasing acceptance that the only way to find out whether someone will do well in the NFL is to try him out in the NFL.
      • The same appears to be true for teaching.
      • And it will also take a long time!
      • If we are serious about securing our economic future, we have to help improve the quality of those teachers already working in our schools…
    • Conclusion (complete/verbatim)
      • Improving educational outcomes is a vital economic necessity, and the only way that this can be achieved is by increasing the quality of the teaching force. Identifying the least effective teachers and deselecting them has a role to play, as does trying to increase the quality of those entering the profession, but as the data and the research studies examined in this chapter have shown, the effects of these measures will be small and will take a long time to materialize. In short, if we rely on these measures to raise student achievement, the benefits will be too small and will arrive too late to maintain the United States’ status as one of the world’s leading economies. Our future economic prosperity, therefore, depends on investing in those teachers already working in our schools.
  • Chapter 2 Outline (verbatim from author unless italic)
    • The Case for Formative Assessment
      • minute-by-minute and day-by-day formative assessment is likely to have the biggest impact on student outcomes
    • Professional Development
      • Having a masters degree as an elementary teacher makes no difference on student scores
      • It is clear that the value added by a teacher increases particularly quickly in the first five years of teaching…


      • Note that there is no requirement for teachers to improve their practice or even to learn anything. The only requirement is to endure 180 hours of professional development.
      • Teachers need professional development because the job of teaching is so difficult, so complex, that one lifetime is not enough to master it.
      • Andre’ Previn quit one day because he “wasn’t scared anymore”.  Teachers wiould never say that.
      • Even the best teacher fail.
      • No teacher is so good—or so bad—that he or she cannot improve.  That is why we need professional development.
      • However, there is consensus that the “one shot deals”—sessions ranging from one to five days held during the summer—are of limited effectiveness, even though they are the most common model.
      • Learning Styles
        • However, there is little agreement among psychologists about what the learning styles are, let alone how they should be defined.
        • Although a number of studies have tried to show that taking students’ individual learning styles into account improves learning, evidence remains elusive.
        • “Students need to learn both how to make the best of their own learning style and also how to use a variety of styles, and to understand the dangers of taking a limited view of their own capabilities.”  (Adey, Fairbrother, Wiliam, Johnson & Jones, 1999, p 36)
      • Educational Neuroscience
        • Another potential area for teacher professional development—one that has received a lot of publicity in recent years—is concerned with applying what we are learning about the brain to the design of effective teaching.
        • Some of the earliest attempts to relate brain physiology to educational matters were related to the respective roles of the left and right sides of the brain in various kinds of tasks in education and training despite clear evidence that the conclusions being drawn were unwarranted (see, for example, Hines, 1987).
      • Content Area Knowledge
        • After all, surely the more teachers know about their subjects, the more their students will learn
        • Subject matter expertise does not make teacher more successful.
        • What we do know is that attempts to increase student achievement by increasing teachers’ subject knowledge have shown very disappointing results.
        • An evaluation of professional development designed to improve second-grade teachers’ reading instruction found that an eight-day content focused workshop…[had] no impact on the students’ reading test scores.
        • Although the professional development had been specifically designed to be relevant to the curricula that the teachers were using…there was no impact on student achievement….
        • …the relationship between teachers’ knowledge of the subjects and their students’ progress is weak…
        • However, there is a body of literature that shows a large impact on student achievement across different subjects, across different age groups, and across different countries, and that is the research on formative assessment.
    • The Origins of Formative Assessment
      • 1967, Michael Scriven, Formative evaluation
      • 1969, Benjamin Bloom, “By formative evaluation we mean evaluation by brief tests used by teachers and students as aids in the learning process.”
      • “Evaluation which is directly related to the teaching-learning process as it unfolds can have highly beneficial effects…”  (Bloom, 1969, p.50)
      • CGI=Cognitively Guided Instruction, integrating assessment with instruction
      • In the original CGI project, a group of twenty-one elementary school teachers participated, over a period of four years, in a series of workshops in which the teachers were shown extracts of videotapes selected to illustrate critical aspects of children’s thinking. The teachers were then prompted to reflect on what they had seen, by, for example, being challenged to relate the way a child had solved one problem to how she had solved or might solve other problems (Fennema et al., 1996). Throughout the project, the teachers were encouraged to make use of the evidence they had collected about the achievement of their students to adjust their instruction to better meet their students’ learning needs. Students taught by CGI teachers did better in number fact knowledge, understanding, problem solving, and confidence (Carpenter, Fennema, Peterson, Chiang, & Loef, 1989), and four years after the end of the program, the participating teachers were still implementing the principles of the program (Franke, Carpenter, Levi, & Fennema, 2001).
      • The power of using assessment to adapt instruction is vividly illustrated in a study of the implementation of the measurement and planning system (MAPS), in which twenty-nine teachers, each with an aide and a site manager, assessed the readiness for learning of 428 kindergarten students.
      • [Fuchs & Fuchs 1986] found that regular assessment (two to five times per week) with follow-up action produced a substantial increase in student learning.
      • Over the next two years, two further research reviews, one by Gary Natriello (1987) and the other by Terence Crooks (1988), provided clear evidence that classroom assessment had a substantial—and usually negative—impact on student learning.
      • In 1998, Paul Black and I sought to update the reviews of Natriello and Crooks.
      • We concluded that the research suggested that attention to the use of assessment to inform instruction, particularly at the classroom level, in many cases effectively doubled the speed of student learning.
      • “Despite the existence of some marginal and even negative results, the range of conditions and contexts under which studies have shown that gains can be achieved must indicate that the principles that underlie achievement of substantial improvements in learning are robust. Significant gains can be achieved by many different routes, and initiatives here are not likely to fail through neglect of delicate and subtle features.” (Black & Wiliam, 1998a, pp. 61–62)
      • Black & Wiliam did more research in England in 2003.
      • Most of the teachers’ plans contained reference to two or three important areas in their teaching in which they were seeking to increase their use of formative assessment, generally followed by details of techniques that would be used to make this happen.
      • …the teachers could be observed implementing some of the ideas they had discussed in the workshops and could discuss…
      • Nevertheless, in this study, using scores on externally scored standardized tests, the students with which the teachers used formative assessment techniques made almost twice as much progress over the year (Wiliam, Lee, Harrison, & Black, 2004).
    • What, Exactly, Is Formative Assessment
      • Paul Black and I defined formative assessment “as encompassing all those activities undertaken by teachers, and/or by their students, which provide information to be used as feedback to modify the teaching and learning activities in which they are engaged” (Black & Wiliam, 1998a, p. 7).
      • What is notable about these definitions is that, however implicitly, formative assessment is regarded as a process.
      • The difficulty with trying to make the term formative assessment apply to a thing (the assessment itself) is that it just does not work.
      • The assessment that the teacher used—an AP calculus examination—was designed entirely for summative purposes. AP exams are designed by the College Board to confer college-level credit so that students passing the exam at a suitable level are exempt from introductory courses in college. However, this teacher used the assessment instrument formatively—what Black and I have called “formative use of summative tests.”
      • Some people (for example, Popham, 2006; Shepard, 2008) have called for the term formative assessment not to be used at all, unless instruction is improved.
        • 1. The provision of effective feedback to students

          2. The active involvement of students in their own learning

          3. The adjustment of teaching to take into account the results of assessment

          4. The recognition of the profound influence assessment has on the motivation and self-esteem of students, both of which are crucial influences on learning

          5. The need for students to be able to assess themselves and understand how to improve

      • Instead, they suggested that it would be better to use the phrase assessment for learning, which had first been used by Harry Black (1986) and was brought to a wider audience by Mary James at the 1992 annual meeting of the ASCD in New Orleans.
      • “If formative assessment tells users who is and who is not meeting state standards, assessment FOR learning tells them what progress each student is making toward meeting each standard while the learning is happening—when there’s still time to be helpful. (Stiggings, 2005, pp. 1–2)
      • The problem, as Randy Bennett (2009) points out, is that it is an oversimplification to say that formative assessment is only a matter of process or only a matter of instrumentation.
      • The original, literal meaning of the word formative suggests that formative assessments should shape instruction—our formative experiences are those that have shaped our current selves—and so we need a definition that can accommodate all the ways in which assessment can shape instruction.
      • Scenarios
        1. Summer math teacher PD used to improve teaching methods so that students do better on ratio and proportion questions on math standardized tests.
        2. Algebra I teachers look at performance on certain problems on state-wide tests and plan to improve delivery.
        3. Interim tests (every 6-10 weeks) determine which students must attend Saturday sessions.
        4. A quiz is given and examined diagnostically to plan future sessions of a class.
        5. Exit passes on bias in historical sources are used to determine whether class can move on or not.
        6. Students are given cards about literary devices and asked to show the appropriate card when a sample sentence is read.
        7. AP calculus teacher has students graph a function on personal whiteboards.
      • In each of these seven examples, evidence of student achievement was elicited, interpreted, and used to make a decision about what to do next.
      • Each can be considered an example of formative assessment.  (Even though time frames may differ.)
      • [Definition] 
      • An assessment functions formatively to the extent that evidence about student achievement is elicited, interpreted, and used by teachers, learners, or their peers to make decisions about the next steps in instruction that are likely to be better, or better founded, than the decisions they would have made in the absence of that evidence.
      • Points about this definition.
        1. formative is used to describe the function that the evidence from the assessment actually serves
        2. Assessment decisions can be made by teachers, learners, or peers.
        3. The third point is that the focus is on decisions instead of on the intentions of those involved…
        4. The probabilistic formulation (that the decisions are likely to be better) reflects the fact that even the best-designed interventions will not always result in better learning for all students.
        5. Here, the term instruction refers to the combination of teaching and learning, to any activity that is intended to create learning (defined as an increase, brought about by experience, in the capacities of an individual to act in valued ways).
        6. The formative assessment might not change the course of action but instead simply show that the proposed course of action was right.
      • The emphasis on decisions as being at the heart of formative assessment also assists with the design of the assessment process.
      • However, if the formative assessments are designed without any clear decision in mind, then there is a good chance that the information from the assessment will be useless.
      • The alternative is to design the assessments backward from the decisions.  (decision-pull versus data-push).
    • Strategies of Formative Assessment
      • The discussion thus far has established that any assessment can be formative and that assessment functions formatively when it improves the instructional decisions that are made by teachers, learners, or their peers.
      • All teaching really boils down to three key processes and three kinds of individuals involved. The processes are: finding out where learners are in their learning, finding out where they are going, and finding out how to get there. The roles are: teacher, learner, and peer.


      • Key strategies of formative assessment
        1. Clarifying, sharing, and understanding learning intentions and criteria for success.
        2. Engineering effective classroom discussions, activities, and learning tasks that elicit evidence of learning.
        3. Providing feedback that moves learning forward.
        4. Activating learners as instructional resources for one another.
        5. Activating learners as the owners of their own learning.
      • The big idea is that evidence about learning is used to adjust instruction to better meet student needs—in other words, teaching is adaptive to the learner’s needs.
    • Assessment:  The Bridge Between Teaching and Learning
      • Assessment occupies such a central position in good teaching because we cannot predict what students will learn, no matter how we design our teaching.
      • [Denvir experiment (Denvir & Brown, 1986a) with student Jy]
        • Knowledge gaps elucidated.
        • Instruction planned and delivered to address gaps.
        • Surprisingly, on the posttest, Jy could not demonstrate mastery of any of the skills that she had been specifically taught…
        • The skills that Jy acquired were consistent with the hierarchies that Denvir had identified—they just weren’t the skills her teacher had taught, and the same was found to be true for other students in the study (Brown & Denvir, 1986b).
      • This is why assessment is the central process in instruction. Students do not learn what we teach. If they did, we would not need to keep gradebooks. We could, instead, simply record what we have taught.
      • The truth is that we often mix up teaching and learning,
      • After all, what sense does it make to talk about a lesson for which the quality of teaching was high but the quality of learning was low?
      • In some languages, the distinction between teaching and learning is impossible to make…
      • To say that learning is more important than teaching is a bit like saying that traveling is more important than driving.
      • Every action that a teacher takes, provided it is intended to result in student learning, is teaching, but the teacher cannot do the learning for the learner; teaching is all the teacher can do.
      • The influence has shifted from “what am I going to teach and what are the pupils going to do?” towards “how am I going to teach this and what are the pupils going to learn?” (Black, Harrison, Lee, Marshall, & Wiliam, 2004, p. 19)
      • Two extremes
        • [Teacher does all the work.]  That is why I often say to teachers, “If your students are going home at the end of the day less tired than you are, the division of labor in your classroom requires some attention.”
        • [Teacher merely facilitates.]  Presumably, the teachers are just hanging around, hoping that some learning will occur.
        • Teaching is difficult because neither of these extremes is acceptable. When the pressure is on, most of us behave as if lecturing works, but deep down, we know it’s ineffective. But leaving the students to discover everything for themselves is equally inappropriate. For this reason, I describe teaching as the engineering of effective learning environments. And sometimes, a teacher does her best teaching before the students arrive in the classroom.
      • Many teachers have had the experience of creating an effective group discussion task in which the students engage completely in a really tricky challenge that they must resolve.  The only problem is there is nothing for the teacher to do.
      • The teacher’s job is not to transmit knowledge, nor to facilitate learning. It is to engineer effective learning environments for the students. The key features of effective learning environments are that they create student engagement and allow teachers, learners, and their peers to ensure that the learning is proceeding in the intended direction. The only way we can do this is through assessment. That is why assessment is, indeed, the bridge between teaching and learning.
    • Conclusion
      • In this chapter, we learned that the regular use of minute-by-minute and day-by-day classroom formative assessment can substantially improve student achievement. Although many different definitions of formative assessment have been proposed, the essential idea is simple. Teaching is a contingent activity. We cannot predict what students will learn as a result of any particular sequence of instruction. Formative assessment involves getting the best possible evidence about what students have learned and then using this information to decide what to do next.
      • There are five key strategies of formative assessment. The next five chapters probe each of these five strategies in more depth, offering details of research studies that provide evidence of their importance and a number of practical techniques that can be used to implement the strategies in classrooms.



Bennett. (2009).  As cited in Wiliam (2011).

Carnoy, Jacobsen, Mishel, & Rothstein. 2005.

Common Core State Standards Initiative. (2009).  Common Core State Standards for Mathematics.  National Governors Association.  Retrieved June 24, 2012 from

Machin & Wilson. 2009.

Meyer, D. (2010, March 6).  TEDxNYED:  Math Class Needs a Makeover.  Retrieved from (2012, June 24). 

National Research Council. (1999). How People Learn: Bridging Research and Practice. Washington, DC: The National Academies Press.

National Research Council. (2012). A Framework for K-12 Science Education: Practices, Crosscutting Concepts, and Core Ideas. Washington, DC: The National Academies Press.

Papert (1998).

Stiggins (2005).  As cited in Wiliam (2011).

Wiliam, D. (2011).  Embedded Formative Assessment.  Bloomington, Indiana:  Solution Tree Press.

Trackbacks are closed, but you can post a comment.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: