Measuring Great Teachers
What makes greats teachers? Does holding a master’s degree make one a better teacher? Do the best teachers hail from elite universities? Did they earn high GPAs in college? Did they major in the subject they are teaching? How much does experience matter? Do traditional, university-based teacher-preparation programs produce the best teachers, or are alternatively certified teachers just as good?
The difficulty from a policy perspective is that the relationship between readily quantifiable attributes–such as a teacher’s highest degree attained or level of experience–and student outcomes is unsubstantial. Many policy makers advocate increasing the quality of teaching, but there is considerable debate over the best way to measure and improve teacher quality. In other words, it is very clear that good teachers make a difference, but what’s unclear is how to truly measure a good teacher.
Teachers and teachers’ unions have widely criticized the value-added approach, arguing that test scores are not good indicators of teacher quality. However, many reformers argue that value-added ratings are some of the most accurate indicators for evaluating teachers and improving student performance.
In 2011, the Los Angeles Times released a searchable database of over 11,000 Los Angeles Unified School teachers, ranked by their VA ratings. The newspaper got access to the data through California’s Public Records Act — and hired a seasoned education analyst to crunch the numbers.
The response started out predictably. The local teachers’ union called for a boycott of the paper. But more than 1,100 teachers answered the paper’s invitation to see their data before it came out. Arguably, a newspaper is not the most ideal forum for teachers to receive performance feedback, however, the more important question is: Why did it take a newspaper to do what the school district should have done years ago?
Research dating back to the 1966 release of Equality of Educational Opportunity (the “Coleman Report???) shows that student performance is not directly related to school quality, but more so teacher quality, which was found to account for the largest portion of the variation in student test scores than all other characteristics of a school.
Much of the research published since the Coleman Report has confirmed the findings that high-quality teachers raise student performance, suggesting that the most important thing a school can do to increase student performance is provide its students with good teachers.
Parents have always worried about where to send their children to school; but the school, statistically speaking, does not matter as much as which adult stands in front of their children. Teacher quality tends to vary more within schools—even supposedly good schools—than among them.
But it’s difficult to identify excellent teachers in a reliable, objective way. The Coleman Report’s finding was based on the influence of a set of quantifiable teacher characteristics, such as years of experience, education levels, and performance on a vocabulary test. Since then, due in large part to the availability of new data sources that link and track teachers and students over a number of years, researchers have been able to estimate the overall contribution of teachers to student learning. This includes not only the effect of easily measurable attributes, such as experience and degrees obtained, but also the effect of harder to measure intangible attributes, such as a teacher’s enthusiasm and skill in conveying knowledge.
An excerpt from Atlantic Magazine, What Makes a Great Teacher
Teach for America, a nonprofit that recruits college graduates to spend two years teaching in low-income schools, began outside the educational establishment and has largely remained there. For years, it has been whittling away at its own assumptions, testing its hypotheses, and refining its hiring and training. Over time, it has built an unusual laboratory: almost half a million American children are being taught by Teach for America, and the organization tracks test-score data, linked to each teacher, for 85 – 90 percent of those kids. Teach for America keeps an unusual amount of data on its 7,300 teachers—a pool almost twice the size of the D.C. system’s teacher corps, and until now, has kept its investigation largely to itself.
Steven Farr is Teach for America’s in-house professor, so to speak. His job is to find and study excellent teachers, and train others to get similar results. Starting in 2002, Teach for America began using student test-score progress data to put teachers into one of three categories: those who move their students one and a half or more years ahead in one year; those who achieve one to one and a half years of growth; and those who yield less than one year of gains.
As Teach for America began to identify exceptional teachers using this data, Farr began to watch them. He observed their classes, read their lesson plans, and talked to them about their teaching methods and beliefs. He and his colleagues surveyed Teach for America teachers at least four times a year to find out what they were doing and what kinds of training had helped them the most.
Right away, certain patterns emerged. First, great teachers tended to set big goals for their students. They were also perpetually looking for ways to improve their effectiveness. For example, when Farr called up teachers who were making remarkable gains and asked to visit their classrooms, he noticed he’d get a similar response from all of them: “They’d say, ‘You’re welcome to come, but I have to warn you—I am in the middle of just blowing up my classroom structure and changing my reading workshop because I think it’s not working as well as it could.’ When you hear that over and over, and you don’t hear that from other teachers, you start to form a hypothesis.??? Great teachers, he concluded, constantly reevaluate what they are doing.
Superstar teachers had four other tendencies in common: they avidly recruited students and their families into the process; they maintained focus, ensuring that everything they did contributed to student learning; they planned exhaustively and purposefully—for the next day or the year ahead—by working backward from the desired outcome; and they worked relentlessly, refusing to surrender to the combined menaces of poverty, bureaucracy, and budgetary shortfalls.
But when Farr took his findings to teachers, they wanted more. “They’d say, ‘Yeah, yeah. Give me the concrete actions. What does this mean for a lesson plan?’??? So Farr and his colleagues made lists of specific teacher actions that fell under the high-level principles they had identified. For example, one way that great teachers ensure that kids are learning is to frequently check for understanding: Are the kids—all of the kids—following what you are saying? Asking “Does anyone have any questions???? does not work, and it’s a classic rookie mistake. Students are not always the best judges of their own learning. They might understand a line read aloud from a Shakespeare play, but have no idea what happened in the last act. “Strong teachers insist that effective teaching is neither mysterious nor magical. It is neither a function of dynamic personality nor dramatic performance,??? Farr writes in Teaching as Leadership, a book coming out in February from Farr and his colleagues.
In 2007, 24 percent of Teach for America teachers moved their students one and a half or more years ahead, according to the organization’s internal reports. In 2009, that number was up to 44 percent. That data relies largely on school tests, which vary in quality from state to state. When tests aren’t available or sufficiently rigorous, Teach for America helps teachers find or design other reliable diagnostics.
Once teachers have been in the classroom for a year or two, who is very good—and very bad—becomes much clearer. But teachers are almost never dismissed. Principals almost never give teachers poor performance evaluations—even when they know the teachers are failing.
Ideally, schools would hire better teachers to begin with. But this is notoriously difficult. How do you screen for a relentless mind-set?
When Teach for America began, applicants were evaluated on 12 criteria (such as persistence and communication skills), chosen based on conversations with educators. Starting in 2000, the organization began to retroactively critique its own judgments. What did the best teachers have in common when they applied for the job?
Once a model for outcomes-based hiring was built, it started churning out some humbling results. “I came into this with a bunch of theories,??? says Monique Ayotte-Hoeltzel, who was then head of admissions. “I was proven wrong at least as many times as I was validated.???
For years, Teach for America also selected for something called “constant learning.??? As Farr and others had noticed, great teachers tended to reflect on their performance and adapt accordingly. So people who tend to be self-aware might be a good bet. “It’s a perfectly reasonable hypothesis,??? Ayotte-Hoeltzel says.
What did predict success, interestingly, was a history of perseverance—not just an attitude, but a track record. In the interview process, Teach for America now asks applicants to talk about overcoming challenges in their lives—and ranks their perseverance based on their answers. Angela Lee Duckworth, an assistant professor of psychology at the University of Pennsylvania, and her colleagues have actually quantified the value of perseverance. In a study published in TheJournal of Positive Psychology in November 2009, they evaluated 390 Teach for America instructors before and after a year of teaching. Those who initially scored high for “grit???—defined as perseverance and a passion for long-term goals, and measured using a short multiple-choice test—were 31 percent more likely than their less gritty peers to spur academic growth in their students. Gritty people, the theory goes, work harder and stay committed to their goals longer. (Grit also predicts retention of cadets at West Point, Duckworth has found.)
But another trait seemed to matter even more. Teachers who scored high in “life satisfaction???—reporting that they were very content with their lives—were 43 percent more likely to perform well in the classroom than their less satisfied colleagues. These teachers “may be more adept at engaging their pupils, and their zest and enthusiasm may spread to their students,??? the study suggested.
In general, though, Teach for America’s staffers have discovered that past performance—especially the kind you can measure—is the best predictor of future performance. Recruits who have achieved big, measurable goals in college tend to do so as teachers. And the two best metrics of previous success tend to be grade-point average and “leadership achievement???—a record of running something and showing tangible results. If you not only led a tutoring program but doubled its size, that’s promising.
Knowledge matters, but not in every case. In studies of high-school math teachers, majoring in the subject seems to predict better results in the classroom. And more generally, people who attended a selective college are more likely to excel as teachers (although graduating from an Ivy League school does not unto itself predict significant gains in a Teach for America classroom). Meanwhile, a master’s degree in education seems to have no impact on classroom effectiveness.
The most valuable educational credentials may be the ones that circle back to squishier traits like perseverance. Last summer, an internal Teach for America analysis found that an applicant’s college GPA alone is not as good a predictor as the GPA in the final two years of college. If an applicant starts out with mediocre grades and improves, in other words, that curve appears to be more revealing than getting straight A’s all along.
Last year, Teach for America churned through 35,000 candidates to choose 4,100 new teachers. Staff members select new hires by deferring almost entirely to the model: they enter more than 30 data points about a given candidate (about twice the number of inputs they considered a decade ago), and then the model spits out a hiring recommendation. Every year, the model changes, depending on what the new batch of student data shows. Farr is more hopeful each year. “When I see not a handful, not dozens, but hundreds of people being successful in a world where most people think success is not possible, I know it can be done,??? he told me.
If school systems hired, trained, and rewarded teachers according to the principles Teach for America has identified, then teachers would not need to work so hard. They would be operating in a system designed in a radically different way—designed, that is, for success.
Read the entire article at What Makes a Great Teacher?
Overall Impact
The effect of a good teacher on a child’s life is monumental. The influence of teacher quality was found to persist for years after a student had a particular teacher.
Economists Eric Hanushek, John Kain, and Steven Rivkin estimated that, at a minimum, variations in teacher quality account for 7.5 percent of the total variation in student achievement–a much larger share than any other school characteristic.
In financial terms, replacing a teacher whose true VA is in the bottom 5% with a teacher of average quality would generate lifetime earnings gains worth more than $250,000 for the average classroom. On the other hand, “If you leave a low value-added teacher in your school for 10 years, rather than replacing him with an average teacher, you are hypothetically talking about $2.5 million in lost income,” said Friedman. (Chetty, et all. 2011)
More recently, policy makers have sought to isolate teachers’ contributions to student performance and assess how much of their overall contribution can be associated with measurable teacher characteristics.
Race to the Top, a contest created to spur innovation and reform in K-12 education at the state and local level is one example. To qualify, states must first remove any legal barriers to linking student test scores to teachers. To win money, states must also begin distinguishing between effective and ineffective teachers—and consider that information when deciding whether to grant tenure, give raises, or fire a teacher or principal. States are awarded points for satisfying certain educational policies, such as performance-based standards for teachers and principals, complying with nationwide standards, promoting charter schools and privatization of education, and computerization.
For teachers, Race to the Top means increased access to professional development opportunities and an equitable evaluation and compensation system that will reward teachers for their ability to positively shape the lives their students.
Given the enormous contribution of good teachers to the lives of students, one would think the organizations that represent teachers would welcome a program that praises great teachers and pushes under performing ones out. This is not necessarily the case. Acknowledging reality would require teacher groups to make distinctions between good and bad teachers and hold them accountable for their performance, something they seem unwilling to practice, much less institutionalize.
Great teachers make a great difference; poor teachers hurt a child’s life chances. Isn’t that all we need to know to embark upon a serious effort to reward good teachers and encourage poor teachers out of the profession? Instead, we tend to attribute the gifts of great teachers to some mystical quality that we can recognize and admire—but not replicate. The great teacher serves as a hero but never, ironically, as a model for change.
