Scoring Teachers: Perversion by Policymakers

Policymakers have perverted teaching, and reduced the evaluation of teachers to a number based on questionable and unreliable data.   Not only do researchers at major universities caution policymakers about using Value-Added Models (VAMs), but using such a system that is based on student test scores will destroy the central character of teaching.

Wordle based on On Teaching by Kahlil Gibran, The Profit.

As a teacher I have always been guided by several core principles to help my students learn.  I believed that my role was to facilitate learning and to help students understand science that would be meaningful to their own lives.  Long ago, I realized that as teachers we need to treat students with respect and communicate genuinely as in person-to-person encounters.  This was crucial if I wanted to create a classroom where students would feel free to collaborate with each other and me.   Secondly I realized that it was important to prize my students, which led to a more trusting environment.  And I learned early in my career that the best teachers in my departments were empathic toward their students.  Trying to understand students from their point of view is an important core condition in my classroom.  Respect, prizing, and empathizing were three core conditions that I found were essential to my own approach to teaching.  The teacher is one that helps lead students to their understandings.  These understanding can not be poured from the teacher’s head to the head’s of the students.  Student needs to construct their own learning. As the poet Kahlil Gibran says in the Wordle shown here, we can can not give our understanding directly to others.


…because much more weight is given to student achievement test scores, we have to wonder how communication between students and teachers will change?  How will the bond of trust between students and teachers be affected?  If one’s evaluation is based on student test scores, won’t this narrow the curriculum to “teach to the test,” and wouldn’t it be smart for teachers to find out which standards (they can’t test them all) are emphasized on high-stakes tests, and center teaching in on these standards? And wouldn’t it be more likely that teachers would prefer not to teach special education, English language learners, and students with behavioral problems?

Evaluating teachers using VAMs isolates teachers, one from another, and further distances teachers from their students.  This is a a perversion.
Continue reading “Scoring Teachers: Perversion by Policymakers”

Charter Schools: Education’s New Elixir?

In her new book, Dr. Lisa Delpit suggests that the original idea of charter school has been corrupted.  She explains that originally, charter schools were designed to be “beacons” for educational excellence.  Charter schools were to be designed to develop new approaches to teaching, especially for the most challenging populations of children.  Their results were to be shared with other public schools.

As Dr. Delpit explains, the initial concept has been corrupted.  She explains:

Now, because of the insertion of the “market model,” charter schools often shun the very students they were intended to help. Special education students, students with behavioral issues, and students who need any kind of special assistance are excluded in a multiplicity of ways because they reduce the bottom line—they lower test scores and take more time to educate properly. Charter schools have any number of ways of “counseling” such students out of their programs. I have been told by parents that many charter schools accuse students of a series of often trivial rule infractions, then tell parents that the students will not be suspended if the parents voluntarily transfer them to another school. Parents of a student with special needs are told that the charter is not prepared to meet their child’s needs adequately and that he or she would be much better served at the regular public school around the corner.

Delpit, Lisa (2012-03-20). “Multiplication Is for White People“: Raising Expectations for Other People’s Children . Perseus Books Group. Kindle Edition.

We have reported on this blog that two major research studies indicate that charter schools do not perform nearly as well as traditional public schools.  In a study published by the Center for Research on Education Outcomes (CREDO) at Stanford, hundreds of charter schools in 15 states and the District of Columbia were studied to find out what was the impact of these charter schools on student learning.

Here are some of their findings from the CREDO study:

  • Of the 2403 charter schools reflected on the curve, 46 percent of charter schools have math gains that are statistically indistinguishable from the average growth among their TPS comparisons.
  • Charters whose math growth exceeded their TPS equivalent growth by a significant amount account for 17 percent of the total.
  • The remaining group, 37 percent of charter schools, posted math gains that were significantly below what their students would have seen if they enrolled in local traditional public schools instead.

Dr. Michael Marder, at the University of Texas has studied not only Texas charter schools, but charter schools in other states including Flordia, New Jersey, New York, and California.  He has found that most charter schools do not do as well as the traditional public schools. Here is a video clip of Dr. Marder explaining his research and findings.  Again, the results do not bode well for supporters of charter schools.

Continue reading “Charter Schools: Education’s New Elixir?”

Suspicions About the Atlanta Journal’s Investigation into Cheating Across the Nation

On Sunday, the Atlanta Journal-Constitution, a Cox newspaper published the results of its investigation into “cheating” in American schools.  The article was entitled Suspicious Scores Across the Nation, and you can read it by following the link. The article was subtitled “Cheating Our Children.”

I was immediately suspicious of the report that the Journal-Constitution published.  They have put into place an agressive team of watchdog reporters, database specialists, and investigative reporters.  In whose interest motivates this team and this newspaper?  After reading the report, I reaffirm my suspicions.  Let me explain.

The data (collection of facts, observations, and measurements) that is available to this team is a database specialists’ fantasy.  Fifty-states provided standardized testing data from 69,000 schools, 14, 743 districts, 13 million students, representing 1.6 million records.  Most states provided the data immediately (public records law), but a few Departments of Education hedged a bit, but eventually sent on their data.

According to the researchers,

For each state, grade, cohort and year, we created a linear regression model, weighted by the number of students in a class, and compared the average score for a class with the score predicted by the model based on the previous year’s average score. We then calculated a p-value — an estimated probability that such a difference would occur by chance — using standardized residuals and the “T” probability distribution, which adjusts the probability upward for classes with fewer students. (Links Mine)

What the researchers looked for was scores rising or falling with probabilities less than 0.05.  These were flagged.  Maybe the bubble sheets were erased and correct answers added?  This would mean that some one cheated.  Wouldn’t it?  Or could there be other explanations?

Is cheating what caused the some scores to change at a probability level that you wouldn’t expect.  According to the Journal-Constitution,

A statistical analysis cannot prove cheating. It can only identify improbable events that can be caused by cheating and should be investigated.

If it smells like cheating, it must be cheating.

But if you read the Journal-Constitution initial article, and one they published today (Cheating our children: AJC’s testing investigation spurs action), the only suggestion that the Journal makes is that the T-scores might indicate cheating.  And indeed, in the latter article, Georgia U.S. Senator Johnny Isakson said that this report is troubling, and if the districts don’t do something about it, then the Governors should.  If they don’t, then “I don’t think Congress should look the otherway.” I am sorry, but the Congress has been looking the other way since the NCLB Act was signed into law ten years ago.

Here is what the Atlanta Journal staff had to say about their research model:

A statistical analysis cannot prove cheating. It can only identify improbable events that can be caused by cheating and should be investigated.

Ideally we would look at how individual student test scores change from year to year, but federal privacy regulations precluded access to that data. The approximate cohorts we used were the only available substitute. It is unlikely that two groups of students in a cohort are perfectly identical. Urban districts in particular have high student mobility.

In the model that the data analysts used, average scores from one year to the next were not necessarily based on the same population of students.  They didn’t have student data.  The only had average data for a school by subject. So it is possible that they are making predictions based unreliable dats.

High student mobility might give us a clue about other possibilities to explain score changes given the demographics of the schools and districts that were highlighted.  For example if you look at the districts that were highlighted in the paper (Amarillo, Atlanta, Baltimore, Chicago, Dallas, Detroit, East St. Louis, Fresno, Gary, IN, Houston, Los Angeles, & Mobile County, AL), all of these districts reported that at least 64% of the students were eligible for free or reduced-price meals.  The poverty concentration of these schools averaged 80%.  Mobility is high in these districts.

Critique of Methodology used by the Atlanta Journal Data Analysts 

Gary Miron, professor of education at Western Michigan University was one of four academics or test specialists that advised USA Today and its affiliated Gannett newspapers newspapers on a multistate analysis of irregularities in assessment data.  In his article published on The Answer Sheet, Dr. Miron writes about some of his concerns regarding the research methodology used by the Atlanta-Journal Constitution.

My review, however, yielded serious concerns about the data used, the methods of analysis employed, and the conclusions drawn. I shared these concerns with journalists at the Dayton Daily News, which is one of the Cox affiliates involved in this story.

To be clear, the Cox analysis may accurately detect large variations in assessment results from year to year. But my own analysis of the data suggests that these irregularities are less likely due to actual cheating than due to mobility in student population (recall the lack of student-level data). Although the Cox news articles on this study offer a disclaimer that their analysis does not actually prove cheating, this disclaimer should be expanded considerably.

The evidence is that the Atlanta Journal may be practicing sensationalism in throwing the data up against the wall, and hoping that some of it will stick.  The map they published is impressive, but it raises more questions than it answers, and the writers of the article were quick draw the “cheat” card. “Cheating” was mentioned 57 times in the first article of 3000 words, and 31 times in the second article, which was 978 words long. Mobility or any mention of changing student populations was mentioned once in both articles.  One might raise the issue that the manner in which the paper published this article is self-serving.  Are they really interesting in uncovering serious issues facing our nation’s schools, or are they more interested in selling newspapers, and receiving awards for investigative reporting.  I really don’t know.  But the thought crossed my mind.

Dr. Miron conveys his concerns about the methodology that was used by the newspaper.  Here are his concerns:

  • As noted, the analysis is based on school-level data and not individual student-level data. Accordingly, it was not possible to ensure that the same students were in the group in both years.
  • The analysis of irregular jumps in test scores should have been coupled with irregularities in erasure data where this data was available.
  • The analysis by Cox generates predicted values for schools, but this does not incorporate demographic characteristics of the student population.
  • The limited details available on the study methods made it impossible to replicate and verify what the journalists were doing. Further, the rationale was unclear for some of the steps they took.
Stop Weighing the Cow, and Start Teaching Students to Think, Problem Solve, and Do Inquiry
Since NCLB was enacted in 2002, all students and schools have been subjected to high-stakes testing.  Its been relentless, and it has had it effect on teaching and learning.  Teaching has been reduced, especially in high poverty schools to teaching to the test.  Students in many of these schools have a full diet of worksheets, drill and practice, and fill in the blanks.  Inquiry, and problem solving is no where to be seen.  It must be seen, however.
In her new book, Multiplication is for White People: Raising Expectations for Other People’s Children, Dr. Lisa Delpit made this important statement early in her book:

I am angry at the machinations of those who, with so little knowledge of learning, of teachers, or of children, are twisting the life out of schools.

She adds that the current use of standardized tests, which promotes competition between schools, and used to evaluate teachers and principals to determine their salaries bring out the worst in adults.  It should be no surprise to the Atlanta Journal staff, but they fail to pursue the data other than to blame schools for the current state of our schools. This is not about cheating.

According to Dr. Delpit, “the problem is that the cultural framework of our country has, almost since its inception, dictated that “black” is bad and less than and in all arenas “white” is good and superior.”

To point to urban schools as the harbinger of cheating and surprising test results leaves us with many questions.  Why don’t we talk about how we got ourselves into this mess in the first place?  What has been the effect of the NCLB policy that has turned schooling into test factories, only interested in finding out what low level of knowledge students might have learned in school?  We simply can not continue to test with out really reforming education.  And simply writing common standards (objectives) for all kids regardless of where they live, and with little teacher flexibility or input from teachers simply will only reinforce the authoritarian nature of schooling.  Testing students doesn’t make them smarter, any more than weighing the cows makes them heavier.

I agree with Dr. Delpit when she calls for us to create excellence in urban classrooms, and she suggests that we must do the following:

  • Recognize the importance of a teacher and good teaching, especially for the “school dependent” children of low-income communities.
  • Recognize the brilliance of poor, urban children and teach them more content, not less.
  • Whatever methodology or instructional program is used, demand critical thinking while at the same time assuring that all children gain access to “basic skills”—the conventions and strategies that are essential to success in American society.
  • Provide children with the emotional ego strength to challenge racist societal views of their own competence and worthiness and that of their families and communities.
  • Recognize and build on children’s strengths.
  • Use familiar metaphors and experiences from the children’s world to connect what students already know to school-taught knowledge.
  • Create a sense of family and caring in the classroom.
  • Monitor and assess students’ needs and then address them with a wealth of diverse strategies.
  • Honor and respect the children’s home cultures.
  • Foster a sense of children’s connection to community, to something greater than themselves. (Delpit, Lisa (2012-03-20). “Multiplication Is for White People”: Raising Expectations for Other People’s Children . Perseus Books Group. Kindle Edition.)

What do you think about the Atlanta Journal’s inference that cheating might be the reason for the unexpected changes in test scores?  Do you think they are helping solve a problem, or are they creating problems?  What can be done to improve urban schools?  

End Punitive Testing: Occupy the DOE in DC, March 30 – April 2

United OPT OUT NATIONAL is organizing an Occupy the Department of Education (DOE) in Washington, D.C. March 30 – April 2, 2012.

Anthony Cody, on his blog Living in Dialog, interviewed two education activists who are part of the leadership spearheading the Occupy the DOE next week.  Anthony interviewed Ceresta Smith, a 23-year veteran teacher and National Board Certified teacher, and Timothy D. Slekar, founding member of United Opt Out and associate professor teacher education.  You can read his full interview here.

As Tim Slekar, one of the organizers of the Occupy the DOE, explains the goal of this action:

Our ultimate goal is the end of punitive high stakes testing in our public schools (The research clearly supports this goal). However, since those of us all have backgrounds in education, we plan on making sure that anybody attending any of the day’s events will leave with a deep understanding of how punitive high stakes testing is harming our public schools (children, teachers, and communities). This is not an easy task considering almost all mainstream media outlets refuse to give the real experts a forum to address the citizens of America. We hope that anyone attending leaves with a determination to “tell the story” of how this DOE and its secretary are actively participating in the dismantling of our public schools and the demonization of the teaching profession.

On the Art of Teaching Science blog we have argued  that the actions of the DOE, especially with the details of the Race to the Top, and the enticement to give some flexibility to the NCLB, are undermining the professional work of teachers and causing havoc with the lives of students and parents.  The perpetual testing machine that was put into motion by the NCLB has caused the worst high-stakes test cheating scandal in American history.  Today, the Atlanta Journal-Constitution released maps and data showing that DOE forced testing has created conditions that are unprecedented in our educational history: wide scale testing scores of cities across the nation.

The uncovering of this data—that it appears that bubble test answer sheets were changed—stops short at really getting at the real cause of this.

In Atlanta, for example, its primarily teachers that are being charged.  Yes, some administrators have been identified, but by and large the finger pointing is toward the teachers.  Nonsense.  In one schools (Parks Middle, Atlanta), it was the principal that coerced and bullied teachers into becoming part of a “culture of fear and intimidation.”  And why did the principals create this environment of fear in the their schools?  It came from the top, although the top administrator in Atlanta has not been charged.  And why did she create this environment of fear in such a large school district?

Raising test scores, especially in school in urban schools, is the only goal that is worthy of attention according to the Georgia Department of Education, and the U.S. Department of Education.  Testing, which can be used as a formative assessment tool to help student learn, has been turned into a curse.

It’s not the teachers who should be targeted in the nationwide testing “scandal,” but the perpetrators of the policy that is destroying the American public school system.  And who would that be?  It would be the U.S. Department of Education policy administrators who carryout the NCLB Act, The Race to the Top, and now the ESEA Flexibility Waivers, AND the department of education in each of the U.S. states.

Ceresta Smith, one of the activist educators interviewed by Anthony Cody in the run-up to the Occupy DOE in Washington, had this say when asked what we can do to get involved to support them?:

First and foremost, people must educate themselves as to what is going on in regards to the damage that is being done to public education. Then, they must speak out and engaging in activist stances by donating time and money to organizations such as Save Our Schools and Parents Across America. It is also important that they petition their legislators to draft and support public school, teacher and student friendly bills. Last, they must work hard to elect political candidates that support quality public education and work equally hard to get rid of those that don’t.

The Occupy the DOE in Washington is an important protest activity.  Pass this post on to let others know what the Occupy movement is doing next week in Washington.

Do you agree that the perpetrators of the policy that is destroying American education lies with the DOE, and state DOE’s?  What do you think?


I dare you to measure the “value” I add

Guest Post by Donna McKenna

This was first published on March 8 on Donna McKenna’s blog, No Sleep ’til Summer.  She is an elementary ESL teacher passionate about language learners and language learning.  Published with permission.

Note: This post is a continuation of yesterday’s post entitled Quality Teaching: We’re Looking in the Wrong Place.  Evaluating teachers using Value-Added Modeling is shameful and degrading, not only because VAM is unscientific and a fraud, but because it does an enormous disservice to professional teachers and their students. Today, we’re looking in one of the right places, into Donna McKenna’s ESL classroom.

After reading this post, I hope that you will be motivated to share your comments about the real value that teachers add to students’ lives.


Tell me how you determine the value I add to my class.

Tell me about the algorithms you applied when you took data from 16 students over a course of nearly five years of teaching and somehow used it to judge me as “below average” and “average”.

Tell me how you can examine my skills and talents and attribute worth to them without knowing me, my class, or my curriculum requirements.

Tell me how and I will tell you:

  • How all of my students come from different countries, different levels of prior education and literacy, and how there is no “research-based” elementary curriculum created to support schools or teachers to specifically meet their needs.
  • How the year for which you have data was the year my fifth graders first learned about gangs, the Internet, and their sexual identities.
  • How the year for which you have data was the year that two of my students were so wracked by fear of deportation, depression and sleep deprivation from nightmares, that they could barely sit still and often fought with other students. How they became best of friends by year end. How one of them still visits me every September.
  • How that year most of my students worked harder than ever, (despite often being referred to as “the low class” or “lower level” within earshot of them), inspiring me and the teachers around us, despite the fact that many of these same students believed they could never go to college because of their immigration status.
  • How that year many of my students vaulted from a first to third and fourth grade reading levels but still only received a meaningless “1? on their report cards because such growth is not valued by our current grading system.
  • How that was the year I quickly gained 6 new students from other countries and had my top 3 transferred out in January to general education classrooms because my school thankfully realized I shouldn’t have 32 students in a multilevel self-contained ESL class.
  • How the year for which you have data was the year that two of my students, twins who had come from China just the year before to live with parents they hadn’t seen since they were toddlers, finally started to speak in May. And smile. And make friends. How they kept in touch with me via edmodo for two years after leaving my school.
  • How that year I taught my class rudimentary American Sign Language for our research project; inspired and excited, they mostly taught themselves the Pledge of Allegiance, songs for our school play, John Lennon’s Imagine, and songs for graduation, all in ASL. Then we created an online video-translation dictionary using their first language, English, and ASL. They wrote scripts for skits we videotaped to teach many of these words in context.

Continue reading “I dare you to measure the “value” I add”