Why Candidates for Governor and State School Superintendent of Georgia Should Oppose High-Stakes Testing?

Latest Story

"Creative Commons Gravel Hill School 1950" by Erin Nekervis is licensed under CC BY-ND 2.0
“Creative Commons Gravel Hill School 1950” by Erin Nekervis is licensed under CC BY-ND 2.0

In an earlier post, I challenged candidates for state school superintendent to oppose the Common Core State Standards.  Today, I am writing to candidates for Governor and State School Superintendent of Georgia to oppose High-Stakes testing.  If they would, they’d open the door to a new paradigm of assessment that would improve education in Georgia beyond their wildest dreams.

Since 2001, the U.S. Department of Education has mandated the annual testing of children as young as 7 years old in mathematics and reading, and most states have added mandated high-stakes testing in writing, science, and social studies.

The American Education Research Association states that it is a violation of professional standards to make decisions about students’ life chances or educational opportunities on the basis of test scores alone.  Yet schools around the state of Georgia and indeed the rest of the country use end-of-the-year tests to make crucial decisions about whether students move on or not.  Additionally, these high-stakes tests have become an even greater burden on students because they know that the test results will be used to grade their teachers.

There is no easy answer to explain why we have an educational system that puts students in harm’s way by the continuous and unparalleled testing program.  When we read the newspapers soon after the release of international, national, or state tests, the emphasis is on who came in first, or who is at the top of the leader board.  No Child Left Behind and the Race to the Top have perpetuated an educational model based on competition and winning.

In some cases, officials will do what ever it takes to make sure they either win, or make the cut so that they place high on the leader board.  The Atlanta Journal-Constitution exposed the cheating scandal in Atlanta, and revealed that cheating (erasing wrong answers and changing to the correct answers on student test forms) was taking place in most cities and states.

Am I advocating the banning of high-stakes testing because it might lead to cheating.  No.

I am advocating banning high-stakes testing because it does not improve student learning, nor does it help teachers change their instruction to improve student learning.   Most of the those who advocate high-stakes testing believe that American education is failing, and that the fundamental goal of schools is improve achievement scores, and the only way to know if that has occurred is to use high-stakes standardized tests every year, and compare the scores from one year to the next.

But, if we do compare the test results from one year to the next, the results are quite astonishing.  First, we discover that in general, academic performance has gradually increased over time.  Secondly, we do see variation in average scores from one year to the next, but the variation is within expected statistical limits.

To give evidence that you might want to use with your constituents or potential voters, I am going to use a few graphs that were prepared by Mr. Ed Johnson, which were published on this blog earlier this year.  I am also going to use charts from the Anne E. Casey Foundation, and pull together data from various state and federal agencies.

NAEP Trial Urban District Assessments (TUDA)

The National Assessment of Educational Progress (NAEP) provides some of the most reliable data on student learning.  The tests given by NAEP are low-stakes, and an individual student takes only part of the test, so they don’t spend hours sitting for the exam.  NAEP has been studying American education since 1969.

About a decade ago, NAEP launched a study of urban school districts which they refer to as TUDA.  They provide telling results that I think will help you with the case of abolishing high-stakes tests.

The four graphs shown below in Figures 1 and 2 were prepared by Mr. Ed Johnson, an expert on W. Edwards Deming’s system of profound knowledge and how to transform organizations that result in continuous improvement.  He also is an expert in using facts to generate flow charts that help us understand how a system is working.

Figure 1 plots math scores for 21 cities over a ten-year period (for a list of the cities, follow this link).  Note that the scores fall within what are called upper control limits and lower control limits.  In no case do scores fall outside these predicted levels.  Yes, there is variation in the scores.  But they are within expected limits, and the variation is small.  For example, the green and red dots follow the Atlanta Public Schools (APS) and the Austin Independent School District (AISD).  If you show this graph to citizens in Georgia and ask if these graphs support the idea that our schools are failing, what is the answer? The answer is No.

Figure 1. Control Charts for NAEP TUDA Math Grades 4 & 8, 2003 - 2013
Figure 1. Control Charts for NAEP TUDA Math Grades 4 & 8, 2003 – 2013. Source Ed Johnson

Figure 2 plots reading scores for the same 21 cities over 11 years. Again, note that for the most part, the scores for each district fall within the expected limits, except for five points of measurement.  Each is labeled and as you see, only Charlotte and Hillsborough fall outside on the 4th grade reading TUDA results.  Think about this.  On only these five instances can we show significant variation from what we expect on the reading test.  The obvious thing to do, is to ask, what are these two districts doing, and how might what they are doing apply in other places.  It might be worth studying their system of education.

But, the real discovery here is to look at all math and reading TUDA results.  There are roughly 408 points of measurement shown in these four graphs, and in only five instances was the variation outside the range expected.   That is 0.012 percent. The systems of teaching math and reading in these 21 cities is predictable and consistent.

We can also see that there are no major swings in the test results.  When we send kids to school, we have a very good idea what to expect.  Another way to say this is that the system is performing as expected.

Or better yet, our teachers are doing it!

But there is always a need for improvement.  In Ed Johnson’s and W. Edwards Deming’s world of human systems, there is always the expectation for improvement.  The methods of improvement do not include the outright firing of department heads, or rank and file workers, any more than would we think that firing principals and teachers and bringing in uncertified and inexperienced teachers would help the situation.   But this is exactly what the Georgia Department of Education mandates when schools “fail” to meet the standards two years in a row.  Schools in this situation are labeled “turnaround schools.”

Here is what you need to know.  The high-stakes testing model is designed to make it very difficult for some schools, especially those schools where most of the students are eligible for free or reduced lunches (a statistic used to identify the poverty level of a school).  We know that students in less affluent schools will not do as well on these tests as students attending affluent schools.  It’s an unsustainable situation because these schools and their neighborhoods are punished by either closing the school or labeling it a turnaround.  But this is a sure ticket for financial rewards for charter management companies and teacher temp agencies including Teach for America and the New Teacher Project.  Follow this link to find out a better way to help these schools.

Labeling schools as failures is not sustainable.  It will not improve instruction. It represents an inaccurate interpretation of testing, and it is perverting a system that should be helping families, rather than punishing them.

Figure 2. NAEP TUDA, Reading, 4th & 8th Grade, 2002 - 2013.  Source: Ed Johnson
Figure 2. NAEP TUDA, Reading, 4th & 8th Grade, 2002 – 2013. Source: Ed Johnson

If We Were to Ban High-Stakes Tests?

Ok.  As a candidate for governor or state school superintendent in Georgia, and you were to go around the state campaigning for the banning of high-stakes tests, the odds are you would be elected.  You will be surprised who will support you, but you will need to tell the rest of the story.

Yes, you will support the idea of banning high-stakes tests.  But you need to clear that you are not suggesting that teachers and administrators all of a sudden stop assessing students.  Nothing could be further from the truth.

If you have been a teacher, you know that the assessment system  you use in your classroom has a major impact on student learning and classroom behavior.  Assessment is an integral aspect of teaching.  As teachers we assess students during every class session, and interaction that we have with them.  Teachers know that assessment, used as part of instruction, does indeed help student learning.  This is not an opinion.  One of the foremost researchers on assessment is Professor Paul Black, King’s College, London and he has found that formative assessment strategies do improve learning for students.  Formative, unlike the high-stakes tests that the government mandates, are embedded in instruction.  In my view, formative assessment is assessment for learning, not of learning.

Formative Assessment

Formative assessment are tools and methods that teachers use to humanize learning, and give students opportunities to apply their learning, and to engage in activities that involve communication, problem solving and team work–the kinds of skills and abilities that are important today, and will be tomorrow.

As a candidate for governor or state school superintendent you should listen to your most important constituency, and this is the professional teachers in public schools.  Last year, there were more than 111,000 teachers in Georgia teaching 1.6 million students.

So, what would happen if you said to nearly 1.6 million students (and their parents) and 111,000 teachers that the state would no longer need high-stakes tests.  Would the education system crumble?  Would students all of sudden not be motivated to learn?

It would thrive.  And it would free up a lot of money that would otherwise go to corporations.

According to a Brookings Institute report, the cost of testing in the U.S. exceeds $1.7 billion.  But according to the report, that is only for payments to testing vendors who also score the tests.  But what is the cost for lost instructional time.  In Georgia, the CRCT exams and high-school end-of-course exams take three-four weeks during the year.  So for about 4 weeks or about 12% of the school year, high-stakes exams dominate the school experience.

What’s the cost of 4 weeks of testing?  Well according to the Brookings Institute, about $600 billion is spent on education in the U.S. per year.  According to the Georgia Budget & Policy Institute, the 2015 fiscal year budget for K-12 education is $7.95 billion.  The cost of high-stakes testing is at least $950 million.

Every school in the state already has experts on assessment, and these educators need to be supported to collaborate with colleagues to develop assessment methods that will improve student learning, and increase student’s love of learning.

We know from many research studies that the best predictor of success in college & career (college & career is the favored purpose of reformers such as Bill Gates) are grades, not test scores.   Teachers are in the best place to assess their students.  Not only are they able to create their own tests, but there are multiple resources available that teachers already use to help their students learn.

Imagine if you were a high school biology teacher, and it was announced that the state would no longer need high-stakes tests.  How would this affect your teaching, and especially your relationship with your students.  One obvious difference is that the curriculum will expand because you no longer would be forced to teach to the test.  No longer would the students in your class be required to take tests that would be used to not only to decide whether they progress to the next science course, but the tests would no longer be used to decide if you keep your job.

In the next post, I’ll go into more detail about what assessment would look like in this alternative paradigm.

In the meantime, are you willing to discuss the possibility of returning the education of students into the hands of professional teachers?


Will the Atlanta Public Schools Make Foolish or Wise Decisions?

Used with permission of Dustin at http://dustn.tv/.
Used with permission of Dustin W. Stout at http://dustn.tv/.

I found this poster while reading over on Dustin Stout’s amazing work on creative design and social media. Dustin was reminded of this Bertrand Russell quote by a colleague, and superimposed it on a photo to make this Russell poster.

Russell’s Fools, fanatics and wise men quote resonated with the most recent posts on this bog that have focused on the Atlanta Public Schools (APS).

Bertrand Russell says that “the problem with the world is that fools and fanatics are always so certain of themselves, but Wiser Men so full of doubts.”

Education reform in this country is being led by people  who have convinced themselves that they know what is best for the education of American students.  They shout their claims with money and shenanigans with legislators on the end of their whips.  Whether these people are fools or fanatics is for you to decide.

However, I do know that there are “wise persons” out there who question the claims of these education reformists, and in their work, they document how the persistent attacks on schools, teachers, and students is unwarranted and unsupportable in research.

For years, I’ve written about how these wise persons have confronted the spread of the foolish ideas that dominate public education.  The Atlanta Public Schools are facing two decisions that will affect the future of the school system.  

The APS is about two make two critical decisions:

1. Select a new superintendent: For more than a year, the APS Superintendent Search Committee has sought candidates for the position.  They found one candidate, who they have presented to the district, and in a few days, the APS School Board will vote to accept the candidate as its new superintendent.

2. The APS also has to make a choice to run as (a) an Investing in Educational Excellence” (IE2) district, (b) a “Charter System,” or (c) a “Status Quo” system.


We’ve been making foolish decisions about how and for what reasons schools should be part of their communities. We’ve listened to the shouts of fools and fanatics who have ignored the signals and the research on learning and instead have injected schools with the virus, GERM (Global Education Reform Movement).  According to Pasi Sahlberg, GERM is a virus that has infected many nations in their march to “reform” education.  In his view, GERM is characterized by standardization (Common Core), core subjects (math, reading, science), teaching to the test, corporate management style, and test-based accountability.    In the meantime, keep in mind that GERM has created opportunity for the flow of money into education beyond our wildest dreams.   Between the Gates Foundation, the National Governors Association, the Council of Chief State Officers, the U.S. Department of Education, billions of dollars has been earmarked to support the GERM virus.  Gates has already spent $2.3 billion on the Common Core, and the U.S. Department of Education has invested more than $4.5 billion in the Race to the Top.  

The GERM virus is enhanced by these organizations who think that the academic performance of students is the most important outcome of K-12 education.

Wise Persons Speak

Yet, there are voices of wise persons who are often drowned out by fools and fanatics.  They oppose the further spread of GERM, and doing something about it.

In many communities, parents are opting their children out of state mandated tests. In other districts, teachers are refusing to give their students some of the mandated tests. All around the country people are raising heck about the Common Core.

In some cities, teacher’s unions, with support from parents, have gone on strike opposing government policies that continue to shut down schools inflicted by that virus.

Closer to home, I’ve reported on the voices of two wise persons, Andrew Young and Ed Johnson.

Leaders In Atlanta

Andrew Young and Ed Johnson are citizens of Atlanta whose work in politics, civil rights, and education are examples of the kind of wise person thinking that follows.

Andrew Young spoke out at the trial of Beverly Hall, suggesting to the judge to end the travesty of trying to convict dr hall and dozens of teachers from the APS. The continuation of this trial will only prolong the healing that is happening in Atlanta. There is evidence that the Atlanta test erasure scandal was an unintended consequence of the GERM virus.  Young is correct to say that we have to move beyond this trial.

Ed Johnson has been speaking and writing (documented here) about how the APS could improve the quality of education in its schools, and has backed up his ideas with research on systems theory, and evidence that achievement scores have not and will not change unless we view education as a humane, public, moral and ethical enterprise.

More than anyone that I know, Ed Johnson has offered the APS advice on how to bring a new sense of leadership to its schools, and how to view and run schools based on Edward Deming’s ideas.  In this regard, he believes schools can not be improved by trying to improve the parts separately.  It is a sure path to failure.

For example, some advocates of educational reform believe that student achievement can be improved by weeding out the bad teachers.  Millions of dollars have been invested in using student high-stakes test scores to check teacher performance using a technique called Value Added Measure (VAM).  Teachers whose VAM scores are low can be identified, and according to these experts, teachers with low scores must be bad teachers.  Getting rid of “defects” in any system will not improve the system or the part that was identified.  Instead, a better investment would be to ask how can we improve the quality of teaching, and what can be done to improve the teaching of all educators.

Systems thinking means that all parts of a school system are interdependent and must be viewed as a whole. The Atlanta Public Schools (APS) is a system of connected and interdependent parts, and to improve the quality of the APS, it is critical to look at the APS as a whole.

Atlanta is set to make two decisions, and these decisions are being made based on the Global Education Reform Model, and Ed Johnson, and others are urging the APS School Board to re-examine their views in light of a quality and systems view of education.

In correspondence with the Atlanta Board of Education, Ed Johnson has worked with other activists in Atlanta to urge the ABE to postpone its vote on a new superintendent until the Search Committee provides a slate of nominees, and not only one choice.  How in a democratic society and for a public education system, can we enable a group to dictate the outcome of such a critical decision.

In a recent email, Ed Johnson talks about the choice that all systems in Georgia have to make about how it will run.  The Georgia Department of Education has mandated that each district choose one of three ways to operate: an Investing in Educational Excellence” (IE2) district;  a “Charter System;” or a “Status Quo” system.

He writes:

During a recent Atlanta Board of Education (ABE) meeting, a board member or the superintendent – someone – mentioned Atlanta Public Schools (APS) needing to prepare a five-year strategic plan. This was mentioned without stating why the plan is needed.

Well, might it be because an APS five-year strategic plan would be necessary in order for the ABE to choose to operate APS as an “Investing in Educational Excellence” (IE2) district, or perhaps a “Charter System?”

The ABE must understand it has moral, ethical, civic, and democracy relevant responsibilities to know to choose but one option: “Status Quo,” pursuant to O.C.G.A. § 20-2-80, Requests for increased flexibility; requirements, paragraph (b), specifically.

The other two options, IE2 and Charter System, violate all that represents moral, ethical, civic, and democratic mindedness and ideals in service to the common good.

Moreover, the options IE2 and Charter System symbolize a kind of abject poverty of thinking some elected officials have brought into our legislative processes, such as this from Rep. Ed Setzler:

“[N]ot only does choice provide options to ALL parents (wealthy parents already have choice), but it creates market forces that reform the nearby public schools who must perform at a higher level to stay attractive to parents; as I watch the NFL playoffs this evening, I wonder if the New England Patriots would be as good as they are if they never had compete[d] with other world-class teams to be successful.”

Then he warns us not to fall for the foolish and intimidating behavior of those who advocate the GERM.  He says:

Please, let not “Choice” proponents, such as Rep. SetzlerRep. Alisha Thomas Morgan, and similar others fool and intimidate with their derisive rhetoric against “Status Quo.” Obviously, they mean for the label “Status Quo” to elicit a repulsive reaction. And that they do this simply exposes the depth of their abject poverty of thinking, in spite of being highly educated, one might suppose.

Contrary to what one or two ABE members and others contend, there is no loss associated with choosing “Status Quo.” There is, however, unfettered opportunity to learn to improve the APS. But that is the rub, as to improve APS requires learning more so than mandating, cooperating more so than competing (à la Rep. Setzler, above), leading more so than managing. It is these matters that challenge “Choice” proponents’ abject poverty of thinking that evidences a laziness to learn to provide for the continual improvement of our public education systems.

Atlanta should reject becoming either an investment or charter district.  It needs to rethink in a systems view the education, such as those described by the work of the Finnish educator, Pasi Sahlberg or the British educator Peter Barnard.  Ed Johnson writes:

But call it what they will, “Status Quo” is the only choice the ABE must consider and choose. “Status Quo” is the only choice that offers the opportunity for the APS to genuinely recover from its recent messes and then go on to leapfrog abject poverties of thinking, such as that of Representatives SetzlerMorgan, and similar others.

The ABE needs to think differently about the future of the APS.  It needs to heed the remarkable thinking of wise persons such as Ed Johnson.  They might also consult with Diane Ravitch, Lisa Delpit, or Deborah Meier.

Technology as Cure-All for Standards, and Even Snow Days


Screen Shot 2014-03-17 at 7.27.41 PM

Technology is viewed by some as the elixir or cure-all for education, and school districts, with lots of money available through grants such as Race to the Top, technology investments from organization such the Gates Foundation, and law edicts,  have embraced technology as a magic bullet.  Virtual classrooms, digital textbooks, flipped classrooms (use of video), lecture-based content websites are examples of the types of technologies that have emerged.  Could it be that these are Trojan Horses being used to drive the Common Standards?

Ed Johnson an Atlanta systems educator and advocate for quality education wrote to me today and reinforced the last two blog posts connecting the Gates Foundation, Common Standards and technology.  He pointed out that the Atlanta Public Schools are using technology by setting up an “Inclement Weather Makeup Materials” website.

In “Why Bill Gates Defends the Common Core,” I argued that,

There is a growing body of evidence that the Common Standards are not the solution to make America more competitive, to make kids smarter in math, reading and science, and any of the other ills that have been cast upon the education system. I’ve reported on this blog that independent research questions the efficacy of a standard-based approach to education as it is now conceived. The standards-based system is a top-down authoritarian system that disregards the professional decision-making ability of classroom teachers. I’ve reported research by Wallace that shows that this authoritarian accountability system is a barrier to teaching and learning.

And in “Is Technology the Trojan Horse of the Common Standard’s Movement?,” it was added,

It is quite clear that Gates is investing (his term) in technology in schools. It’s no surprise. But we must keep in mind the word technology is a seductive term, especially when used in the context of schools. But the history of top-down technology projects has not served classroom teachers very well. Too often, the technology is used to replace what was already going on in classrooms, or to use a tablet as a textbook.

Ed Johnson, then asked us to consider this:

Atlanta Public Schools has developed a comprehensive plan to increase time and opportunity for students to receive the critical instruction lost this school year during the district’s six inclement weather days.”  APS Launches Virtual Classroom for All Students, Reported by East Atlanta Patch

Is Inclement weather being used as a Trojan Horse to carry out Bill Gates’ technology-dependent common standards?

To explore this a bit more, I went to the Inclement Weather Makeup Materials website, which is shown in Figure 1.   There are links, such as, for 3rd thru 5th and 6th thru 12th make up materials.  When we dig deep into the site, we finally come to content links for language arts, math, science, and social studies.   It’s not a very imaginative way to engage students in make up activity.  I have to wonder why this is being done in the first place.

Figure 1. Inclement Weather Makeup Materials Site. Source: APS
Figure 1. Inclement Weather Makeup Materials Site. Source: APS

For example,  the 3rd grade science link brings you to a three column page.  The first two columns are 3rd Grade science performance standards written in technical language of the Common Standards.   The last column lists Online Learning Support and Activities for the standards.  Some links take you to sites where students have to watch commercials, while others take you to “activities” that lack any sense of wonder, imagination, or inquiry.

Figure 2. Example of an Inclement Weather Make-up Site. Source: APS.

Figure 2. Example of an Inclement Weather Make-up Site. Source: APS.

It seems to me that it might be better to ask students to read an interesting book, and not spent time doing these types of activities. What do you think?

Is the Atlanta School Board Going to Think Differently?

Is the Atlanta School Board (APS) going to think differently? Its composition is different than it was a year ago. Six of the nine member board were elected to the Board in November.

On January 14th, the Atlanta School Board met to discuss the nature of the school system, and to draft an APS Statement of Purpose.

In addition to the members of the board, and Dr. Cathy Mincberg of the Center for Reform of School Systems (CRSS), Mr. Ed Johnson was in attendance, and participated in some aspects of the meeting. (Note: Mr. Johnson, an advocate for quality public education, ran for a seat on the board in the last election, but was not elected. However, he has for years been actively collaborating with Atlanta school officials, including the board).

Keep in mind that Atlanta is in the middle of the process to hire a new school superintendent. According to Ed Johnson, the new board is interested in formulating a statement of core beliefs and strategies to aid in the selection process.

In a letter shared by Mr. Johnson after the January 14th meeting, it appears as if some of the board members are interested in thinking differently. Here is part of Mr. Johnson’s letter:

It will be my pleasure to offer, by your invitation, a recap of my observations of your work today with Dr. Cathy Mincberg of the Center for the Reform of School Systems (CRSS) to help you articulate core beliefs and strategies for interviewing and selecting a new superintendent.

For now, I say thanks to ABOE member Matt Westmoreland for inviting me to comment on an aspect of your proceedings. I also say thanks to ABOE Chair Courtney English for going with my suggestion to synthesize your collective wisdom into a Wordle (“word cloud”) to be examined for noteworthy words that might come together in an APS Statement of Purpose. I hope to share that Wordle with others; it is quite interesting, both for words that stand out and words that do not stand out. Dr. Mincberg had solicited and written down, in some detail, your collective wisdom, save that of ABOE member Jason Esteves who was absent.

APS Board Wordle: Be to Every Community…

The wordle below is a kind of summary of the “collective wisdom” of the board during a the meeting.  Ed Johnson has used the “collective wisdom” depicted in the wordle to write a statement of purpose for the Atlanta Public Schools.

In a letter to the APS Board, Mr. Johnson had this to say:

Good afternoon, ABOE Members and Dr. Grant:

Again, your Wordle (“word cloud”) is quite interesting.  Because it has been so begging of my attention, I have drafted from it a sample APS Statement of Purpose.  Kindly see that attachment.

Again, it is just a sample, an idea for you to consider. Still, I am by bcc, here, also sharing the attachment with the greater community with the invitation to reply to you with their ideas about it.

Too, how about this?  Present the APS Statement of Purpose along with your Wordle to each superintendent candidate finalist and ask: “What story do you see in this?”

Screen Shot 2014-01-17 at 6.15.20 PM

Reading Assignment

Another important part of this meeting is that Ed Johnson is a “systems thinker,” and has written and spoken about why schools need to embrace the world view of systems thinking to improve schools.  Clearly, his synthesis of the APS Board wordle into this statement sets the tone for what education in Atlanta could become.

Be to Every community where Students, Teachers, and Parents Gather to Engage in Joyful Learning within a Wholesome Culture—Ed Johnson

To help the APS Board members understand the nature of systems thinking as it relates to school, he said this:

On the matter of the superintendent search, kindly allow me the pleasure to give each of you a copy of the book “The System Thinking School: Redesigning Schools From the Inside-Out (Leading Systemic Improvement),” by Peter A. Barnard.

Expect 13 copies to arrive from amazon.com addressed to Attn: Dr. Howard Grant.  The total shipment may arrive in parts at different times.  Nonetheless, the total shipment will also include a copy for Dr. Alexis Kirijan and Mr. Steve Smith as well as for the Superintendent Search Committee Chair, Ms. Anne Crammer.  As always, there are no strings attached, and everyone may do with her or his copy as she or he pleases.

But in this case I do offer this advice:  Read in the book BEFORE you hire the new APS superintendent.

Thinking Different

It appears as if the APS Board is off to an interesting start.  The book that Mr. Johnson has ordered for each member is a powerful document on school reform.  But to embrace the ideas in Peter Barnard’s new book, The Systems Thinking School, the leaders in Atlanta will have to think different.  Here is opening paragraph of the Foreword to Barnard’s book:

Peter Barnard might as well have called this book Thinking because that is what he invites readers to do. It is time, he argues passionately, to set aside all the tinkering that sits at the heart of modern school improvement and reform and recognize that the problem is not performance. The problem is the linear system we are stuck with.  Barnard, Peter A. (2013-09-19). The Systems Thinking School: Redesigning Schools from the Inside-Out (Leading Systemic School Improvement) (Kindle Locations 34-37). R&L Education. Kindle Edition.

What do think about these events? Will the Atlanta School Board take a different path when they come to the fork in the road?

In Spite of the “System,” Urban Teachers Have a Record of Success

In spite of the “System” the evidence is that urban teachers have a record of success, not one that is spiraling down.  The present state of reform of American education is based on the idea that American students are doing poorly, and this will lead to disastrous economic consequences, and the loss of American’s place in the global economic competition.

But, education (for our students) should not be a competition.  There is no need to have winners and losers as outcomes of the school experience.  Education is about learning, and in an environment that has as its core belief that learning is the fundamental goal of schooling.  Students are living in the present, and their school experience should be based on their lives now, and should not be based on furthering the economic prosperity of society.  Schooling should not be based on job training, career readiness or college entry.  It should be based on fostering the creative and innovative aspects of youth, and create school as a learning environment designed to help students learn to collaborate, work with others to solve problems, and engage in content from the arts and the sciences that has personal meaning.

We’ve been told that urban education in America needs to be saved by pouring advise and money from the élite and influential corporations and philanthropic groups.  The problem is that these groups are focused on only one set of outcomes that all come down to increasing student academic performance measured by high-stakes examinations.

I want to show here that urban teachers have held their own for the past decade and half in spite of the problems they face in their schools day-to-day.  They not only have held their own, but the evidence shows that academic performance of their students (in mathematics at the 8th grade) in the example below has slowly but surly increased as shown in Figure 1.  As you can see, in Atlanta, students at the 25th, 50th and 75th percentile increased performance on NAEP tests given from 2003 through 2013.

Figure 1. Atlanta NAEP 8th Grade Mathematics Scores for Selected Percentiles 2003 - 2013.
Figure 1. Atlanta NAEP 8th Grade Mathematics Scores for Selected Percentiles 2003 – 2013.

Reading and Math in Urban Schools

Take a look at the next four figures (Figures 2 – 5).  They were compiled by Mr. Ed Johnson in his study of the NAEP Trial Urban District Assessment (TUDA).  Johnson, who is a student W. Edwards Deming, examined the TUDA results through a Deming Lens.  A Deming Lens means that to understand the behavior of a system, one must look at the system.  Breaking down a system into its parts (goals, policies, finances, curriculum, teachers, administrators, parents, directors) loses one’s ability to understand the system.

Each of the graphs below shows the behavior of these four systems over ten years.  You will notice that there is variation in the achievement scores of students in reading (grades 4 and 8) and mathematics (grades 4 and 8) from one testing period to the next.  But the variation is within upper and lower limits that would be expected in each system.

Causes of Variation in Scores

According to W. Edwards Deming 94% of the variation is due to the nature of the system, not the people who work in or make the system work.  Only 6% are attributable to special causes.  (W. Edwards Deming. The New Economics for Industry, Government, Education (p. 33). Kindle Edition).

As you look over the graphs you will see ONLY FOUR instances where the variation in scores lies outside the Upper Control Limit (UCL), and then only in 4th grade reading  Examples include: Charlotte, 2009, Austin, 2011, Charlotte, 2011, and Hillsborough, 2011.  Except for these four instances, all the variation is due to the nature of the system.

Figure 2. Location of Cities in the Trial Urban District Assessment; Gold triangle-higher than large city; Circle--not different; Blue triangle-lower than large city. Source: Nations Report Card
Figure 2. Location of Cities in the Trial Urban District Assessment; Gold triangle-higher than large city; Circle–not different; Blue triangle-lower than large city. Source: Nations Report Card.  Click on Map for more details.

The graphs below plot reading and math scores for 21 school urban school districts.  Mr. Johnson highlighted Atlanta (in red) and DC Public Schools (purple).  As you can note in the following graphs, achievement scores in reading and math for Atlanta and DC Public Schools fell within the Upper and Lower Control limits.  There is no radical change in scores, either up or down.  It appears that the teachers in these urban schools are doing the job they were hired to do and that is help their students learn how to read, and do mathematics.  And they’ve done this in spite of all the issues that surround schools in urban communities.

In systems thinking, as Mr. Johnson would tell us, there are two types of causes of variation in any system.  The most important cause of variation in any system is what we call “common causes” of variation that is really a function of the system itself.  Examples of common cause variation will fall within control limits on a graph (as shown below in Figures 2 – 5). Examples of common causes that influence variation (scores on tests, for example, or graduation rates) include 

  • High percentage of children from low SES groups.
  • Where the school is located. It’s zip code.
  • Age of the school building.
  • Size of the school system.
  • Underpinning policies, practices, procedures of the school which determines it’s culture.
  • Inadequate resources.

According to Deming, nearly all outputs of schooling are the result of common cause variation, and these would include drop out rates, achievement test scores, violence, bullying, gang activity, low self esteem, attitudes, under performance and literacy skills.

Defying Gravity

When we examine a school system from a systems thinking view, these outputs are causes by the day to day effects of common causes of variation.  As Deming and other systems thinkers, such as Ed Johnson would say, trying to seek achievement scores beyond what see in the graphs (Figures 2 – 5) is to “defy gravity.”  Reformers have charged ahead as if they can “defy gravity” and have put the blame of not improving test scores in the wrong place.

Managers (administrators) and workers (teachers) are not “common cause” variables.  However, since schools are based on a linear factory model, “reformers” ignore common causes, and instead claim that teachers and administrators can overcome the challenges posed by common causes.  When reformers insist on market reforms, and they don’t work, they blame the teachers and principles.  And to make matters worse, they use student test scores (which are the result of common causes) to evaluate teachers on the basis of false assumptions about schooling.

We think that the present system of reading and mathematics is fairly stable.  The output in reading and math (as measured by a test score) vary little, and one can make predictions about future reading and math output.

reading 4th

Screen Shot 2014-01-11 at 8.27.50 PM

Screen Shot 2014-01-11 at 8.28.18 PM

Screen Shot 2014-01-11 at 8.28.31 PM

We will explore systems thinking in future posts.  But for now, what do think about the analysis of the NAEP TUDA data as compiled by Ed Johnson?