Truth Be Told: Power, Money and the Georgia Opportunity School District

Truth Be Told: Power, Money and the Georgia Opportunity School District

Some politicians not only seek office, they relish in the power that elected officials have once they get there.  There is also a lot of money in politics, and there is money to be made, especially if you have connections.  You know what I mean?

The Georgia Opportunity School District is a politically motivated plan to enable the Georgia Governor’s office to take at least 20 schools per year out of the hands of local public schools, fire the principal and nearly 1/2 of the faculty at each school, and then turn the schools over to a for-profit charter management company which will come in create charter schools.  There is power and money here.  New Orleans did this just before Katrina, and we now know that destroying the public education system was a disaster, and the devil is in the details of recent NAEP test results.

Governor Nathan Deal is at the center of this effort. He adores the New Orleans Recovery School District.  He took a group of cronies on a junket last winter on the dime of a private company that stands to profit from Deal’s Opportunity School District.

Deal has, without any research evidence to support his view, decided that there are schools in Georgia that need to be rescued, and the best way to do that is to copy plans that have been enacted in New Orleans (New Orleans’ Recovery School District (RSD), and Tennessee.  These plans have been shown to be ineffective and have instead ripped the public schools in question from local control, and turned them over to outside charter groups.  In New Orleans, there is documented evidence that the RSD has been a failure.

University of Arizona researchers Francesca López and Amy Olson, using NAEP data, compared achievement between charter schools and public schools. The study compared charters in Louisiana, most of which are in New Orleans, to Louisiana public schools, controlling for factors like race, ethnicity, poverty and whether students qualified for special education. On eighth-grade reading and math tests, charter-school students performed worse than their public-school counterparts by enormous margins—2 to 3 standard deviations (please see “10 Years After Katrina, New Orleans’ All-Charter School System Has Proven a Failure, In These Times, August 2015)

Maybe this research was not available to Governor Deal, and the officials at the Georgia Department of Education.


Dr. Julian Vasquez Heilig collated research in August 2015 from Louisiana authors including Jason France, Mike Deshotels, Mercedes Schneider, Francesca Lopez, and Amy Olson.

In the research reported by Dr. Heilig, Louisiana had the largest disparity in student achievement between charters and traditional public schools.  Most of the charters in Louisiana are in New Orleans.

What was Deal and others in state government thinking when they modeled the Georgia Opportunity School District after the New Orleans’ Recovery School District?

Well, how about power and money.

When politicians such as Nathan Deal use questionable ethics, and little to no research to make a sweeping changes in Georgia education, it is our responsibility to question Deal, and vote NO on question 1 on the November 8th ballot.

The Opportunity School District is a politically charged football that is providing just the kind of outcome that unethical politicians love to have a hand in (and perhaps a hand out).

Questions for the Governor

  • Governor Deal, why don’t you tell the truth about the Opportunity School District?
  • Tell us who is being enriched by your plan, and why is it that your relatives are benefiting financially from the OSD?
  • Will you follow the same plan carried out in New Orleans in which they laid off thousands of staff and teachers?
  •  Will you tell us how the plan will be financed, and how much it will cost the citizens in Georgia?

Give us a shout, or email me at

Why Achievement Test Scores are Poor Indicators of Student Learning and Teacher Effectiveness

The U.S. Department of Education (ED) has established a single variable as the way to reward and punish schools, teachers, students and their parents.  The fact that I have used the terms “rewards” and punishments” is evidence enough that the ED is stuck in 19th century psychology.

In 2001, the Congress approved the No Child Left Behind Act which mandated the testing of all students in reading and math.  Immediately, this set in motion the most devastating impact on curriculum in the elementary schools by narrowing the curriculum, and putting such emphasis on reading and math.

In 2009, the Congress approved the Race to the Top Fund (RT3), which earmarked about $4.5 billion for a U.S. competition among the 50 states and the District of Columbia.  Of these entities, only 18 were winners.  The rest lost, except for four states which choose not to compete).

The Race to the Top, in my view, is even worse for education than the NCLB.  In the RT3, achievement test scores are given even more importance because those states that got the money were required to tie student test scores to teacher evaluation using the Value Added Modeling (VAM) system.

Many states, even those that did not receive RT3 money now require at least 50% of a teacher’s evaluation be based on the VAM scores generated by a mythical statistical model.  If you think I am kidding, here is the formula for determining a teachers worth as measured by adding value to student learning.

 Figure 2. The statistic value-added model (covariate adjustment model) used to evaluate Florida teachers.

Figure 1. The statistic value-added model (covariate adjustment model) used to evaluate Florida teachers.

Aside from the fact that VAM scores are unreliable, often the scores of very competent teachers end up being at the bottom of the list.  Further, the tests upon which the VAM is calculated measure only a very small aspect of student learning.  In fact, much of what we think is really important in school–communication skills, ability for work collaboratively with others to solve problems, creative thinking, empathy, and ethics–are not measured on achievement tests.

Why does the ED insist on this simple and behavioristic model of teaching?  It does so because it thinks that school is like a factory, and runs much like a machine.  Some call this mechanistic thinking.  Everything can be broken down into components, such as teacher behavior, teacher training, computers in the classroom, number of students in the class, access to technology, standards, academic tests, courses, homework, etc.   Mechanistic thinking leads to a “fix it” mentality.  That is, we can fix the problem of schooling by changing one or more of these variables.

The big problem in the minds of the mechanistic thinkers, who I am also going call the Neo-School Reformers, such as Bill Gates, Michelle Rhee, Joe Klein, and Arne Duncan, is that they believe that American schools are inferior to schools in other nations, especially countries including Finland, and most of the Asian nations.  Our schools are inferior, and they prove it by citing test scores on PISA and other international tests.  But they don’t tell you the rest of the story.

The Neo-School Reformers solution to what ails our schools is the Global Education Reform Model (GERM).  Although not named by Gates and associates, it was described by one of Finland’s leading educators, Dr. Pasi Sahlberg.

There is a growing body of research that shows that the GERM model is an ineffective model of educational reform.  As Sahlberg points out, GERM is primarily practiced by the North Atlantic Alliance of Schools (primarily the U.S. Europe, and Australia).

Indeed, if you compare the PISA test results of these nations, its difficult to distinguish one from the other.

Thinking In Terms of Systems Theory

The Neo-education reforms are “heads in the sand” reformers.  They fail to look around.  They can’t.  Their necks are stuck in the muck of their own arrogance, and ignorance.  They fail to take their heads out of the box of a classroom or a school, and think about the larger ecosystem in which the school is placed.  They really get mad at teachers or education researchers if they bring up out-of-school factors that might affect student achievement.  They have a code or a motto: No Excuses Education (NEE).

Here is the thing. I’ve learned from a group of scholars, including Ed Johnson, Diane Ravitch, Russell Ackoff, Peter Barnard, W. Edwards Deming, & Lisa Delpit, that there is an other and more humane way to look at schools.

When we try to isolate the effect of teachers on any of the outputs of the school, we are sure to fail.  Think about learning as a system.

Ed Johnson, a scholar and activist in Atlanta has taught me this.  When we try to break the system apart, it loses its essential properties. In this case the output as measured by student test scores is the product of the system, which is due to interactions and interdependencies that the teacher is only one small part.

To ignore the effects of the “system” on student achievement is ignore the large body of research on the effects of poverty on the emotional and social aspects of childhood, acute and chronic stressors, cognitive lags, and health and safety issues.

Just ask any teacher about his or her students.  Ask them how is the achievement of their students affected by inadequate school resources, living in poverty, not having a home, parents who struggle to earn a living, the size of the school and district, the location of the school, students coming to school each day hungry or inadequately fed, school policies, and so on?

Systems of Achievement in Race to the Top States

Take look at Figure 2.  I’ve selected seven winners of the Race to the Top competition, and plotted their math achievement level (at or above proficient) as measured by the National Assessment of Educational Progress (NAEP).   In addition to the seven winners (Florida, Georgia, Massachusetts, New York, North Carolina, Tennessee, District of Columbia) we also have included data for the United States.

The RT3 funding began in 2010, and is now in its fourth year for many of the winning states.  Notice, however, that five of states hover near the U.S. average, but  Massachusetts and the District of Columbia lie above and below the other states, respectively.  Why is this?


Now take a look at Figure 3. It’s the same graph but in this case its marked up.  The six states, and DC received from $75 to $700 million to improve education in their respective states.  In all cases, the single variable used to check effectiveness of the system is student achievement scores.  In  figure 3, we examine the results from a system’s point of view, a method that I learned from Ed Johnson.

Figure 2. 8th Grade Math as a System. All states, except for Massachusetts fall within the framework of Upper and Lower Control Limits.  Any variation within this zone is due to system causes, and not special causes.  Source: The Annie E. Casey Foundation, KIDS COUNT Data Center,
Figure 2. 8th Grade Math as a System. All states, except for Massachusetts fall within the framework of Upper and Lower Control Limits. Any variation within this zone is due to system causes, and not special causes. Source: The Annie E. Casey Foundation, KIDS COUNT Data Center,

In the graph below, most of the state scores fall within expected limits (Upper control limits–UCL and Lower control limits–LCL).  Any variation in scores for North Carolina, New York, Florida, Georgia, and Tennessee for the most part was random, but there is evidence that some special causes were at work in Massachusetts, and we might hypothesize that special cause  effects might be at work in DC..

Georgia, Florida, Tennessee, New York and North Carolina are U.S. examples of what Finnish educator Pasi Sahlberg calls the Global Education Reform Movement.  In each of these states, GERM has spread across these states, and we see classic GERM conditions, including the adoption of common standards, narrowing of curriculum focusing on math, writing and reading, high-stakes testing, a corporate management model which is data driven, and a system of accountability based on student test scores.

The graph below shows that the GERM model for most states is ineffective in changing math achievement.  I’ve examined reading in the same states during the same period, and the graphs are nearly identical.

The reforms that are in place in Georgia and other Race to the Top states will not affect student achievement in real ways.  The reforms are narrow and they ignore the ecology of learning by not seeing the school as part of a larger system.  For example, I asked in the last post why there was very little mention of poverty in Georgia’s reporting of their new method of grading schools.

Here is one reason.  Here is another graph of the same states, but this time showing poverty.  The graph is almost an inverse of the graphs shown in Figures 1 and 2. Notice that most states level of children living in poverty, except for Massachusetts (15%), has converged to the U.S. average which is about 23%.  What is the effect of poverty on student learning. Until we come look at the effects of the system on learning, we’ll make little progress in learning.

Using achievement scores is a poor indicator of student learning, and an even worse measure of teacher evaluation.

What do you think about the reforms that have been put into place as part of the Race to the Top?

Why High-Stakes Tests Should Not Be Used to Measure Student or School Performance

Latest Story

In my earlier post, I urged Atlanta’s new superintendent to explain to the Atlanta community why using high-stakes achievement measures, such as increasing achievement scores while raising the bar, should not be used to measure school performance.

Yet, for the next month, nearly 40 million students will take high-stakes achievement tests which will be used to check school, teacher, and student performance.  High-stakes tests (such as high-school exit exams) should not be used to test schools, teachers or students.  In fact, high-stakes tests are actually counterproductive.  Instead of improving student learning, high-stakes tests are actually linked to lower achievement scores, and increase drop out rates (Berliner and Glass, 2014, p. 219-221).

Why High-Stakes Testing?

For decades now, we have used high-stakes achievement tests as measures of success in school.

After the Soviet Union launched Sputnik, the world’s first satellite, the U.S. went into a state of frenzy and assumed its educational system was inferior to the USSR, and thus began a long chapter of  blaming schools for technological, scientific and economic problems.  The sky began to fall and all forms of reform began invading American classrooms.

But with the passage of the No Child Left Behind Act in 2001, the nation’s schools are strapped into a place that offers little flexibility to administrators or teachers.  NCLB mandated high-stakes testing for all students starting with six-year-old first-graders.  With an increasing battery of tests now in operation in schools, the curriculum has narrowed to such an extent that parents see schools as a place where kids go to prepare for  tests.  The state of Georgia spends about $25 million on testing each year, while the nation spends more than $1.7 billion.

Up, Up & Away

However, since 1971, the National Assessment of Educational Progress (NAEP) has tested American students in reading, mathematics, and science.  Rather than discovering that American education is failing, the results over the last 40 years show progress in reading, mathematics and science.  For example, Figure 1 shows average mathematics scores of 9-year-old students on the NAEP Long-Term Trend Assessments results by year on a scale in which the highest score is 500.   As you can see, students average scores have increased during this period; they have increased from 219 in 1973 to 240 in 2012.   Results for reading and science show the same trend scores increasing over time.

Screen Shot 2014-04-17 at 7.50.55 PM
Figure 1. NAEP Mathematics Scores from 1978  SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP)

Performance Levels are Up

NAEP also uses achievement or performance levels as another reporting technique.  The performance levels include advanced, proficient, basic and below basic.  The NAEP Governing Board established these levels as a way to communicate student performance.  Cut off scores are used to decide these levels.  For the subject area scales, performance levels were set at 50-point increments from 150 through 350.

Figure 2 shows performance  levels for 9-year-olds on the mathematics NAEP test.  If you focus on just the green, or Level 250, you will see that students scoring at the this level or above has increased from 3% in 1978 to 26% in 2012.  All results, except for 2008 are significant improvements from 2012.

However, the cut off levels used are opinions of experts.  Using 50 point increments is not based on science.  It is a qualitative measure, based on expert opinion. In fact if you read the descriptions of the levels (descriptions are specific for math, reading and science), you will see that they are qualitative descriptions or performances students are expected to be able to do.

The Global Education Reform Movement leaders, however, use these performance levels to harp on teachers, and make the claim that we aren’t doing the job.  All of our students are not performing at the advanced level.  As Diane Ravitch says in her book, The Reign of Error, “only in the dreams of policy makers and legislators is there a world where all students reach “proficiency” and score an A.”

Figure 2. NAEP Long Term Performance Levels, Mathematics, Age 9

If you look at the NAEP data (Figures 1 & 2), you will note that American teachers and their students have made progress in all the areas of testing (reading, mathematics and science).  Diane Ravitch summarizes it as follows:

NAEP data show beyond question that test scores in reading and math have improved for almost every group of students over the past two decades: slowly and steadily in the case of reading, dramatically in the case of mathematics. Students know more and can do more in these two basic skills subjects now than they could twenty or forty years ago.  Ravitch, Diane (2013-09-17). Reign of Error: The Hoax of the Privatization Movement and the Danger to America’s Public Schools (Kindle Locations 1155-1158). Knopf Doubleday Publishing Group. Kindle Edition.

NAEP Long Term Trend data has shown that American education is not broken, and that schools are making progress with all students.

What is the Purpose of Testing

In Georgia, the purpose of high-stakes tests, is to “measure” how well more info

students acquire the skill and knowledge described in the state mandated content standards in reading, English/language arts, mathematics, science and social studies.  According to the state these assessments yield information on academic achievement at the student, class, school, system and state levels.

But then the state claims that the information is used to diagnose individual student strengths and weaknesses as related to the state standards, and gauge the quality of education throughout Georgia.

For example, this week in DeKalb County School District, more than 40,000 students will be preparing for the Criterion Referenced Competency Test (CRCT) which will be given the week of April 21 – 25.  While talking to one parent from DeKalb today, she said that her three children, ages 8, 10 and 13 have been working with worksheets to prepare for next week’s CRCTs.  In Atlanta Public Schools, elementary and middle school students will take the CRCT April 16 – 24, while high school students will take their tests April 29 – May 12.

In Georgia, all students in grades one through eight take the CRCT in the content areas of reading, English/language arts, and mathematics.  Students in grades three through eight are also tested in science and social studies.  For high school students, End of Course Tests (EOCT) are given in mathematics (four courses), social studies (two courses), science (two courses), and English Language Arts (two courses).

So, for the most part all Georgia students will be involved in state testing from now until May 12, or for about four weeks.

So for four weeks, the state gives these tests to students, and according to the state, to diagnose our students.  This is an outright oversight or lie.  The results are not known until the summer.  Students have gone home.  Teachers are in graduate schools.  The results have little to nothing to do with helping student learn.  In fact, the evidence is that these high-stakes tests increase drop-out rates and decreased achievement (Berliner and Glass, 2014, p. 220-221).

But here is an even more outrageous problem with high-stakes tests. Just like the NAEP, states also use performance levels to decide how students did on these tests.  But the same problem with NAEP tests, is true for state tests, like Georgia’s CRCT and EOCT tests.  The performance levels are a pure opinion.  In fact, in most states, they push the performance levels higher each year, making it more difficult for students to reach, especially our struggling students.

There is little Reason to Use High-Stakes Tests

High-stakes testing have nothing to do with improving student achievement, or improving the quality of school learning.  Please note, that I am not saying that testing, or evaluation has nothing to do with learning.  Formative and diagnostic assessments have been shown to improve student learning, but summative, end-of-year, high-stakes assessments do not improve learning, and only assess a narrow range of student learning.

It’s a myth that American public education is failing, and the research done by the NAEP is clear and conclusive.

There is no reason to use high-stakes tests to make decisions about student, teacher and school performance.  Using a single test to claim that it determines student learning in a course or subject area makes little sense.  The skills that are measured on these tests are not necessarily the skills that are important for job or college success.

This week, in Marietta, Georgia, a couple has refused to allow their two children enrolled in West Side Elementary to take Georgia’s high-stakes tests, the CRCT.  After informing the school principal of their decision, they were met by the Marietta Police the next time they brought their children to school.  The police officer told them they would be trespassing and that their children were not allowed on school property.  According to a report in the Marietta Journal, the school district has not made a decision.

The school system superintendent thinks that parents need to be educated as to the reason these tests are given.  If the superintendent would listen to Mary Finney, the parent refusing to let her children take the tests, she would soon realize that this parent already knows why the CRCT’s are given, and simply doesn’t agree with the rationale.  As Ms. Finney said, there is no way that a single test will measure what her children know.

Let’s hope that the school district works with this family to resolve the issue, and does not use some form of punishment on the family and the children.

This family in Marietta, which is where I live has support from many parents, teachers and administrators.  In Florida comes this report:

Kelsey Lewis, joined with teachers, parents and school officials Thursday afternoon at a news conference to urge people to reach out to legislators and end high-stakes testing like the Florida Comprehensive Assessment Test.

“The testing mania has created a burden on our ability to teach our children with little to no educational value. It has achieved nothing academically, but successfully created a pressure-packed atmosphere on our children, their parents and our educational professionals,” said Mark Castellano, president of the Teachers Association of Lee County.


The  Finney family is courageous.  Will the Marietta superintendent and John Barge, Georgia’s Superintendent of Education act courageously and support these parents?  We’ll be watching.



Ed Johnson: Atlanta Needs to Reconsider Its Choice for New Superintendent


Photo credit:
Photo credit:

Ed Johnson, an advocate for quality education in Atlanta, provides commentary and data questioning Atlanta’s decision to hire Austin’s current superintendent for Atlanta’s superintendency. According to Mr. Johnson, there is great controversy in the process, as well as the choice for superintendent.

According an email I received from Ed Johnson, on April 7th, members of the clergy and community, including parents and educators, held a press conference to voice concerns about the undemocratic process that the Atlanta Board of Education is using to hire Atlanta Public Schools next superintendent.

According to the press release, the district paid thousands of dollars to consultants and engaged a search committee for several months to present several finalists to the public. But in a surprising move, the district presented a “sole” finalist after stating that over 400 names were submitted for consideration. The district is moving rapidly to hire the “sole” finalist without respecting a more democratic and open process that would engage and allow citizens more choices along with a more publicized and inclusive vetting process. Sadly, the board’s actions have taken the “public” out of public education.

Using his unique understanding of systems theory, he provides comparative data on these two school systems in the context of reading and mathematics performance over the past ten years.   This is an analysis you will not see performed by the Atlanta Public School Board, but whose members know him, and should take note of his thinking and caring for the students in the APS.

His analysis,  Austin Independent School District and Atlanta Public Schools Viewed through NAEP TUDA 2005-2013, can be viewed at this link, and augments his comments, which follow.

He writes:

Every two years NAEP, commonly known as “The Nation’s Report Card” and respected for being untainted by political ideologies and agenda, administers the Trial Urban District Assessment (TUDA) to voluntarily participating urban school districts.  TUDA Reading was first administrated in 2002 in Grades 4 and 8 to six urban school districts, including Atlanta Public Schools (APS or “Atlanta”).  TUDA Math was first administered in 2003 in Grades 4 and 8 to ten urban school districts, again including Atlanta. By 2013, TUDA had twenty-one urban school districts participating.  The next TUDA administration will be in 2015.  Austin Independent School District (AISD or “Austin”) participated for the first time in 2005.

TUDA results are reported as average scale scores that range from zero (0) to 500, as with NAEP results.  Looked at over time, TUDA results may be put to simple elementary level arithmetic to extract powerful, holistic insight into the performance of all the participating urban school districts taken as a system.  The same can be done to extract such insight into the performance of any one urban school district taken as a system of Reading, Math, or other subject as assessed by TUDA.  The attachment does both; just be aware that its use of the requisite arithmetic is atypical of simplistically ranking data and of computing percentage change from two data points in time as done in business-style financial reports.  Such reports inherently fail to preserve context and give no rational basis for predicting future outcome.

The USS Urban School Reform Ship

Pages 4 and 5 in the attachment look at all TUDA participating urban school districts as a system and represents, by way of analogy, that since the first TUDA administration in 2002, all the districts, save possibly a few, have been on the same ship carrying them all in the same direction no matter the disricts’ ever changing relative positions aboard ship.  Insight, here, is that virtually no urban school district has improved or declined that would amount to having “jumped ship” for the better or for the worse.  Since the ship first set sail in 2002, the outcome has been status quo maintained; no improvement of urban school districts all taken as a system.

Similarly, pages 6 through 9 in the attachment look at Austin and Atlanta TUDA Reading and Math results put side-by-side for 2005 through 2013 and represent that Atlanta stands relatively more capable to experience systemic improvement in both Reading and Math than does Austin, although Austin would “rank” higher than Atlanta.  Any ranking, however, would be only from the standpoint of Austin and Atlanta being passengers aboard the ship, thus pointless.  So stay mindful that the ship has all passengers going in the same direction.  It should matter least, or even not at all, that one urban school district may be more port-side and another may be more starboard; they all fall within natural limits at the widest point of the ship, at its beam.

Figure 1. It should matter least, or even not at all, that one urban school district may be more port-side and another may be more starboard; they all fall within natural limits at the widest point of the ship, at its beam.
Figure 1. It should matter least, or even not at all, that one urban school district may be more port-side and another may be more starboard; they all fall within natural limits at the widest point of the ship, at its beam.

Interestingly, during 2005-2013, Atlanta TUDA Reading and Math results continuously increased, but Austin TUDA Reading and Math results continually increased.  And during 2009-2013, Austin TUDA Reading and Math results began to appear suppressed and flattened, even stalled.  Consequently, Atlanta is now port-side and has become much closer to Austin.  (Caution: Three more continuous increases in any TUDA subject and grade results by Atlanta will be a total of eight continuous increases since 2005. Eight continuous increases should prompt conducting a study to learn the root cause(s) of the increases, lest another Atlanta test cheating crisis be in the making.  The reason is simple to illustrate: Getting eight heads in row or eight snake eyes in a row is possible but improbable with a fair coin or dice.)

By the way, if the ship were to be given a name then “The USS Urban School Reform” seems reasonable.

Kindly consider the attachment.  Consider, too, that Atlanta school board members seem unanimously committed to vote the affirmative come April 14, 2014, to hire the sole superintendent finalist they selected to present to the public, a behavior that bespeaks disregard for effective public engagement.  The sole superintendent finalist is Austin’s current superintendent, Dr. Meria Carstarphen.  Dr. Carstarphen has been Austin’s superintendent since 2009, since the time Austin’s TUDA results began to appear suppressed and flattened, even stalled.

Having considered the attachment, now kindly consider a few questions: Why does the ideology of “urban superintendent” persists when NAEP TUDA results make clear that the ideology’s transporter, the USS Urban School Reform, has provided and will in the future provide for its passengers, the urban school districts, to move around onboard ship but not improve?  Austin seems a case in point.  Will Atlanta become a case in point with Dr. Carstarphen as yet another “urban superintendent,” however well-trained by Harvard?

It will be my pleasure to have conversation about anything presented here that in any way interests you.

Ed Johnson
Advocate for Quality in Public Education

Atlanta GA
(404) 505-8176 |

In Spite of the “System,” Urban Teachers Have a Record of Success

In spite of the “System” the evidence is that urban teachers have a record of success, not one that is spiraling down.  The present state of reform of American education is based on the idea that American students are doing poorly, and this will lead to disastrous economic consequences, and the loss of American’s place in the global economic competition.

But, education (for our students) should not be a competition.  There is no need to have winners and losers as outcomes of the school experience.  Education is about learning, and in an environment that has as its core belief that learning is the fundamental goal of schooling.  Students are living in the present, and their school experience should be based on their lives now, and should not be based on furthering the economic prosperity of society.  Schooling should not be based on job training, career readiness or college entry.  It should be based on fostering the creative and innovative aspects of youth, and create school as a learning environment designed to help students learn to collaborate, work with others to solve problems, and engage in content from the arts and the sciences that has personal meaning.

We’ve been told that urban education in America needs to be saved by pouring advise and money from the élite and influential corporations and philanthropic groups.  The problem is that these groups are focused on only one set of outcomes that all come down to increasing student academic performance measured by high-stakes examinations.

I want to show here that urban teachers have held their own for the past decade and half in spite of the problems they face in their schools day-to-day.  They not only have held their own, but the evidence shows that academic performance of their students (in mathematics at the 8th grade) in the example below has slowly but surly increased as shown in Figure 1.  As you can see, in Atlanta, students at the 25th, 50th and 75th percentile increased performance on NAEP tests given from 2003 through 2013.

Figure 1. Atlanta NAEP 8th Grade Mathematics Scores for Selected Percentiles 2003 - 2013.
Figure 1. Atlanta NAEP 8th Grade Mathematics Scores for Selected Percentiles 2003 – 2013.

Reading and Math in Urban Schools

Take a look at the next four figures (Figures 2 – 5).  They were compiled by Mr. Ed Johnson in his study of the NAEP Trial Urban District Assessment (TUDA).  Johnson, who is a student W. Edwards Deming, examined the TUDA results through a Deming Lens.  A Deming Lens means that to understand the behavior of a system, one must look at the system.  Breaking down a system into its parts (goals, policies, finances, curriculum, teachers, administrators, parents, directors) loses one’s ability to understand the system.

Each of the graphs below shows the behavior of these four systems over ten years.  You will notice that there is variation in the achievement scores of students in reading (grades 4 and 8) and mathematics (grades 4 and 8) from one testing period to the next.  But the variation is within upper and lower limits that would be expected in each system.

Causes of Variation in Scores

According to W. Edwards Deming 94% of the variation is due to the nature of the system, not the people who work in or make the system work.  Only 6% are attributable to special causes.  (W. Edwards Deming. The New Economics for Industry, Government, Education (p. 33). Kindle Edition).

As you look over the graphs you will see ONLY FOUR instances where the variation in scores lies outside the Upper Control Limit (UCL), and then only in 4th grade reading  Examples include: Charlotte, 2009, Austin, 2011, Charlotte, 2011, and Hillsborough, 2011.  Except for these four instances, all the variation is due to the nature of the system.

Figure 2. Location of Cities in the Trial Urban District Assessment; Gold triangle-higher than large city; Circle--not different; Blue triangle-lower than large city. Source: Nations Report Card
Figure 2. Location of Cities in the Trial Urban District Assessment; Gold triangle-higher than large city; Circle–not different; Blue triangle-lower than large city. Source: Nations Report Card.  Click on Map for more details.

The graphs below plot reading and math scores for 21 school urban school districts.  Mr. Johnson highlighted Atlanta (in red) and DC Public Schools (purple).  As you can note in the following graphs, achievement scores in reading and math for Atlanta and DC Public Schools fell within the Upper and Lower Control limits.  There is no radical change in scores, either up or down.  It appears that the teachers in these urban schools are doing the job they were hired to do and that is help their students learn how to read, and do mathematics.  And they’ve done this in spite of all the issues that surround schools in urban communities.

In systems thinking, as Mr. Johnson would tell us, there are two types of causes of variation in any system.  The most important cause of variation in any system is what we call “common causes” of variation that is really a function of the system itself.  Examples of common cause variation will fall within control limits on a graph (as shown below in Figures 2 – 5). Examples of common causes that influence variation (scores on tests, for example, or graduation rates) include 

  • High percentage of children from low SES groups.
  • Where the school is located. It’s zip code.
  • Age of the school building.
  • Size of the school system.
  • Underpinning policies, practices, procedures of the school which determines it’s culture.
  • Inadequate resources.

According to Deming, nearly all outputs of schooling are the result of common cause variation, and these would include drop out rates, achievement test scores, violence, bullying, gang activity, low self esteem, attitudes, under performance and literacy skills.

Defying Gravity

When we examine a school system from a systems thinking view, these outputs are causes by the day to day effects of common causes of variation.  As Deming and other systems thinkers, such as Ed Johnson would say, trying to seek achievement scores beyond what see in the graphs (Figures 2 – 5) is to “defy gravity.”  Reformers have charged ahead as if they can “defy gravity” and have put the blame of not improving test scores in the wrong place.

Managers (administrators) and workers (teachers) are not “common cause” variables.  However, since schools are based on a linear factory model, “reformers” ignore common causes, and instead claim that teachers and administrators can overcome the challenges posed by common causes.  When reformers insist on market reforms, and they don’t work, they blame the teachers and principles.  And to make matters worse, they use student test scores (which are the result of common causes) to evaluate teachers on the basis of false assumptions about schooling.

We think that the present system of reading and mathematics is fairly stable.  The output in reading and math (as measured by a test score) vary little, and one can make predictions about future reading and math output.

reading 4th

Screen Shot 2014-01-11 at 8.27.50 PM

Screen Shot 2014-01-11 at 8.28.18 PM

Screen Shot 2014-01-11 at 8.28.31 PM

We will explore systems thinking in future posts.  But for now, what do think about the analysis of the NAEP TUDA data as compiled by Ed Johnson?