Why Candidates for Governor and State School Superintendent of Georgia Should Oppose High-Stakes Testing?

Latest Story

"Creative Commons Gravel Hill School 1950" by Erin Nekervis is licensed under CC BY-ND 2.0
“Creative Commons Gravel Hill School 1950” by Erin Nekervis is licensed under CC BY-ND 2.0

In an earlier post, I challenged candidates for state school superintendent to oppose the Common Core State Standards.  Today, I am writing to candidates for Governor and State School Superintendent of Georgia to oppose High-Stakes testing.  If they would, they’d open the door to a new paradigm of assessment that would improve education in Georgia beyond their wildest dreams.

Since 2001, the U.S. Department of Education has mandated the annual testing of children as young as 7 years old in mathematics and reading, and most states have added mandated high-stakes testing in writing, science, and social studies.

The American Education Research Association states that it is a violation of professional standards to make decisions about students’ life chances or educational opportunities on the basis of test scores alone.  Yet schools around the state of Georgia and indeed the rest of the country use end-of-the-year tests to make crucial decisions about whether students move on or not.  Additionally, these high-stakes tests have become an even greater burden on students because they know that the test results will be used to grade their teachers.

There is no easy answer to explain why we have an educational system that puts students in harm’s way by the continuous and unparalleled testing program.  When we read the newspapers soon after the release of international, national, or state tests, the emphasis is on who came in first, or who is at the top of the leader board.  No Child Left Behind and the Race to the Top have perpetuated an educational model based on competition and winning.

In some cases, officials will do what ever it takes to make sure they either win, or make the cut so that they place high on the leader board.  The Atlanta Journal-Constitution exposed the cheating scandal in Atlanta, and revealed that cheating (erasing wrong answers and changing to the correct answers on student test forms) was taking place in most cities and states.

Am I advocating the banning of high-stakes testing because it might lead to cheating.  No.

I am advocating banning high-stakes testing because it does not improve student learning, nor does it help teachers change their instruction to improve student learning.   Most of the those who advocate high-stakes testing believe that American education is failing, and that the fundamental goal of schools is improve achievement scores, and the only way to know if that has occurred is to use high-stakes standardized tests every year, and compare the scores from one year to the next.

But, if we do compare the test results from one year to the next, the results are quite astonishing.  First, we discover that in general, academic performance has gradually increased over time.  Secondly, we do see variation in average scores from one year to the next, but the variation is within expected statistical limits.

To give evidence that you might want to use with your constituents or potential voters, I am going to use a few graphs that were prepared by Mr. Ed Johnson, which were published on this blog earlier this year.  I am also going to use charts from the Anne E. Casey Foundation, and pull together data from various state and federal agencies.

NAEP Trial Urban District Assessments (TUDA)

The National Assessment of Educational Progress (NAEP) provides some of the most reliable data on student learning.  The tests given by NAEP are low-stakes, and an individual student takes only part of the test, so they don’t spend hours sitting for the exam.  NAEP has been studying American education since 1969.

About a decade ago, NAEP launched a study of urban school districts which they refer to as TUDA.  They provide telling results that I think will help you with the case of abolishing high-stakes tests.

The four graphs shown below in Figures 1 and 2 were prepared by Mr. Ed Johnson, an expert on W. Edwards Deming’s system of profound knowledge and how to transform organizations that result in continuous improvement.  He also is an expert in using facts to generate flow charts that help us understand how a system is working.

Figure 1 plots math scores for 21 cities over a ten-year period (for a list of the cities, follow this link).  Note that the scores fall within what are called upper control limits and lower control limits.  In no case do scores fall outside these predicted levels.  Yes, there is variation in the scores.  But they are within expected limits, and the variation is small.  For example, the green and red dots follow the Atlanta Public Schools (APS) and the Austin Independent School District (AISD).  If you show this graph to citizens in Georgia and ask if these graphs support the idea that our schools are failing, what is the answer? The answer is No.

Figure 1. Control Charts for NAEP TUDA Math Grades 4 & 8, 2003 - 2013
Figure 1. Control Charts for NAEP TUDA Math Grades 4 & 8, 2003 – 2013. Source Ed Johnson

Figure 2 plots reading scores for the same 21 cities over 11 years. Again, note that for the most part, the scores for each district fall within the expected limits, except for five points of measurement.  Each is labeled and as you see, only Charlotte and Hillsborough fall outside on the 4th grade reading TUDA results.  Think about this.  On only these five instances can we show significant variation from what we expect on the reading test.  The obvious thing to do, is to ask, what are these two districts doing, and how might what they are doing apply in other places.  It might be worth studying their system of education.

But, the real discovery here is to look at all math and reading TUDA results.  There are roughly 408 points of measurement shown in these four graphs, and in only five instances was the variation outside the range expected.   That is 0.012 percent. The systems of teaching math and reading in these 21 cities is predictable and consistent.

We can also see that there are no major swings in the test results.  When we send kids to school, we have a very good idea what to expect.  Another way to say this is that the system is performing as expected.

Or better yet, our teachers are doing it!

But there is always a need for improvement.  In Ed Johnson’s and W. Edwards Deming’s world of human systems, there is always the expectation for improvement.  The methods of improvement do not include the outright firing of department heads, or rank and file workers, any more than would we think that firing principals and teachers and bringing in uncertified and inexperienced teachers would help the situation.   But this is exactly what the Georgia Department of Education mandates when schools “fail” to meet the standards two years in a row.  Schools in this situation are labeled “turnaround schools.”

Here is what you need to know.  The high-stakes testing model is designed to make it very difficult for some schools, especially those schools where most of the students are eligible for free or reduced lunches (a statistic used to identify the poverty level of a school).  We know that students in less affluent schools will not do as well on these tests as students attending affluent schools.  It’s an unsustainable situation because these schools and their neighborhoods are punished by either closing the school or labeling it a turnaround.  But this is a sure ticket for financial rewards for charter management companies and teacher temp agencies including Teach for America and the New Teacher Project.  Follow this link to find out a better way to help these schools.

Labeling schools as failures is not sustainable.  It will not improve instruction. It represents an inaccurate interpretation of testing, and it is perverting a system that should be helping families, rather than punishing them.

Figure 2. NAEP TUDA, Reading, 4th & 8th Grade, 2002 - 2013.  Source: Ed Johnson
Figure 2. NAEP TUDA, Reading, 4th & 8th Grade, 2002 – 2013. Source: Ed Johnson

If We Were to Ban High-Stakes Tests?

Ok.  As a candidate for governor or state school superintendent in Georgia, and you were to go around the state campaigning for the banning of high-stakes tests, the odds are you would be elected.  You will be surprised who will support you, but you will need to tell the rest of the story.

Yes, you will support the idea of banning high-stakes tests.  But you need to clear that you are not suggesting that teachers and administrators all of a sudden stop assessing students.  Nothing could be further from the truth.

If you have been a teacher, you know that the assessment system  you use in your classroom has a major impact on student learning and classroom behavior.  Assessment is an integral aspect of teaching.  As teachers we assess students during every class session, and interaction that we have with them.  Teachers know that assessment, used as part of instruction, does indeed help student learning.  This is not an opinion.  One of the foremost researchers on assessment is Professor Paul Black, King’s College, London and he has found that formative assessment strategies do improve learning for students.  Formative, unlike the high-stakes tests that the government mandates, are embedded in instruction.  In my view, formative assessment is assessment for learning, not of learning.

Formative Assessment

Formative assessment are tools and methods that teachers use to humanize learning, and give students opportunities to apply their learning, and to engage in activities that involve communication, problem solving and team work–the kinds of skills and abilities that are important today, and will be tomorrow.

As a candidate for governor or state school superintendent you should listen to your most important constituency, and this is the professional teachers in public schools.  Last year, there were more than 111,000 teachers in Georgia teaching 1.6 million students.

So, what would happen if you said to nearly 1.6 million students (and their parents) and 111,000 teachers that the state would no longer need high-stakes tests.  Would the education system crumble?  Would students all of sudden not be motivated to learn?

It would thrive.  And it would free up a lot of money that would otherwise go to corporations.

According to a Brookings Institute report, the cost of testing in the U.S. exceeds $1.7 billion.  But according to the report, that is only for payments to testing vendors who also score the tests.  But what is the cost for lost instructional time.  In Georgia, the CRCT exams and high-school end-of-course exams take three-four weeks during the year.  So for about 4 weeks or about 12% of the school year, high-stakes exams dominate the school experience.

What’s the cost of 4 weeks of testing?  Well according to the Brookings Institute, about $600 billion is spent on education in the U.S. per year.  According to the Georgia Budget & Policy Institute, the 2015 fiscal year budget for K-12 education is $7.95 billion.  The cost of high-stakes testing is at least $950 million.

Every school in the state already has experts on assessment, and these educators need to be supported to collaborate with colleagues to develop assessment methods that will improve student learning, and increase student’s love of learning.

We know from many research studies that the best predictor of success in college & career (college & career is the favored purpose of reformers such as Bill Gates) are grades, not test scores.   Teachers are in the best place to assess their students.  Not only are they able to create their own tests, but there are multiple resources available that teachers already use to help their students learn.

Imagine if you were a high school biology teacher, and it was announced that the state would no longer need high-stakes tests.  How would this affect your teaching, and especially your relationship with your students.  One obvious difference is that the curriculum will expand because you no longer would be forced to teach to the test.  No longer would the students in your class be required to take tests that would be used to not only to decide whether they progress to the next science course, but the tests would no longer be used to decide if you keep your job.

In the next post, I’ll go into more detail about what assessment would look like in this alternative paradigm.

In the meantime, are you willing to discuss the possibility of returning the education of students into the hands of professional teachers?

 

Are Georgia School Superintendent Candidates Willing to Oppose the Common Core & High-Stakes Tests?

Creative Commons School of Neon Fusiller School" by Malcoml Browne is licensed  under CC BY-ND 2.0.
Creative Commons School of Neon Fusiller” by Malcolm Browne is licensed under CC BY-ND 2.0.

Dear Candidates for Georgia School Superintendent,

Today, I want to challenge you to not only oppose Georgia’s adoption of the Common Core State Standards (CCSS), but also the use of high-stakes tests such as the CRCT. In this post, I’ll offer some facts you can use to discuss why to oppose the CCSS.  In the next post, we’ll give reasons why high-stakes tests need to abolished.

For nearly two decades, Standards and high-stakes testing have dominated teaching and learning in every Georgia public school.  I’ve shown in earlier posts, that standards are barriers to student learning, and if teachers are not given autonomy over the use of standards, then they tend to impede innovation, creativity, and teaching that focuses on the needs of children and youth.  Communication skills, problem solving, team work, and innovation are the kinds of experiences that are important to students now, and will be in the future.  The standards in the context of high-stakes tests impedes these goals.

Common Core State Standards

It’s time, however, to break these connections, at least for standards and high-stakes, and look for different ways to help students learn.

Making a one-size fits all curriculum for every student in Georgia makes little sense. We know that the “real” curriculum for our students is what happens in their classrooms with their peers and teachers. The curriculum should not be determined by non-educators from a highly financed organization (as was the Common Core State Standards), but should be an effort carried out by teachers and educators–Georgia has a top-notch teaching force, and some of the countries major universities.

Figure 1 shows the mathematics achievement level of Georgia students compared to students across the nation using National Assessment of Education Progress (NAEP) tests. Note that we have an upward trend in mathematics achievement in Georgia as well as the nation.   Figure 2 is a table showing the percentage of 8th grade students meeting or exceeding state standards on CRCT tests in math.  From the data presented in Figures 1 and 2, the trend in 8th grade mathematics achievement, as measured by the NAEP tests, and the state of Georgia CRCT, is positive, showing steady improvement.  If we look at results in math at other grade levels, as well as reading and science scores, the trends are similar.

Figure 1. NAEP 8th Grade Math Achievement At or Above Basic for Georgia compared to the United States 2000 - 2013. Source: Kids Count data center, Annie E. Casey Foundation.
Figure 1. NAEP 8th Grade Math Achievement At or Above Basic for Georgia compared to the United States 2000 – 2013. Source: Kids Count data center, Annie E. Casey Foundation.
Figure 2. Georgia 8th Grade Students Meeting or Exceeding State Standards on CRCT Tests in Math, 2006 - 2012
Figure 2. Georgia 8th Grade Students Meeting or Exceeding State Standards on CRCT Tests in Math, 2006 – 2012.  Source: Kids Count Data Center, Annie E. Casey Foundation

Georgia’s state standards did not result in a tailspin of student achievement.  The Common Core State Standards, which were implemented a year ago, have not resulted in any major variation in testing.  In fact, the variation in test scores that we see not only in Georgia, but nearly all states (Massachusetts is an exception) is within the limits of what we would expect.  Figure 3 compares the average scores in 8th grade math achievement based on NAEP tests between seven states, including Georgia.

Using the control chart approach of W. Edwards Deming and Donald Wheeler and David Chambers (which I learned from Ed Johnson), we see in Figure 3 that for over a decade the achievement scores in most of these states and District of Columbia fall within expected limits.  In fact for most of these Any variation for these states, except for Massachusetts, is NOT due to any special cause (new curriculum, new standards, using high-stakes tests), but are simply what we expect in a system that is operating as it should.  In general we can conclude that education in these states is not a failure, but schools are doing what we expect them to do.  The continuous improvement that we see in the scores is not due to any innovation or special cause, but is simply the result of the way the education system works.  And it doesn’t matter whether we look at scores from suburban communities, and compare them to urban environments.

As a candidate, you will hear the oft mentioned phrase, that “America’s schools are failing and they need to be reformed.”  In fact, this phrase has been repeated so often, that in a recent survey over 70% of parents agree that schools were failing.  But over 80% said that the school their children attended was doing very well.

You need to use facts to show that our schools are not failing, and help your potential constituents realize that they’ve been sold down the river that America’s schools are failing.  They are not.

 

Figure 2. 8th Grade Math as a System. All states, except for Massachusetts fall within the framework of Upper and Lower Control Limits.  Any variation within this zone is due to system causes, and not special causes.  Source: The Annie E. Casey Foundation, KIDS COUNT Data Center, datacenter.kidscount.org
Figure 3. 8th Grade Math as a System. All states, except for Massachusetts fall within the framework of Upper and Lower Control Limits. Any variation within this zone is due to system causes, and not special causes. Source: The Annie E. Casey Foundation, KIDS COUNT Data Center, datacenter.kidscount.org

Schools, They Aren’t Failing

You might ask, what about urban schools.  Are our urban schools failing?  The fact is, there are lots of people who will tell you that schools in urban environments are failing, and what they need is help from charter management companies, and temp teacher preparation organizations such as Teach for America and the New Teacher Project.

They need none of this.

Take a look at Figure 4.  It’s an analysis done by Ed Johnson using data over the past decade comparing 21 city school districts, with the Atlanta Public Schools (APS) and the Austin Independent School District (AISD) highlighted in green and red, respectively.   As you can see in the graph, there is variation in math NAEP test scores over the ten-year period.  The variation we see is consistent.  There are not wide swings in the data.  Indeed, all the points of measurement fall within statistical control limits.

 

Figure 2.  Systemic Story of NAEP 8th Grade Math Scores on the TUDA Assessment for 21 Districts, including Atlanta and Austin.  This graph shows 8th grade math as a system.
Figure 4. Systemic Story of NAEP 8th Grade Math Scores on the TUDA Assessment for 21 Districts, including Atlanta and Austin. This graph shows 8th grade math as a system.

 

If there is any truth to this kind of data, there is no need to radically change the system.  But, as Deming and others have found, there is always the need for continuous improvement of the system.  If we want to improve schooling, say in Atlanta, we need to improve the system, not “turn it around.”

But the mantra you will hear is that we need to close schools, or turn the school around by firing the school principal and most of the staff, and then replace them with a new principal, and new teachers who are inexperienced, uncertified, and will only stay there for 2 years.  If you are elected State School Superintendent, you will find that there are questionable relationships among the Georgia Department of Education, Teach for America, the New Teacher Project, and charter management companies.

Don’t fall into the “our schools are failing” trap.

Improving Schools

As a candidate for the top job in education, you surely want to figure ways to help Georgia school districts improve their schools.  Adopting the Common Core State Standards will not improve our schools.

We want, what Ed Johnson explains, continuation improvement.

How do we do this?  First, we need to act on the idea that education is a human system.  It’s about people.  It’s about parents sending their students to schools enjoy learning, and not to be there to serve the state by simply being a number, and someone who is required to take tests throughout their school days.  If you ask parents what they like about their children’s school, they always talk about how their children are treated and accepted, and helped to learn.  They talk about the kind of communication among their children’s peers and teachers.

Improving schools means we need to think differently and bring to the front what we know about successful organizations.  In a recent post, I discussed some steps that we should take that have a greater likelihood of establishing an environment that will result in continuous improvement.

  1. Put high confidence in teachers and principals and learning.  The focus on meaningful learning must be at the school level.  Superintendents need to get out-of-the-way, stop micro-managing, and entrust education to well prepared teaching staff.
  2. Create a systemic environment which encourages teachers and students to try new ideas and approaches.  Encourage principals to work with teachers to push for curiosity, imagination and creativity in the classroom, and make that the focus of learning.
  3. Fill classrooms with well experienced and well-educated teachers who are not only knowledgeable in the content, but more importantly understand how to teach and how to experiment with different pedagogies.
  4. Empower principals to be the leaders of change, not superintendents.  Superintendents are too far away from the day-to-day life of students to encourage the kind of creative teaching that can be supported by principals.
  5. Teachers should have masters degrees in education and be knowledgeable in their field of teaching.  Reliance on uncertified and inexperienced teachers will in the long run lead to failure.

In the next post, I will provide evidence to support the second thing that I would like you to oppose, and that is the use of high-stakes tests.  High-stakes tests are the biggest impediment to real improvement of schooling for students.  I hope you’ll check out the next post.

NAEP Math Scores Insignificantly Affected by the Common Standards

image

The Common Core State Standards (Common Standards) have been implemented for about four years. According to the developers (the folks over at Achieve) and it’s billionaire financiers, such as Bill Gates, the Common Standards are benchmarked against high performing international standards, and should result in higher achievement scores for American students.

According to Achieve, the Common Standards are guidelines for what every student should know and be able to do in math and English language arts from kindergarten through 12th grade.

According to proponents, the Common Standards should result in higher achievement scores and an increase in the nation’s ability to compete globally. Proponent-in-chief, billionaire Bill Gates, said in an interview with ABC News, “that in a decade, the scores and competence for U.S. students in math can be improved. This is going to be a big win for education.”

A recent report based on peer-reviewed studies in the Brookings 2014 Brown Center Report on American Education sheds light on these claims. The results will be a disappointment for some.

The 2014 Brookings report includes sections on the PISA international – Shanghai controversy, homework, and the Common Core. I’m focusing only on the Common Core section in this article.

CCSS Shows No Signs of Boosting Math Scores

There has been a lot of talk that the U.S. needs new standards to ratchet up our student’s abilities in mathematics (reading & science, too). The Brown Report provides empirical support to find out how the Common Standards are affecting American student’s 8th grade math scores.

According to an earlier Brown study (2012), it was predicted that the new Common Standards would not affect student learning. The 2014 study confirms this. Here is the concluding paragraph in the 2014 Brown Report.

The 2012 Brown Center Report predicted, based on an empirical analysis of the effects of state standards, that the CCSS will have little to no impact on student achievement.  Supporters of the Common Core argue that strong, effective implementation of the standards will sweep away such skepticism by producing lasting, significant gains in student learning. So far, at least— and it is admittedly the early innings of a long ballgame—there are no signs of such an impressive accomplishment.

Math Scores Lagged During the CCSS Rollout

The NAEP collects achievement scores every two years for 9, 13, and 17-year-old students. The graph in Figure 1 is the trend of 8th grade math achievement from 1990 – 2013. The average score in 1990 was , while average math score in 2013 is 285.

The trend line does not show a significant change in the slope of line during the time the Common Standards were implemented (the area on the line encircled in red.  One would have expected a bump during this period.  It isn’t there.

It turns out that a little math will tell the story. Between 1990 – 2013 there was a 22 point increase in 8th grade math. Over the 23 years this amounts to about a 1 point increase per year. However, the average score increase from 2009 – 2013, the years the Common Core has been used, has only increased 0.30 points per year, much less than before the roll out of the Common Core.

Figure 1. NAEP 8th Grade Math Scores, 1990-2013. Source. U.S. Department of Education

 

Digging Deeper

The Brown Report on the Common Core begins with an examination of research at Michigan State University by William H. Schmidt and Richard T. Houang.  These researchers published a study in 2012 that included an analysis of the math standards in place in all 50 states in the 2008 – 2009 about how they stacked up to the Common Core State Standards in Mathematics.  The question was how well do the state standards align with the CCSSM in terms of congruence, focus and rigor.  (Schmidt, W. H. & Houang, R.T. Curricular Coherence and the Common Core State Standards for Mathematics Educational Researcher November 2012 41: 294-308). 

Table 1 shows the results of the analysis of state math standards and their consistency with the CCSSM in 2008.  The table shows five categories based on the degree of concurrence with the Common Standards in math.  Those rated high (5) include states such Alabama, Georgia, Michigan.  Those rated low (1) include Arizona, Kansas, New Jersey.   Schmidt and Houang did an analysis to find if there was a relationship between achievement scores on the NAEP math test and the degree of congruence of the state standards with the CCSS in math.  There was no relationship to achievement.  However, their analyses did show two groups of states (Group A and Group B) that did show significant coefficients for congruence.

Table 1. Existing State Standards' Consistency with the Common Core Math Standards. Loveless, T. How Well Are American Students Learning. The Brown Center Report on American Education, March 2014, Volume 3, Number 3, p. 28.
Table 1. Existing State Standards’ Consistency with the Common Core Math Standards. Source: Schmidt, W.H. (2012), Presentation at the National Press Club, Extracted from Loveless, T. How Well Are American Students Learning. The Brown Center Report on American Education, March 2014, Volume 3, Number 3, p. 28.

Brown Analysis

According to the Michigan State study, states numberswiki.com

adopting the Common Core in math should expect to see an increase on NAEP 8th grade math scores. The Brown research extended the Michigan study by looking specifically at gain scores at the state level since 2009. By using NAEP math scores for 2011 and 2013, the Brown researchers had a way to test the predictive ability of the earlier Michigan study.

We should realize that this is only four years into the adoption of the Common Core. However, the findings ought to provide some feedback on the current status of the Common Core, and help us predict future effects of the standards on math achievement,

Table 2 presents state NAEP changes organized by the Michigan rating system of congruence with Common Core math standards. Recall, the a rating of “5” meant that those states were most like the CCSS math standards, those with a “1” were most divergent from the Common Core math standards. Accordingly, we would expect that there would be a trend in the gain scores favoring the higher rated states vs the lower rated states.

Table 2 shows there is no trend or systematic relationship between the Michigan ratings and changes in NAEP math scores. Note that states that had very divergent standards from the Common Core actually gained more than states that were rated most like the Common Standards.  As Loveless says, the data are lumpy.  When one expects high scores, the results are low scores, and visa versa.  There is no pattern in these findings.

Table 5. State NAEP Changes, by Michigan Ratings of Congruence with CCSS, (in scale score points, 2009 - 2013).
Table 2. State NAEP Changes, by Michigan Ratings of Congruence with CCSS, (in scale score points, 2009 – 2013).

How did the Common Core Math Implementation Fare?

In 2009, the U.S. Department of Education survey the states to find out how the Common Standards reform was doing and being implemented.  The survey asked states if they had: (1) adopted the CCSS; (2) provided, guided, or funded professional development on the CCSS; (3) provided curriculum/instructional materials for the CCSS; and (4) worked with a consortium to develop assessments aligned with the CCSS (Brown Study, p. 31).

In the 2014 Brown study, they researchers used the survey results to categorized the state’s implementation rating as strong, medium, or non-adopters, as shown in Table 3.   Note that the strong implementors of the Common Core made the largest gains, while non-adopters showed the smallest gains.

Table 3. Changes in NAEP Scores (in scale score points) by Implementation of CCSS. Extracted from Loveless, T. How Well Are American Students Learning. The Brown Center Report on American Education, March 2014, Volume 3, Number 3, p.32.

 
However, if we compare the gains shown during the four years of CCSS adoption of math standards to the entire history of NAEP scores, we find that during the transition period, scores have only gained 0.33 points, where the gain score per year over the entire history of NAEP math testing is more than “1” point per year.

Surprised?

Some will use these results to claim the Common Core is a failure. Others will say that more time is needed to “test” the new standards. Others might claim that the standards do not have much affect on how students do on NAEP tests.

In an earlier post, I reported that Brown (2012) researchers concluded that we should not expect much from the Common Core.  In an interesting discussion of the implications of their findings, Tom Loveless, the author of the report, cautions us to be careful about not being drawn into thinking that standards represent a kind of system of “weights and measures.”  Loveless tells us that standards’ reformers use the word—benchmarks—as a synonym for standards.  And he says that they are used too often.  In science education, for example, we’ve had a long history of using the word benchmarks, and Loveless reminds us that there are not real, or measured benchmarks in any content area.  Yet, when you read the standards—common core or science—there is the implication we really know–almost in a measured way–what standards should be met at a particular grade level.

Loveless also makes a strong point when he says the entire system of education is “teeming with variation.”  To think that creating a set of common core standards will reduce this variation between states or within a state simply will not succeed. As he puts it, the common core (a kind of intended curriculum) sits on top of the implemented and achieved curriculum.  The implemented curriculum is what teachers do with their students day-to-day.  It is full of variation within a school.  Two biology teachers in the same school will get very different results for a lot of different factors.  But as far as the state is concerned, the achieved curriculum is all that matters.  The state uses high-stakes tests to determine whether schools met Adequate Yearly Progress (AYP).

As the Brown report suggests, we should not depend on the common core or the Next Generation Science Standards having any effect on students’ achievement.  The report ends with this statement:

The nation will have to look elsewhere for ways to improve its schools.

What do think of the research reported here about the Common Core?

In Spite of the “System,” Urban Teachers Have a Record of Success

In spite of the “System” the evidence is that urban teachers have a record of success, not one that is spiraling down.  The present state of reform of American education is based on the idea that American students are doing poorly, and this will lead to disastrous economic consequences, and the loss of American’s place in the global economic competition.

But, education (for our students) should not be a competition.  There is no need to have winners and losers as outcomes of the school experience.  Education is about learning, and in an environment that has as its core belief that learning is the fundamental goal of schooling.  Students are living in the present, and their school experience should be based on their lives now, and should not be based on furthering the economic prosperity of society.  Schooling should not be based on job training, career readiness or college entry.  It should be based on fostering the creative and innovative aspects of youth, and create school as a learning environment designed to help students learn to collaborate, work with others to solve problems, and engage in content from the arts and the sciences that has personal meaning.

We’ve been told that urban education in America needs to be saved by pouring advise and money from the élite and influential corporations and philanthropic groups.  The problem is that these groups are focused on only one set of outcomes that all come down to increasing student academic performance measured by high-stakes examinations.

I want to show here that urban teachers have held their own for the past decade and half in spite of the problems they face in their schools day-to-day.  They not only have held their own, but the evidence shows that academic performance of their students (in mathematics at the 8th grade) in the example below has slowly but surly increased as shown in Figure 1.  As you can see, in Atlanta, students at the 25th, 50th and 75th percentile increased performance on NAEP tests given from 2003 through 2013.

Figure 1. Atlanta NAEP 8th Grade Mathematics Scores for Selected Percentiles 2003 - 2013.
Figure 1. Atlanta NAEP 8th Grade Mathematics Scores for Selected Percentiles 2003 – 2013.

Reading and Math in Urban Schools

Take a look at the next four figures (Figures 2 – 5).  They were compiled by Mr. Ed Johnson in his study of the NAEP Trial Urban District Assessment (TUDA).  Johnson, who is a student W. Edwards Deming, examined the TUDA results through a Deming Lens.  A Deming Lens means that to understand the behavior of a system, one must look at the system.  Breaking down a system into its parts (goals, policies, finances, curriculum, teachers, administrators, parents, directors) loses one’s ability to understand the system.

Each of the graphs below shows the behavior of these four systems over ten years.  You will notice that there is variation in the achievement scores of students in reading (grades 4 and 8) and mathematics (grades 4 and 8) from one testing period to the next.  But the variation is within upper and lower limits that would be expected in each system.

Causes of Variation in Scores

According to W. Edwards Deming 94% of the variation is due to the nature of the system, not the people who work in or make the system work.  Only 6% are attributable to special causes.  (W. Edwards Deming. The New Economics for Industry, Government, Education (p. 33). Kindle Edition).

As you look over the graphs you will see ONLY FOUR instances where the variation in scores lies outside the Upper Control Limit (UCL), and then only in 4th grade reading  Examples include: Charlotte, 2009, Austin, 2011, Charlotte, 2011, and Hillsborough, 2011.  Except for these four instances, all the variation is due to the nature of the system.

Figure 2. Location of Cities in the Trial Urban District Assessment; Gold triangle-higher than large city; Circle--not different; Blue triangle-lower than large city. Source: Nations Report Card
Figure 2. Location of Cities in the Trial Urban District Assessment; Gold triangle-higher than large city; Circle–not different; Blue triangle-lower than large city. Source: Nations Report Card.  Click on Map for more details.

The graphs below plot reading and math scores for 21 school urban school districts.  Mr. Johnson highlighted Atlanta (in red) and DC Public Schools (purple).  As you can note in the following graphs, achievement scores in reading and math for Atlanta and DC Public Schools fell within the Upper and Lower Control limits.  There is no radical change in scores, either up or down.  It appears that the teachers in these urban schools are doing the job they were hired to do and that is help their students learn how to read, and do mathematics.  And they’ve done this in spite of all the issues that surround schools in urban communities.

In systems thinking, as Mr. Johnson would tell us, there are two types of causes of variation in any system.  The most important cause of variation in any system is what we call “common causes” of variation that is really a function of the system itself.  Examples of common cause variation will fall within control limits on a graph (as shown below in Figures 2 – 5). Examples of common causes that influence variation (scores on tests, for example, or graduation rates) include 

  • High percentage of children from low SES groups.
  • Where the school is located. It’s zip code.
  • Age of the school building.
  • Size of the school system.
  • Underpinning policies, practices, procedures of the school which determines it’s culture.
  • Inadequate resources.

According to Deming, nearly all outputs of schooling are the result of common cause variation, and these would include drop out rates, achievement test scores, violence, bullying, gang activity, low self esteem, attitudes, under performance and literacy skills.

Defying Gravity

When we examine a school system from a systems thinking view, these outputs are causes by the day to day effects of common causes of variation.  As Deming and other systems thinkers, such as Ed Johnson would say, trying to seek achievement scores beyond what see in the graphs (Figures 2 – 5) is to “defy gravity.”  Reformers have charged ahead as if they can “defy gravity” and have put the blame of not improving test scores in the wrong place.

Managers (administrators) and workers (teachers) are not “common cause” variables.  However, since schools are based on a linear factory model, “reformers” ignore common causes, and instead claim that teachers and administrators can overcome the challenges posed by common causes.  When reformers insist on market reforms, and they don’t work, they blame the teachers and principles.  And to make matters worse, they use student test scores (which are the result of common causes) to evaluate teachers on the basis of false assumptions about schooling.

We think that the present system of reading and mathematics is fairly stable.  The output in reading and math (as measured by a test score) vary little, and one can make predictions about future reading and math output.

reading 4th

Screen Shot 2014-01-11 at 8.27.50 PM

Screen Shot 2014-01-11 at 8.28.18 PM

Screen Shot 2014-01-11 at 8.28.31 PM

We will explore systems thinking in future posts.  But for now, what do think about the analysis of the NAEP TUDA data as compiled by Ed Johnson?

Is the Purpose of Education Economic Development? The State of Georgia Says Yes.

Which of the following is the most important purpose of education in Georgia?

  1. to prepare students to become responsible citizens
  2. to enhance personal happiness and enrich lives
  3. to support the economic development of the state
  4. to get a job
  5. to learn how to learn

According to Governor Deal’s website, the answer is either 3 or 4, since it is clearly stated on the Governor’s website that education is economic development (Figure 3).  Or another way to put this is that students are in school because they will be future workers who will develop the economy.  The purpose is to get a job.

There are five points outlined on the Governor’s website, and “education is economic development” is number 1.  The purpose of schooling, according to the Governor’s website is based on the rationale that Georgia students (as well as students in the rest of the nation) have “lost ground” to their peers around the world, and to get our students up to speed we’ll go out-of-the-way to push student’s academic achievement higher and higher so that the U.S. ranks at the top (aka Race to the Top).

Not are graduates of American quite able to compete globally, they have built the world’s largest economy.  How can the state make the claim that students are in the middle of the pack.  Oh, that’s very easy.  All they do is look at the results on one of the international achievement tests such as PISA or TIMSS, and they claim that U.S. students fall somewhere in the middle of average scores of sixty or more nations.  In math and science, American students are not falling behind, and are not in the middle of the pack.

American students are not in the middle of the pack.  In fact they are quite high in the league standings chart, and indeed, if Massachusetts was a country, it would be number one on the PISA international tests in math and science.

Furthermore, over the past 20 years, NAEP long-term test results show that American students (elementary, middle and high school, white, African-American, Hispanic and Asian-American) scores increasing in mathematics, reading and science.  Take a look at mathematics score from 1973 – 2008.  Where is the failure here?  Scores seem to be going up, and if you go the NAEP Long Term Trend site (not available right now because of the Government shutdown), you will find scores for African-American and Hispanic students following this pattern.

Figure 1. NAEP Mathematics Scores of American students, 1973 - 2008.
Figure 1. NAEP Mathematics Scores of American students, 1973 – 2008.

Here are two more graphs that the Governor of Georgia and his staff should study and use to rewrite their opinions of education in Georgia.

Figure 2.  Long term trend reading scores of American elementary students.
Figure 2. Long term trend reading scores of American elementary students.

Yet, even with these FACTS, the state of Georgia keeps repeating the mantra that students and schools are failing and that the solution is to hire uncertified teachers from Teach from America (my critique of TFA), turn around failing schools by firing the staff and then converting it to charter school run by management companies that probably have its corporate office in Florida.

Governor of Georgia’s Statement on Education Issues

Figures 3 and  show copies of the Governor’s statement on education.  Figure 3 is unmarked.  I’ve used red and blue arrows and comments to highlight some of the problems with the State of Georgia’s approach to education as shown in Figure 4.

Screen Shot 2013-10-03 at 9.45.51 PM
Figure 3. Statement on the Issues related to Education on the Governor of Georgia’s Website.

 

Screen Shot 2013-10-04 at 8.45.04 PM
Figure 4. Marked Up copy of the Governor of Georgia statement on education.

This is what happens when politics and corporate influence and power misinterpret the purpose of schooling in a democratic society.