Why Achievement Test Scores are Poor Indicators of Student Learning and Teacher Effectiveness

The U.S. Department of Education (ED) has established a single variable as the way to reward and punish schools, teachers, students and their parents.  The fact that I have used the terms “rewards” and punishments” is evidence enough that the ED is stuck in 19th century psychology.

In 2001, the Congress approved the No Child Left Behind Act which mandated the testing of all students in reading and math.  Immediately, this set in motion the most devastating impact on curriculum in the elementary schools by narrowing the curriculum, and putting such emphasis on reading and math.

In 2009, the Congress approved the Race to the Top Fund (RT3), which earmarked about $4.5 billion for a U.S. competition among the 50 states and the District of Columbia.  Of these entities, only 18 were winners.  The rest lost, except for four states which choose not to compete).

The Race to the Top, in my view, is even worse for education than the NCLB.  In the RT3, achievement test scores are given even more importance because those states that got the money were required to tie student test scores to teacher evaluation using the Value Added Modeling (VAM) system.

Many states, even those that did not receive RT3 money now require at least 50% of a teacher’s evaluation be based on the VAM scores generated by a mythical statistical model.  If you think I am kidding, here is the formula for determining a teachers worth as measured by adding value to student learning.

 Figure 2. The statistic value-added model (covariate adjustment model) used to evaluate Florida teachers.

Figure 1. The statistic value-added model (covariate adjustment model) used to evaluate Florida teachers.

Aside from the fact that VAM scores are unreliable, often the scores of very competent teachers end up being at the bottom of the list.  Further, the tests upon which the VAM is calculated measure only a very small aspect of student learning.  In fact, much of what we think is really important in school–communication skills, ability for work collaboratively with others to solve problems, creative thinking, empathy, and ethics–are not measured on achievement tests.

Why does the ED insist on this simple and behavioristic model of teaching?  It does so because it thinks that school is like a factory, and runs much like a machine.  Some call this mechanistic thinking.  Everything can be broken down into components, such as teacher behavior, teacher training, computers in the classroom, number of students in the class, access to technology, standards, academic tests, courses, homework, etc.   Mechanistic thinking leads to a “fix it” mentality.  That is, we can fix the problem of schooling by changing one or more of these variables.

The big problem in the minds of the mechanistic thinkers, who I am also going call the Neo-School Reformers, such as Bill Gates, Michelle Rhee, Joe Klein, and Arne Duncan, is that they believe that American schools are inferior to schools in other nations, especially countries including Finland, and most of the Asian nations.  Our schools are inferior, and they prove it by citing test scores on PISA and other international tests.  But they don’t tell you the rest of the story.

The Neo-School Reformers solution to what ails our schools is the Global Education Reform Model (GERM).  Although not named by Gates and associates, it was described by one of Finland’s leading educators, Dr. Pasi Sahlberg.

There is a growing body of research that shows that the GERM model is an ineffective model of educational reform.  As Sahlberg points out, GERM is primarily practiced by the North Atlantic Alliance of Schools (primarily the U.S. Europe, and Australia).

Indeed, if you compare the PISA test results of these nations, its difficult to distinguish one from the other.

Thinking In Terms of Systems Theory

The Neo-education reforms are “heads in the sand” reformers.  They fail to look around.  They can’t.  Their necks are stuck in the muck of their own arrogance, and ignorance.  They fail to take their heads out of the box of a classroom or a school, and think about the larger ecosystem in which the school is placed.  They really get mad at teachers or education researchers if they bring up out-of-school factors that might affect student achievement.  They have a code or a motto: No Excuses Education (NEE).

Here is the thing. I’ve learned from a group of scholars, including Ed Johnson, Diane Ravitch, Russell Ackoff, Peter Barnard, W. Edwards Deming, & Lisa Delpit, that there is an other and more humane way to look at schools.

When we try to isolate the effect of teachers on any of the outputs of the school, we are sure to fail.  Think about learning as a system.

Ed Johnson, a scholar and activist in Atlanta has taught me this.  When we try to break the system apart, it loses its essential properties. In this case the output as measured by student test scores is the product of the system, which is due to interactions and interdependencies that the teacher is only one small part.

To ignore the effects of the “system” on student achievement is ignore the large body of research on the effects of poverty on the emotional and social aspects of childhood, acute and chronic stressors, cognitive lags, and health and safety issues.

Just ask any teacher about his or her students.  Ask them how is the achievement of their students affected by inadequate school resources, living in poverty, not having a home, parents who struggle to earn a living, the size of the school and district, the location of the school, students coming to school each day hungry or inadequately fed, school policies, and so on?

Systems of Achievement in Race to the Top States

Take look at Figure 2.  I’ve selected seven winners of the Race to the Top competition, and plotted their math achievement level (at or above proficient) as measured by the National Assessment of Educational Progress (NAEP).   In addition to the seven winners (Florida, Georgia, Massachusetts, New York, North Carolina, Tennessee, District of Columbia) we also have included data for the United States.

The RT3 funding began in 2010, and is now in its fourth year for many of the winning states.  Notice, however, that five of states hover near the U.S. average, but  Massachusetts and the District of Columbia lie above and below the other states, respectively.  Why is this?

 

Now take a look at Figure 3. It’s the same graph but in this case its marked up.  The six states, and DC received from $75 to $700 million to improve education in their respective states.  In all cases, the single variable used to check effectiveness of the system is student achievement scores.  In  figure 3, we examine the results from a system’s point of view, a method that I learned from Ed Johnson.

Figure 2. 8th Grade Math as a System. All states, except for Massachusetts fall within the framework of Upper and Lower Control Limits.  Any variation within this zone is due to system causes, and not special causes.  Source: The Annie E. Casey Foundation, KIDS COUNT Data Center, datacenter.kidscount.org
Figure 2. 8th Grade Math as a System. All states, except for Massachusetts fall within the framework of Upper and Lower Control Limits. Any variation within this zone is due to system causes, and not special causes. Source: The Annie E. Casey Foundation, KIDS COUNT Data Center, datacenter.kidscount.org

In the graph below, most of the state scores fall within expected limits (Upper control limits–UCL and Lower control limits–LCL).  Any variation in scores for North Carolina, New York, Florida, Georgia, and Tennessee for the most part was random, but there is evidence that some special causes were at work in Massachusetts, and we might hypothesize that special cause  effects might be at work in DC..

Georgia, Florida, Tennessee, New York and North Carolina are U.S. examples of what Finnish educator Pasi Sahlberg calls the Global Education Reform Movement.  In each of these states, GERM has spread across these states, and we see classic GERM conditions, including the adoption of common standards, narrowing of curriculum focusing on math, writing and reading, high-stakes testing, a corporate management model which is data driven, and a system of accountability based on student test scores.

The graph below shows that the GERM model for most states is ineffective in changing math achievement.  I’ve examined reading in the same states during the same period, and the graphs are nearly identical.

The reforms that are in place in Georgia and other Race to the Top states will not affect student achievement in real ways.  The reforms are narrow and they ignore the ecology of learning by not seeing the school as part of a larger system.  For example, I asked in the last post why there was very little mention of poverty in Georgia’s reporting of their new method of grading schools.

Here is one reason.  Here is another graph of the same states, but this time showing poverty.  The graph is almost an inverse of the graphs shown in Figures 1 and 2. Notice that most states level of children living in poverty, except for Massachusetts (15%), has converged to the U.S. average which is about 23%.  What is the effect of poverty on student learning. Until we come look at the effects of the system on learning, we’ll make little progress in learning.

Using achievement scores is a poor indicator of student learning, and an even worse measure of teacher evaluation.

What do you think about the reforms that have been put into place as part of the Race to the Top?

Terrill L. Nickerson: The Paradox of the Common Core

rockies2 Terrill Nickerson commented on the previous post on this blog, 6 Reasons Why the Common Core is Not Progressive Ideology.  I thought his comments were important to share as a separate post.  Terrill Nickerson has written an interesting article on how he approaches the Common Core and high-stakes testing in his context of teaching, which is in communities serving marginalized and underrepresented families.

He writes:

In my twenty-six years teaching in schools with large numbers of marginalized, and underrepresented families, I do not agree with the assertion that high-stakes testing and Common Core State Standards (CCSS) sprang out of progressive ideology.  Most of my colleagues that work with these populations tend to believe the exact opposite.  The common feeling is that the high-stakes testing tends to be biased against the children that come from culturally different, marginalized, or economically poor families. Likewise, my colleagues would accuse the CCSS of failing to take into account the realities of the worldview and paradigms experienced by the these groups.

Realistically, I know these biases and shortcomings exist.  I have seen them firsthand, especially with regard to the high-stakes testing.  However, my paradox arises with the arguments, pro and con about the Common Core Standards.  I began my professional career as a scientist, not a science teacher.  After a decade of working in the professional science ranks, I decided to become a teacher.  I also continued to learn and progress, as I completed my M.S. Ed. in Science Curriculum and Instruction, while teaching.  I was working in a Native American school system and community.  So my professional growth and learning was applied to this community.

However, the communities, in which I taught realized that getting a mainstream education was the only way that their communities could survive into the future.  I was encouraged to challenge my students and present them with the highest level of education that I could.  I was also challenged to learn, and use the cultural strengths to carry out this task.  I did not find a contradiction in these expectations.

As a scholar and scientist, I see the value in creating a more consistent set of academic expectations.  Knowing what I know about what the science professions and the universities expect, I do not see the Common Core as a threat to our children.   The problem does not lie with the Standards themselves, but rather with the interpretation of how they should be implemented. I always insisted that if you teach sound scientific procedures and problem solving skills, students will do well on the high-stakes tests.

Teaching solid practices, regardless of your choice of content material, still builds a solid foundation.  This foundation teaches students solid test-taking skills by teaching them to be critical thinkers and to recognize inconsistencies and errors in logic through elimination.  My students were successful, and still are, even though the present educational setting insists that I follow the Standards more closely than before.

The Common Core doesn’t tell us how to teach.  Instead, it provides teachers with a guideline for what type of knowledge and information is both topical and cutting edge in keeping up with advances in our discipline.   Despite the emphasis upon the Standards teaching, I still find time to diverge and create projects for my students that are hands-on, project-based, and steeped in engineering and science methodologies, and still do justice to the Standards.

As I’ve always said, “I teach my high school students at a college level, with an understanding that the outcomes will reflect a high school level of sophistication and development, and grade accordingly  Do not tell them you are doing this, just expect it of them, and work with them in tandem to achieve it. They will rise to the occasion and expectations, and begin to accept them as the normal level at which they should be working.’  I have very few failures.

About Terrill Nickerson

Terrill Nickerson is veteran high school science teacher with 26 years experience.  His first 15 years teaching science began in the Native American community, beginning on the Hopi Reservation in NE Arizona, and then on to teach at Santa Fe Indian School in Santa Fe, NM.  He is now teaching in various charter schools in New Mexico and Southern Colorado.  He holds bachelor degrees in Archaeology and Geology, a Masters of Science in Education, and is working on his Ph.D.  After several years as a professional archaeologist and paleontologist, and experiences writing curriculum for CDC, he pursued a career in science teaching.  Terrill says that because of the width and breath of his experiences, he is able to bring real-life experiences to the classroom, and use the practical science experiences he used in the field.  He brings project-based teaching to his students, involving them in designing data collection devices to be used in their own investigations.  His work in the Native American community led him to become a practitioner of Gardner’s Multiple Intelligences.  He now teaches in a small rural, agricultural community, with a large migrant work population.  

6 Reasons Why the Common Core is NOT Progressive Ideology

imageA growing criticism of the Common Core State Standards and the Next Generation Science Standards is that its a way for progressives to inject their philosophies and ideology onto children and youth in American schools.

One reader of this blog made this comment about my post in which I discuss why Bill Gates defends the common core.

Common Core is Progressive Speak for a Nationalized (Common), Centrally Planned (Core), Agenda (Education), System (Standards). It will become a continuous accelerated march to Socialism and to the destruction of America through indoctrination of our kids. It is the means by which Socialists can insinuate better control of children and destroy the influence of parents on kids views, via electronic media teaching (See comment by Wordwaryor, March 26, 2014.

Any thought the standards movement is an idea hatched by progressives is without merit.  Indeed, the idea of standards is a conservative idea that proposes that what students learn is out there, and that what is out there can be expressed in discrete sentences or objectives, performances or standards.  Further, the idea is that not only can we tell students what they should learn, the standards spell out when.

On this blog I’ve written many posts summarizing the work of others who take a critical look at the standards movement, and its associated high-stakes testing mania. Here are six criticisms of the standards, and their effect on student learning.  None of these support the idea that the Common Core was the brainchild of progressive educators: Hint–think John Dewey, Maria Montessori, Alfie Kohn, Lisa Delpit. 

1. Brick Walls

 In the face of teaching and learning, standards are like brick walls. According to research published by Dr. Carolyn S. Wallace, a professor at the Center for Science Education, Indiana State University, science standards are barriers to teaching and learning in science. She makes this claim in her 2011 study, published in the journal Science Education, entitled Authoritarian Science Curriculum Standards as Barriers to Teaching and Learning: An Interpretation of Personal Experience.

One of the key aspects of her study is her suggestion “that there are two characteristics of the current generation of accountability standards that pose barriers to meaningful teaching and learning in science.”

  • The tightly specified nature of successful learning performances precludes classroom teachers from modifying the standards to fit the needs of their students.
  • The standards are removed from the thinking and reasoning processes needed to meet them.

And then she adds that these two barriers are reinforced by the use of high-stakes testing in the present accountability model of education.

Dr. Wallace’s suggestions are significant in that nearly every state has adopted the Common Core State Standards, bringing America very close to having a national set of common standards and possibly a national curriculum, at least in English language arts and mathematics, with science next in line to be adopted by each state.

An important point that Wallace highlights is that teachers (and students) are recipients of the standards, and not having been a part of the process in creating the standards. By and large teachers are not participants in the design and writing of standards. But more importantly, teachers were not part of the decision to use standards to drive school science, first. That was done by élite groups of scientists and educators.

In the rhetoric of the standards, especially Achieve, the U.S. system of science and mathematics education is performing below par, and if something isn’t done, then millions of students will not be ready to compete in the global economy. Achieve cites achievement data from PISA and NAEP to make its case that American science and mathematics teaching is in horrible shape, and needs to fixed.  The solution to fix this problem is to make the American dream possible for all citizens by writing new science (and mathematics) standards. According to Achieve, quality science teaching is based on content standards “that are rich in content and practice, with aligned curricula, pedagogy, assessment and teacher preparation.

The Common Core State Standards (CCSS) and Next Generation Science Standards (NGSS) are theorized to improve learning because the new standards are superior to the existing state standards. Indeed, two groups that studied the state standards did conclude the that Common Core standards were of higher quality. A second improvement to learning is that expectations will be higher than those that now exist in the Common Core and science. The claim here was that the states set their expectations too low, resulting in “inflated” results. And the third area of improvement in learning is that standardizing might lead to higher quality textbooks and other resources since they would only have to be aligned to one set of content standards.

2. The Social-Emotional Consequences

Anxious teachers, sobbing children was the title of an opinion article published in the Atlanta newspaper.  The article, written by Stephanie Jones, professor of education at the University of Georgia, asks “What’s the low morale and crying about in education these days? Mandatory dehumanization and emotional policy-making — that’s what.”

Policy makers, acting on emotion and little to no data, have dehumanized schooling by implementing authoritarian standards in a one-size-fits-all system of education. We’ve enabled a layer of the educational system (U.S. Department of Education and the state departments of education) to carry out the NCLB act, and high-stakes tests, and use data from these tests to decide the fate of school districts, teachers and students. One of the outcomes of this policy is the debilitating effects on the mental and physical health of students, teachers and administrators.

The emotional and behavioral disorders that youth experience have only been amplified by the NCLB act.
In research by Ginicola and Saccoccio, entitled Good Intentions, Unintended Consequences: The Impact of NCLB on Children’s Mental Health, they report that NCLB is indirectly damaging children by disproportionately stressing childhood education and blatantly disregarding other areas of child development. Their research on NCLB is enlightening and disturbing.

3. Dehumanization of Students and Teachers

In 2001, the U.S. Congress enacted the No Child Left Behind (NCLB). NCLB requires that each state develop assessments in basic skills, mathematics and reading, at first, but it has now expanded to other areas. The “testing game” is an annual event making every boy and girl take part (starting at grade 3) to make sure that their state and school continue to receive federal funding. The testing games that children and youth are annually required to take part in are used to find winners and losers. Unlike the Hunger Games, children are used to decide winning schools, teachers and districts. No one dies. However, we are testing the life out of our children and youth.

Here is how the testing games work. Student scores decide whether a school has done a good or bad job. Schools which receive Federal ESEA funding must make progress (known as Adequate Yearly Progress) on test scores. Schools compare scores from one year to the next, and use the difference to decide how well or poorly the children and youth did.

Students are not televised when they take these tests. However, the results are published in the local newspapers, and using the students’ test scores, schools that didn’t make AYP are labeled and their names published in the papers. And one more thing. Policy makers are hunting for bad teachers. To do this, they have required states to begin using VAM (Value Added Modeling) to rate teachers, and to then humiliate the teachers by publishing VAM scores in the local papers. Check Los Angeles. Check New York City. Check the State of Florida.

In the scenarios described above, The Hunger Games and The Testing Games, (read a fictional account of the testing games here) youth are dehumanized and used as gladiators, or in the case of The Testing Games pawns, where their moves are used to punish or reward states, districts, schools and teachers. On Valerie Strauss’ blog, there was a recent post that gets to the heart of the tragedy of The Testing Games, and how it is not only a dehumanizing event, but has nothing to do with helping students find out about their own learning.

4. The Research Evidence Is Not Supportive for the Standards

According to the 2012, Brown Center Reports, the Common Core State Standards will have little to no effect on student achievement. Author Tom Loveless explains that neither the quality or the rigor of state standards is related to state NAEP scores. Loveless suggests that if there was an effect, we would have seen it since all states had standards in 2003.

For example in the Brown Center study, it was reported (in a separate 2009 study by Whitehurst), that there was no correlation of NAEP scores with the quality ratings of state standards. Whitehurst studied scores from 2000 to 2007, and found that NAEP scores did not depend upon the “quality of the standards,” and he reported that this was true for both white and black students (The Brown Center Report on American Education, p.9). The correlation coefficients ranged from -0.6 to 0.08.

The researchers concluded that we should not expect much from the Common Core. In an interesting discussion of the implications of their findings, Tom Loveless, the author of the report, cautions us to be careful about not being drawn into thinking that standards represent a kind of system of “weights and measures.” Loveless tells us that standards’ reformers use the word—benchmarks—as a synonym for standards. And he says that they use too often. In science education, we’ve had a long history of using the word benchmarks, and Loveless reminds us that there are not real, or measured benchmarks in any content area. Yet, when you read the standards—common core or science—there is the implication we really know–almost in a measured way–what standards should be met at a particular grade level.

Then, in the 2014 Brown Center Report on Education, it was reported that the Common Core in math had insignificant effects on student math achievement.

5. Injustice

The authoritarian standards and high-stakes testing movement conjure up for me the use of power and privilege to create injustices for not only schools and teachers, but for students and their parents. Using invalid test scores, the government has cast a net around schools that have high poverty ratesresulting in many of them being labeled as failures with teachers and administrators fired, and replaced by teachers, many of whom are un-certified, and lack the teaching experience needed for these schools.

And all of this is done with data that is not only invalid, but is not reliable. As Dr. Michael Marder says, “the masses of nationwide data do point to the primary cause of school failure, but it is poverty, not teacher quality.” So what do we do? We create a system in which life changing decisions are made about teachers and students based on data that is not examined in the context of power, privilege, and income. This leads to a corrupt system in which we predicate schools’ and teachers’ performance on false data, and use these results to embarrass and destroy careers of highly educated teachers, and bring havoc to families. Why are we doing this?

6.  Testing

Many bloggers have added to the conversation about standards, and especially its companion, high-stakes testing.  One of the important voices in this discussion is that of Anthony Cody, a former science educator and curriculum developer who blogs over on Living in Dialog on Education Week.  Anthony has written extensively on standardized tests, and you can see all of his posts on this topic here.

Anthony brings to the table a strong knowledge base on current educational reform, perhaps more than any other blogger.  In one post, he explored some of the ideas of Governor Jerry Brown of California.  Brown strongly takes issue with a system of education that depends on experts from afar who impart their opinions about what should be taught and when, and who should decide what students are learning.  He is more concerned with how we teach our children, as he is with what.  In his view, education is about the “early fashioning of character and the formation of conscience.”

But more importantly his ideas are considered in the context of the state of California which has six million students and 300,000 teachers.  And three million of California’s school age students speak a language at home that is different from English, and there are more than 2 million students living in poverty.

He’s very clear on his place on testing.  Here is one comment he made in the State of the State speech:

The laws that are in fashion demand tightly constrained curricula and reams of accountability data. All the better if it requires quiz-bits of information, regurgitated at regular intervals and stored in vast computers. Performance metrics, of course, are invoked like talismans. Distant authorities crack the whip, demanding quantitative measures and a stark, single number to encapsulate the precise achievement level of every child.

In stark contrast to the place that poverty, violence, joblessness, home environment have little effect on academic performance,  he suggested the following for the coming year:

My 2013 Budget Summary lays out the case for cutting categorical programs and putting maximum authority and discretion back at the local level–with school boards. I am asking you to approve a brand new Local Control Funding Formula which would distribute supplemental funds — over an extended period of time — to school districts based on the real world problems they face. This formula recognizes the fact that a child in a family making $20,000 a year or speaking a language different from English or living in a foster home requires more help. Equal treatment for children in unequal situations is not justice.

Standards and Progressive ideology?

I want to be clear that neither the Common Core, nor the Next Generation of Science Standards are based on progressive ideology.

The progressive movement was about inclusiveness.  It was grass-roots movement that fought to change the economic, social, legal and educational problems that many Americans endured.  The standards are a top-down, authoritarian system–just the opposite of progressive thinking.

So, there you have it.  What is your opinion about the relationship between the standards movement and progressive thinking?

 

NAEP Math Scores Insignificantly Affected by the Common Standards

image

The Common Core State Standards (Common Standards) have been implemented for about four years. According to the developers (the folks over at Achieve) and it’s billionaire financiers, such as Bill Gates, the Common Standards are benchmarked against high performing international standards, and should result in higher achievement scores for American students.

According to Achieve, the Common Standards are guidelines for what every student should know and be able to do in math and English language arts from kindergarten through 12th grade.

According to proponents, the Common Standards should result in higher achievement scores and an increase in the nation’s ability to compete globally. Proponent-in-chief, billionaire Bill Gates, said in an interview with ABC News, “that in a decade, the scores and competence for U.S. students in math can be improved. This is going to be a big win for education.”

A recent report based on peer-reviewed studies in the Brookings 2014 Brown Center Report on American Education sheds light on these claims. The results will be a disappointment for some.

The 2014 Brookings report includes sections on the PISA international – Shanghai controversy, homework, and the Common Core. I’m focusing only on the Common Core section in this article.

CCSS Shows No Signs of Boosting Math Scores

There has been a lot of talk that the U.S. needs new standards to ratchet up our student’s abilities in mathematics (reading & science, too). The Brown Report provides empirical support to find out how the Common Standards are affecting American student’s 8th grade math scores.

According to an earlier Brown study (2012), it was predicted that the new Common Standards would not affect student learning. The 2014 study confirms this. Here is the concluding paragraph in the 2014 Brown Report.

The 2012 Brown Center Report predicted, based on an empirical analysis of the effects of state standards, that the CCSS will have little to no impact on student achievement.  Supporters of the Common Core argue that strong, effective implementation of the standards will sweep away such skepticism by producing lasting, significant gains in student learning. So far, at least— and it is admittedly the early innings of a long ballgame—there are no signs of such an impressive accomplishment.

Math Scores Lagged During the CCSS Rollout

The NAEP collects achievement scores every two years for 9, 13, and 17-year-old students. The graph in Figure 1 is the trend of 8th grade math achievement from 1990 – 2013. The average score in 1990 was , while average math score in 2013 is 285.

The trend line does not show a significant change in the slope of line during the time the Common Standards were implemented (the area on the line encircled in red.  One would have expected a bump during this period.  It isn’t there.

It turns out that a little math will tell the story. Between 1990 – 2013 there was a 22 point increase in 8th grade math. Over the 23 years this amounts to about a 1 point increase per year. However, the average score increase from 2009 – 2013, the years the Common Core has been used, has only increased 0.30 points per year, much less than before the roll out of the Common Core.

Figure 1. NAEP 8th Grade Math Scores, 1990-2013. Source. U.S. Department of Education

 

Digging Deeper

The Brown Report on the Common Core begins with an examination of research at Michigan State University by William H. Schmidt and Richard T. Houang.  These researchers published a study in 2012 that included an analysis of the math standards in place in all 50 states in the 2008 – 2009 about how they stacked up to the Common Core State Standards in Mathematics.  The question was how well do the state standards align with the CCSSM in terms of congruence, focus and rigor.  (Schmidt, W. H. & Houang, R.T. Curricular Coherence and the Common Core State Standards for Mathematics Educational Researcher November 2012 41: 294-308). 

Table 1 shows the results of the analysis of state math standards and their consistency with the CCSSM in 2008.  The table shows five categories based on the degree of concurrence with the Common Standards in math.  Those rated high (5) include states such Alabama, Georgia, Michigan.  Those rated low (1) include Arizona, Kansas, New Jersey.   Schmidt and Houang did an analysis to find if there was a relationship between achievement scores on the NAEP math test and the degree of congruence of the state standards with the CCSS in math.  There was no relationship to achievement.  However, their analyses did show two groups of states (Group A and Group B) that did show significant coefficients for congruence.

Table 1. Existing State Standards' Consistency with the Common Core Math Standards. Loveless, T. How Well Are American Students Learning. The Brown Center Report on American Education, March 2014, Volume 3, Number 3, p. 28.
Table 1. Existing State Standards’ Consistency with the Common Core Math Standards. Source: Schmidt, W.H. (2012), Presentation at the National Press Club, Extracted from Loveless, T. How Well Are American Students Learning. The Brown Center Report on American Education, March 2014, Volume 3, Number 3, p. 28.

Brown Analysis

According to the Michigan State study, states numberswiki.com

adopting the Common Core in math should expect to see an increase on NAEP 8th grade math scores. The Brown research extended the Michigan study by looking specifically at gain scores at the state level since 2009. By using NAEP math scores for 2011 and 2013, the Brown researchers had a way to test the predictive ability of the earlier Michigan study.

We should realize that this is only four years into the adoption of the Common Core. However, the findings ought to provide some feedback on the current status of the Common Core, and help us predict future effects of the standards on math achievement,

Table 2 presents state NAEP changes organized by the Michigan rating system of congruence with Common Core math standards. Recall, the a rating of “5” meant that those states were most like the CCSS math standards, those with a “1” were most divergent from the Common Core math standards. Accordingly, we would expect that there would be a trend in the gain scores favoring the higher rated states vs the lower rated states.

Table 2 shows there is no trend or systematic relationship between the Michigan ratings and changes in NAEP math scores. Note that states that had very divergent standards from the Common Core actually gained more than states that were rated most like the Common Standards.  As Loveless says, the data are lumpy.  When one expects high scores, the results are low scores, and visa versa.  There is no pattern in these findings.

Table 5. State NAEP Changes, by Michigan Ratings of Congruence with CCSS, (in scale score points, 2009 - 2013).
Table 2. State NAEP Changes, by Michigan Ratings of Congruence with CCSS, (in scale score points, 2009 – 2013).

How did the Common Core Math Implementation Fare?

In 2009, the U.S. Department of Education survey the states to find out how the Common Standards reform was doing and being implemented.  The survey asked states if they had: (1) adopted the CCSS; (2) provided, guided, or funded professional development on the CCSS; (3) provided curriculum/instructional materials for the CCSS; and (4) worked with a consortium to develop assessments aligned with the CCSS (Brown Study, p. 31).

In the 2014 Brown study, they researchers used the survey results to categorized the state’s implementation rating as strong, medium, or non-adopters, as shown in Table 3.   Note that the strong implementors of the Common Core made the largest gains, while non-adopters showed the smallest gains.

Table 3. Changes in NAEP Scores (in scale score points) by Implementation of CCSS. Extracted from Loveless, T. How Well Are American Students Learning. The Brown Center Report on American Education, March 2014, Volume 3, Number 3, p.32.

 
However, if we compare the gains shown during the four years of CCSS adoption of math standards to the entire history of NAEP scores, we find that during the transition period, scores have only gained 0.33 points, where the gain score per year over the entire history of NAEP math testing is more than “1” point per year.

Surprised?

Some will use these results to claim the Common Core is a failure. Others will say that more time is needed to “test” the new standards. Others might claim that the standards do not have much affect on how students do on NAEP tests.

In an earlier post, I reported that Brown (2012) researchers concluded that we should not expect much from the Common Core.  In an interesting discussion of the implications of their findings, Tom Loveless, the author of the report, cautions us to be careful about not being drawn into thinking that standards represent a kind of system of “weights and measures.”  Loveless tells us that standards’ reformers use the word—benchmarks—as a synonym for standards.  And he says that they are used too often.  In science education, for example, we’ve had a long history of using the word benchmarks, and Loveless reminds us that there are not real, or measured benchmarks in any content area.  Yet, when you read the standards—common core or science—there is the implication we really know–almost in a measured way–what standards should be met at a particular grade level.

Loveless also makes a strong point when he says the entire system of education is “teeming with variation.”  To think that creating a set of common core standards will reduce this variation between states or within a state simply will not succeed. As he puts it, the common core (a kind of intended curriculum) sits on top of the implemented and achieved curriculum.  The implemented curriculum is what teachers do with their students day-to-day.  It is full of variation within a school.  Two biology teachers in the same school will get very different results for a lot of different factors.  But as far as the state is concerned, the achieved curriculum is all that matters.  The state uses high-stakes tests to determine whether schools met Adequate Yearly Progress (AYP).

As the Brown report suggests, we should not depend on the common core or the Next Generation Science Standards having any effect on students’ achievement.  The report ends with this statement:

The nation will have to look elsewhere for ways to improve its schools.

What do think of the research reported here about the Common Core?

The Common Core: A Dream Come True for the Publishing & Media Industries

>Screen Shot 2014-03-21 at 6.15.08 PM

Imagine what it would be like if every school district in the U.S. used the same core (standards) curriculum, and that every few years, new textbooks and media products needed to purchased.

If the Common Standards are fully adopted across the nation, then it will be a booming business for media and publishing companies.

But it is not as simple as that.  There are heavy hitters out there that have grasped control of not only digital and print publishing of text material, but also control over high-stakes testing that will be based on one of two systems, the PARCC Assessment System, or the Smarter Balanced Assessment Consortium.

In an article by Beth Bacon over on Digital Book World, the environment for publishing is booming thanks in part to the adoption by most states of the Common Core.  Bacon also points out that falling tech prices, and the hunger for digital tools and materials has contributed to the boom.  Bacon uses the term “democratization of publishing,” but as I will show ahead, this is hardly the case.   However, it is the fact that American schools will be held accountable to the same set of standards that has propelled publisher ambitions.  In her article, Bacon quotes Justin Hamilton, Senior Vice President of Corporate Communications at Amplify.  Here is part of that conversation:

“The Common Core is the equivalent of the transcontinental railroad,” said Hamilton, “We’ve moving from a patchwork of standards across the country to common standards from coast to coast. It opens doors to inspire the best and the brightest to develop new, effective curricula.”

These new Common Core-based curricula often integrate digital media and internet connectivity via tablets. Some of the curricula are being developed by new publishers like Hamilton’s company, Amplify. Other digital textbooks are being created by traditional publishers. Still others are being designed by innovative school districts that have developed successful programs and want to share that success with others.

In the past, Hamilton has observed, conventional publishers created textbooks for the largest four states (Texas, New York, Florida and California) then “tinkered around” with those main textbooks to create textbooks the other 46 states. “It’s like conventional publishing has been serving four entries with unlimited side dishes,” said Hamilton. With that model, the driving force in educational publishing had been marketing, not innovation. (Bacon, B. 3 Reasons Educational Publishing is Booming. DBW. July 13, 2013).

But there is something very interesting about Bacon’s article, and that is the companies that she gives a mention: Amplify (education division of News Corp), Khan Academy, c-K12, and Desmos.

But Amplify gets the lions share of space in the article, and here is why.

Amplify is the education division of News Corp, a multinational mass media corporation,   To expand on my concern about the connection between the Common Standards and publishing corporation, New Corp is a good example.  News Corporation owns the following: New Unlimited in Australia, News International in the U.K., The Times, and the Sun in the U.K, Dow Jones & Company, The Wall Street journal, HarperCollins, Fox Entertainment Group, 20th Century Fox film studio, and the Fox Broadcasting Company.  It was also the owner of News of the World, the company responsible for phone hacking.  This is a huge company that seemingly has limitless resources, as well as powerful connections in the world of education.  It’s revenue in 2012 was $33.7 billion, and 48,000 employees.

In 2010, Joel Klein, former New York City School Chancellor was named executive vice president for News Corporation, and then later brought with him executives from New York City to head-up News Corp’s education division.

Amplify, according to its website, is built on the foundation of Wireless Generation, a company that creates mobile assessments and instruction analytics to schools across America.  Amplify’s digital products are “data-driven” and they are rolling out mobile learning for a new world of digital curriculum and assessment.

Doesn’t this sound very familiar.  It sounds like Bill Gates was whispering in the ears of executives at News Corp.

So here we have a gigantic company with unlimited resources ready to create digital materials that will be matched to one set of standards in mathematics and English/Language Arts.

So, Amplify will not only develop digital curriculum (in English/language arts, math, and science) matched to the Common Standards, it just announced a new tablet that was designed by Intel Education.  The new Amplify ELA curriculum is integrated with the new tablet. The new tablet will be out for the 2014 – 2015 school year at about $199 per year per student for three years.   The content that students will use using the tablet will be preloaded and developed by Amplify partners including cK-12, Desmos and others.

The Common Core has created an environment in which big corporations with power and resources will take the largest share of the education market, a market which has become standardized.   But more than that, it will build and sell tools that will result in vast amounts of data being collected (tablets in a wireless environment) on every move a student makes.

In the Digital Ocean, which I wrote about earlier this week, the delivery of instruction in a digital era will make it easy to track and test students, more so than can be done today.

What do you think is the effect of adopting the Common Standards the nature of teaching materials that will reach the classroom?

Photo Sean MacEntee, Creative Commons Attribution