by on February 21, 2012 Leave a Comment

A Perfect Storm Hits Public Schools

Guest Post by Steven Sellers Lapham

Note: Steven Sellers Lapham and Jack Hassard worked together on this post.

Public schools in America are under attack from many directions, and the U.S. Department of Education (ED) seems bent on delivering a lethal one-two-three punch. This decade will likely witness more neighborhood schools shutting down, crowded classrooms, excellent teachers fired, and children fobbed off to “online learning programs.” Let’s recall that Prince Edward County, Virginia closed its schools 1959-64, creating a “lost generation” of children who were hobbled, as adults, by years of missed education. Today, a school district in Delaware County, Pennsylvania, cash strapped and unable to pay its teachers, is being kept open only by a federal court order.

We now face the prospect of a school closing because the local tax base has withered, the state government is under water, and the federal government has deemed the school to be unworthy of aid due to lackluster scores on high-stakes student tests. ED, which should be the strongest defender of public schools, is making the problem worse.

Punch #1: Punish the Poor.

The slogan “Race to the Top” is social Darwinism at its most ugly: Reward those who are doing well (inevitably, schools in wealthier neighborhoods) and punish those who are struggling (predictably, schools in America’s poorer neighborhoods). A child in Oklahoma, Mississippi, or North Dakota should not have to rely on a state administrator’s clever grant-writing skills in order to receive a good education. Certainly, some grant monies should be available for innovation and experimentation in schools. But to make “success” the guiding star of educational policy is wrong.

Punch #2: Death by Paperwork.

States might avoid the draconian punishments of the No Child Left Behind law (NCLB) by applying to ED for a “waiver.” The mad rush is on. To date, eleven states have submitted a “flexibility report.” Georgia’s is 249 pages. California estimates that enacting all of the waiver requirements (unfunded mandates) would cost at least $2 billion and has declined to apply. ED could make the waiver process useful by placing a 1,000-word limit on applications (a bit longer than this essay) and asking only for a brief description of a state’s educational goals. This would free up teachers and administrators to do real work.

Punch #3: Absurd Metrics.

Teacher evaluations will be based on “student growth.” There is, however, no scientific basis for doing this. The practice contradicts a 2011 National Academy of Science report, “Incentives and Test-Based Accountability in Education.”

Using test scores to measure the efforts of teachers is a pseudoscience akin to phrenology of the 1800s, which purported to measure one’s intelligence according to the shape of one’s skull. It also brings to mind journalists and social scientists of the 1920-60s who misued prison statistics to “prove” that black people are genetically inclined toward criminal behavior.  In his Harvard University Press book, Khalil Gibran Muhammad established how a racial and racist, ‘scientific’ discourse promoted this idea.  Today, we use high-stakes test scores to “prove” that embattled schools are “failures,” and that hard-working professionals aren’t working hard enough.

There are many reasons why student test scores might not mount endlessly upward, such as an influx of non-English speaking immigrants; a rise in divorces; the town’s factory closes; family transience; a rise in home foreclosures; a sad absence of parents, who are serving in Afghanistan; etc. Or maybe the for-profit company that created the test got a little sloppy when it wrote the test questions, skewing the results. These powerful influences cannot be adequately controlled in a statistical analysis on the small scale of a single school district, a single school, and least of all, a single teacher.

Pushing Back

We must ban the use of standardized tests to make high-stakes decisions of any kind. Standardized test scores might be used ethically as a diagnostic tool (“Apply first aid here!”), but never as an excuse for punishment (“Bleed the patient dry!”). As a study by Fairtest has revealed, the system has placed an inhumane burden on teachers and administrators on the ground, resulting in cheating scandals in 32 states and the District of Columbia. Valerie Strauss reports that the “misuse of standardized tests mandated by public officials has created a climate in which increasing numbers of educators feel they have no choice but to cross ethical lines.”

Of course, teachers, like any professional group, should be evaluated and held to high standards. Experienced teachers and administrators in the school itself have personal knowledge of the teacher, students, local community and curriculum. Peer observation and evaluation have been a part of healthy educational settings for centuries. There are rigorous protocols for teacher evaluation provided by professional and subject-discipline associations. Let’s use those.

In New York State, 1,200 principals (and even more teachers) have signed a letter protesting the use of students’ test scores to evaluate their job performance. California, with more public school students than any other state, has jumped ship. So has Pennsylvania, apparently. “The emphasis on testing under the waiver plan is as heavy-handed as it has been under NCLB,” said educational historian Diane Ravitch, who served as assistant secretary of education.

Replacing NCLB with a new law could propel our nation’s educational standing. Secretary of Education Arne Duncan’s most cherished goal is to return the United States to first place in the percentage of the population who graduate from college. To do that, let’s provide every child who could benefit from daycare with free admission to Head Start, which is the most powerful predictor of success for children born into poverty. Then we can strive to make every school in every neighborhood in America a center of excitement and excellence, not just the chosen few.

Until Congress passes a new federal education law, ED can write its rules and marshal its resources to assist students, teachers, and schools – and stop punishing them. And it can adopt a new slogan to match this new ethic. How about “Raise All Boats!”

Have the Federal Government’s education acts (No Child Left Behind and the Race to the Top) created conditions that have led to the “perfect storm” hitting American education?  What do you think?

————————————-

– Steven Sellers Lapham is an editor at a nonprofit educational association. The opinions expressed are his own.

Share on TwitterSave on DeliciousShare via email
Filed Under: Assessment, Reform, Standards Tagged With: , , , , , , ,
by on February 16, 2012 Leave a Comment

Test-Based Reform: What Values are we Adding?

Post 2.  Read Post 1 here.

This post  was published on Anthony Cody’s blog, Living in Dialogue.

Practicing teachers, clinical professors, and researchers who work in the field know that assessing teachers or students requires much more than simply looking at test scores.  And indeed, researchers who have examined the value-added assessment system which purports to measure the “teacher effect” on student achievement test scores, question it’s validity and more important reliability.

The Data Used to Make High-Stakes Decisions on Teachers and Students

Value Added Effect

For example, Terry Hibpshman, of the  Kentucky Education Professional Standards Board, did an in-depth review of value-added models and concludes that even though VAM has been implemented in some locations (Tennessee, and Dallas), the methodologies “should not be considered mature or well-formed at this point in its history.”  Dr. Hibpshman goes on to explain that VAM models, by their very nature, are extremely complex, and unless one understands the statistical nature of these models, people are quick to make policy decisions without understanding the limitations of these models.

That said, the U.S. Department of Education (ED) has figured out a way to mandate linking student achievement test scores to teacher assessment using VAM.  If one reads the details of the NCLB Waivers, states must implement teacher and administrator evaluation that is tied in some way to student progress on high-stakes achievement tests.  This was initially a requirement for states receiving Race to the Top funds.  The Secretary of Education figured out a way to hold all states accountable to using VAM, because since he knew most states were chomping at the bit to reduce the hold the U.S. Department of Education because of the nutty No Child Left Behind act.  Now, any state getting a NCLB Waiver will have to use VAM as part of their assessment of teachers and administrators.

Now we have created a situation where states will use a system that has not been shown to be scientifically valid or reliable (VAM) by using high-stakes test scores which assume that the results on these tests tell us what students have learned in the course being tested, but also how much the teacher contributed (value added) to student progress.

Willis D. Hawley and Jacqueline Jordan Irvine, explain why students’ cultural identities are integral to “measuring” teacher effectiveness.

Drs. Hawley and Irvine  believe that the practices that teachers use should be part of any teacher assessment system.  Teaching practices, to be used in teacher assessment, need to be observed, or need to be described by teachers themselves.  In particular, the authors suggest that there are teaching practices that are called “culturally responsive pedagogy (CRP), and that these need to be included in any “high-stakes teaching evaluation.”

As Hawley and Irvine point out, culturally response teachers,

  • understand that all students, regardless of race or ethnicity, bring their culturally influenced cognition, behavior, and dispositions to school.
  • understand how semantics, accents, dialect, and discussion modes affect face-to-face interactions.
  • know how to adapt and employ multiple representations of subject-matter knowledge using students’ everyday lived experiences.

Hawley and Irvine identify six examples of CRP that taken individually can make a huge difference in embodying the racial and ethnical effects on student learning.  These practices are not new, but they reflect a more indirect approach to teaching and learning, and in all cases, the nature of the students is seen as fundamental in teaching.  Highly effective teachers use practices such as these, and they should be an integral part of the assessment of teachers.

  • Learning from family and community engagement
  • Developing caring relationships with students
  • Engaging and motivating students
  • Assessing student performance
  • Grouping students for instruction
  • Selecting and effectively using learning resources

Finally, we should add that the Board on Testing and Assessment (BOTA) of The National Academies issued a letter to the Department of Education on the Race to the Top Fund (RTTT).  The essence of the letter was a critique of the RTTT Fund’s insistence on linking student test scores to teacher effectiveness.  In the letter, the BOTA had this to say:

The initiative should support research based on data that links student test scores with their teachers, but should not prematurely promote the use of value-added approaches, which evaluate teachers based on gains in their students’ performance, to reward or punish teachers.

Achievement Test Scores

Do they measure what students have learned in a course of study?

Are achievement tests that are used as high-stakes assessments at the end of the school year a valid  measure of the curriculum standards specific to each teacher’s classroom, or are they estimates of what the curriculum should be, and estimates of what students should learn?

High-stakes test scores that are reported for students, schools, and districts are far from the reality of what students do and should learn.  We’ve been fooled into believing that test scores are valid measures of student performance.  Let’s look into this claim.

Let’s say we want to design a high-stakes test for mathematics for 8th graders in the state of Georgia.  The first item of business is to check the Georgia mathematics standards for grade 8.   According to the Georgia Department of Education , 8th graders:

will  understand various numerical representations, including square roots, exponents and scientific notation; use and apply geometric properties of plane figures, including congruence and the Pythagorean theorem; use symbolic algebra to represent situations and solve problems, especially those that involve linear relationships; solve linear equations, systems of linear equations and inequalities; use equations, tables and graphs to analyze and interpret linear functions; use and understand set theory and simple counting techniques; determine the theoretical probability of simple events; and make inferences from statistical data, particularly data that can be modeled by linear functions.

Please note this is only a summary of the 8th grade math standards!

There are 98 standards for 8th grade.  You can only create a test that a student can take in 1.5 – 3 hours.  We have to worry about student test stamina.  How long can an 8th grader sit for goodness sake.  Let’s say that we design a test with 75 items for the 98 standards.  First we note that not all of the curriculum can be “covered” in a single test, so the test makers must make a decision about which standards not to test.  We then realize if we are going to “test’ all of the standards, we only allocate one test item per standard, and then use the results on the test to claim that we have measured what a student has learned in 8th grade mathematics.  All done in one day, when in fact the student was enrolled in course that was at least 170 days of instruction.

The Long and Short of It All

We’ve created a standards-based testing system that is remarkably short on telling us what students have learned. We have also learned that the statistical model (Value Added Model) that has been shown to be inconsistent, and of questionable reliability.

How can we possibly use data from a complex system, the education of American students, to determine what the contribution of a teacher has on student learning?

What is the “tell” that creates this information on a teacher?  None.

There is more to teaching than simply preparing students for the test. There is attitude and effort, collaboration and teamwork, and the development of character.  There is inquiry, problem solving, creativity and innovation.

There is also more to preparing people to become teachers than dropping them into classroom with little or no preparation.  Why do we have it in our head that teaching requires little to no preparation.  Why do we entrust children with teachers who not licensed, when in the state of Georgia, a manicurist must take 9 months of intense training and pass two tests?

Do you think that a teacher’s effectiveness can be measured by using a complicated mathematical model that is based on student test scores?

 

 

Share on TwitterSave on DeliciousShare via email
Filed Under: Assessment, Reform, Standards Tagged With: , , , , , , ,
by on February 16, 2012 3 Comments

Test-Based Reform: Where is the Common Core Leading Us?

Part 1

Posted on Anthony Cody’s Living in Dialogue blog.

In a post last week, I reported that Georgia’s Cobb County School System rejected the superintendent’s proposal to hire 50 Teacher for America teachers for schools located in South Cobb.  Many of the South Cobb schools are underperforming schools.  I suggested that this was a good decision, but also indicated that it was done by default.  The default is, that the proposal never made it to agenda of the board and thus was withdrawn for the time being.  Is this another avoidance tactic?

Today, we expand our thinking to explore the nature of the achievement data that is used to make decisions about student progress and teacher effectiveness.  Our over obsession with test data has led to the narrowing of curriculum, and led to the goal of education to single a outcome—the improvement of test scores.  Period.

In the next post the challenge of teacher education is investigated, and the data is to show that high quality teaching results from teacher preparation programs that are clinically and experientially based.  Recommendations are then made for how alternative teacher education programs (including Teach for America) can be improved.

The Simple Mindedness of Test-Based Reform 

One of the serious issues plaguing education is that so many of us want a simplistic solutions to such a complex and diverse system.  We’ve been convinced that test score is a valid measure of student achievement, so much so that we willing to use the test scores to reward or punish students, teachers and schools.

Does One Size Fits All?

Have we made producing workers as the purpose of schooling? Is schools performance the basic tasks of creating enough workers with “adequate” abilities and the right attitude to become an employee?  (M. Peterson, 2009, http://www.wholeschooling.net/, extracted February 15, 2012).

The approach is, test- and standards-based reform seems to mean that school improvement is only based on a minimum set of core standards.

Achieve, Inc., has already developed the Common Core State Standards in Mathematics and Reading/Language Arts, and now in the writing phase of creating the Next Generation of Science Standards.  And right behind the common standards are the Partnership for Assessment of Readiness for College and Careers (PARCC), a state consortium led again by Achieve, Inc. to develop the next generation K-12 assessments in English and math.  Does this work? Does one size fitsall?

This model is rooted in the myth that the United States is not competitive in the global market place because our students don’t perform at high enough levels on guess what: achievement tests.  The truth is that the U.S. is very competitive, and has been for decades.  With basing their thinking on test scores, politicians and think tank types have convinced the public that American schools are a failure, and the one kind of reform that will help us “race to the top” is driven by just one fact: we must raise test scores, and they must be raised every year.  Get a grip.

Competitiveness of U.S. Citizens. The United States is economically competitive as reported in the World Economic Forum’s 2010-2011 Global-Competitiveness report, and as reported by Iris Rotberg in her book Balancing Change and Tradition in Global Education Reform.  According to the World Economic Forum report, the U.S. is one of only 35 countries in the world that are at the highest stage of development—the innovation-driven economy.

The United States now ranks fifth in the world in global competitiveness.  This ranking has fallen one position, from a higher 4th to a lower 5th in the last year.  At this time, the U.S. economy is the largest in the world.  However, the World Economic Forum researchers have concluded that the U.S. economic competitiveness has weaknesses.  The report reads that the weaknesses include the business communities’ criticism of the public and private institutions, that there is a great lack of trust in politicians, and a lack of a strong relationships between government and business.   And the U.S. debt continues to grow. (World Economic Forum Report, 2011 – 2012. www3.weforum.org/docs/WEF_GCR_Report_2011-12.pdf, p. 14, extracted February 15, 2012). 

According to the World Economic Forum, student test scores on international tests in reading, mathematics and science were not related to the weakening of the U.S.’s ability to compete. Period.

In Balancing Change and Tradition in Global Education Reform, Iris C. Rotberg, Research Professor of Education Policy at The George Washington University, concluded that continuing to use student test scores is not a valid argument to understand a nation’s competitiveness.  According to Rotberg, a nation’s competitiveness is too complicated, and is impacted by other variables as identified in the World Economic Report.  She puts it this way:

Other variables, such as outsourcing to gain access to lower-wage employees, the climate and incentives for innovation, tax rates, health-care and retirement costs, the extent of government subsidies or partnerships, protectionism, intellectual-property enforcement, natural resources, and exchange rates overwhelm mathematics and science scores in predicting economic competitiveness.

There is ample evidence that student test scores are not a barometer of U.S. economic growth, or depression.  U.S. test scores did not cause or contribute to the Great Recession, any more than they caused the Economic Boom of the 1990s.

For example, I have included a graph below which shows the United States GDP growth rate from 1957 (the year of Sputnik) to 2012.  If you scroll down to Table 2, which shows the NAEP trends in 13 year old math scores, you’ll note that student math scores rose during the last 39 years, while the GDP rose and fell during the same period of time.  What caused the GDP fluctuations?  Student test scores?  I don’t think so.

Table 1. U.S. GDP 1957 – 2012.  Source: http://www.tradingeconomics.com/united-states/gdp-growth, extracted February 15, 2012.

Rising Test Scores. We’ve been raising test scores ever since the National Assessment of Education Progress  (NAEP) began administering low-stakes tests in 1969 to a nationally representative sample of American students at grades 4, 8, and 12 for main assessments, and ages 9, 13, and 17 for long term assessments. For example, the trend in mathematics scores for 13-year old students has shown an overall increase in scores since 1973, when testing began by NAEP.  The data for 9 year-olds follows the same trend, while the for 17 year olds there is not a significant difference from 1973 – 2008, although it is higher now.   Long term trends in reading slowly rose from 1971 – 2008, in the same manner as mathematics scores.  The same can be said about science scores.

Table 2. Trend in 13 year old math scores 1973 – 20008.  SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), various years, 1973–2008 Long-Term Trend Mathematics Assessments.

Achievement Gaps. The achievement gap that exists between students by race/ethnicity is shown in Table 3 and by family income as shown in Table 4.  The NAEP data shows that compared to 2004, there was no significant change in the average scores of White, Black, or Hispanic students at age 17.  This is significant because this was the period during which the No Child Left Behind Act was implemented.  One of the underlying premises of the act was to decrease the gap among white, Black and Hispanic students.  The data shows that this did not happen.

Family income also has a powerful impact on achievement scores.  As shown in Table 4, there are significant differences the achievement scores on NAEP tests from 203 – 2011 based on family income.  Family income is determined by eligibility for free or reduced lunches.  As shown in the graph, the differences are quite apparent.  Abbott and Joireman have shown that family income level may contribute more to the disparity among students scores than race/ethnicity. Gerald Coles reports on Anthony Cody’s Living in Dialogue blog that the achievement gap between the rich and the poor has grown, especially in the last decade.

It isn’t enough to simply acknowledge the gap that exists among groups by race/ethnicity or family income.  We must go deeper and ask why and how the present form of educational reform, which purported to help solve the problem, has actually contributed to preventing any progress.

Lisa Delpit, Eminent Scholar and Executive Director of the Center for Urban Education and Innovation at Florida International University, and author of Other People’s Children: Cultural Conflict in the Classroom, and the forthcoming book, Multiplication is for White People: Raising Expectations for Other People’s Children, suggests that the “programmed, mechanistic strategies designed to achieve the programmed, mechanistic goal of raising test scores” strips away the humanity that should be the basis for education (Delpit, Other People’s Children, 2005).  The present obsession with standards and test scores has driven us further away from realistic goals that she calls for.  She puts it this way:

Nowhere is the result more glaring than in urban classrooms serving low-income children of color, where low test scores meet programmed, scripted teaching.  The reductionism spawned has created settings in which teachers and students are treated as nonthinking objects to be manipulated and “managed.”

Dr. Delpit explains that she is more concerned now with the development of the character of students than she was years before.  The reductionist goals of the present reform movement are not serving most children in American schools.  She speaks about Hyde Schools as an exemplar of schools that focus on the character of students, not on their test scores.  In her first book, Other People’s Children she quotes the founder of the Hyde Schools, Joe Gauld explaining that the message to students is:

  • that they have an important purpose on this earth and the unique potential to fulfill it.
  • that their true worth is measured not by their social status, intellect, or talents, but by the strength of their character
  • that we admire their attitude and effort, and care less about their actual achievements, because these will come with time if they develop character traits like those emblazoned  on the Hyde School shield: courage, integrity, concern, curiosity, and leadership.

Schools, like the Hyde Schools, base their work and curriculum of a set of goals that are very different than the narrow interpretation of our test- and standards-based school culture.  As Hyde Schools director and founder Joseph A. Gauld explains:

Over the years, making academic proficiency the purpose of American education has shifted the benefits of learning away from students and families, onto schools, colleges, businesses, and the education industry itself.

In her new book, Dr. Delpit reminds us that there is no achievement gap at birth, and is concerned that the conversation about student’s education has become limited and restricted.  She asks:

  • What happened to the societal desire to instill character?
  • To develop creativity?
  • To cultivate courage and kindness?
  • How can we look at a small bundle of profound potential and see only a number describing inadequacy?

Dr. Delpit believes that classrooms can be created that speak to “children’s strengths rather than hammering them with their weakness, and about building connections to cultures and communities.” (Delpit, 2012, p. XXI).

Dr. Coles suggests that family income or wealth can make significant differences in ending the achievement gap.  Work that provides a family with a decent income, work with reasonable hours, health care for all citizens, housing, and college education—these would help children in ways that Dr. Delpit documents so well in her research.

 

Table 3. Trend in NAEP Mathematics Average Scores for 17-year-old students, by race/ethnicity. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), various years, 1973–2008 Long-Term Trend Mathematics Assessments.  Click on the graph to go to the NAEP site where you can explore the data.

Table 4. Average 2003 – 2011 Mathematics Scores for 8th Graders by Family Income as Determined by Eligibility for Free or Reduced Lunches. Click on the graph to go to the NAEP site where you can explore the data.

What do you think about test- and standards-based reform?  Should we continue using high-stakes tests?

 

Share on TwitterSave on DeliciousShare via email
Filed Under: Assessment, Reform, Standards Tagged With: , , , , , , , , , ,
by on January 30, 2012 3 Comments

We Have Low Expectations for American Students in Math & Science!

Who the #@!% would make such a statement? Why would such a statement be made about America’s youth?

If you go the Broad Foundation Education page you will find the answer to the first question.  This is the first of four statements about American youth, followed by “stark” statistics.  The Broad Foundation says:

We have low expectations for American students.

Shame on them!

Image attributed to http://www.tagxedo.com

This is the foundation that has channeled over $400 million into education, primarily in charter schools, training of administrators, and online education.  It’s a very good time to be in the business of influencing and undermining public education these days, especially if you run a very well endowed foundation or corporation.

For years now, these same foundations and corporations are using statistics that misrepresent and pervert what is actually the case.  Data from tests, especially international test results are used by politicians, foundation heads, the media, and even the U.S. Department of Education to make proclamations about the status the country’s educational system.  Needless to say, American youth are beat over the head for not meeting someone else’s expectations.

TIMSS and PISA: The Super Bowls of Education

Two international assessments are: Trends in International Mathematics and Science Study (TIMSS) and the Program for International Assessment (PISA).  Each of these international organizations test students in mathematics, reading and science.  PISA studies 15 year-olds, while TIMSS assesses students in grades 4 and 8.  TIMSS claims to assess students’ performance on the curriculum, whereas PISA claims to test student’s abilities to apply what they have learned to real-world problems.  But please keep in mind, these are low stakes bubble tests comprised of a pool of questions that in general are without a context.

Since about 65 countries participate in these assessments, there is the general feeling that the results are important, and provide us with a glimmer of the nature of science education in these various nations.  Some would agree, others would argue that the real issues facing any nation’s educational system are masked by looking at average scores, and simple rankings.  Still others report that the findings are inconsistent.  For example, a country might score low on TIMSS, yet higher on PISA.  Most researchers urge that we use caution when interpreting the results, and not rely of simple averages (now someone’s thinking) to make judgements about student performance.

That said, Dr. Svein Sjoberg, Professor of Science Education, University of Oslo, and Director of the ROSE project (The Relevance of Science Education), an international comparative research project that gathers information about attitudes of students toward science & technology, makes this point regarding PISA:

the main focus in the public reporting is in the form of simple ranking, often in the form of league tables for the participating countries. Here, the mean scores of the national samples in different countries are published. These league tables are nearly the only results that appear in the mass media. Although the PISA researchers take care to explain that many differences (say, between a mean national score of 567 and 572) are not statistically significant, the placement on the list gets most of the public attention. It is somewhat similar to sporting events: The winner takes it all. If you become no 8, no one asks how far you are from the winner, or how far you are from no 24 at any event. Moving up or down some places in this league table from PISA2000 to PISA2003 is awarded great importance in the public debate, although the differences may be non-significant statistically as well as educationally.

If a team doesn’t win the Super Bowl, is that team a failure?  What do you think?  What does the public think?

Are our schools failing?  Is is a fair claim to say we have low expectations for American students?  The answer is no!

Let’s take a look.

The Math and Science Conundrum

It is easy to to make a quick decision about what you think about math and science education when you read headlines in the newspaper that report that the sky is falling on our educational system, or that we are experiencing another Sputnik moment.  But the teaching and learning of mathematics or science, as seen by practicing teachers and collaborating researchers is much more complex (and interesting) than the questions that make up the tests that PISA or TIMSS uses to assess mathematics and science in more than 60 nations.

The conundrum is this.  The vision of science that each of these tests measures gives meaning to scientific literacy that looks inward at the canon of orthodox science—the concepts, processes and products of science.  Science is seen through the lens of the content of science.  But added to this the fact that we have a second vision of science.  This vision of science includes public understanding of science and science literacy about science-related situations.  In this vision we are more interested in the context of learning, asvwell as the meaning that students attach to science and mathematics, and how it relates to their world.  The lens we use here to view science is within the framework of socioscientific issues (SSI).

TIMSS, because it is tied to the current traditional curriculum, is likely measuring the outcomes of vision I.  PISA claims to be measuring students abilities to apply what they learned to real situations.  But science education researchers Troy Sadler and Dana Zeidler disagree with this, and suggest that the test items that have been released publicly seem quite removed from the intent of the SSI movement.

Given this analysis, we are quite safe to claim that these tests are measuring Vision I of science education, and do not provide a full picture of what actually is happening in many classrooms, schools and districts.  Science education is more than learning terms, and concepts.  It should include problem-solving and inquiry, and investigations into problems that are relevant to students lived experiences.

Standings

Where do we stand?

PISA and TIMSS are favorite sources of data for foundations and corporations, and especially the U.S. Department of Education (ED) to use to show how poorly American students are doing in mathematics and science.  The Program for International Student Assessment ( PISA) is a system of international assessments that tests 15-year-olds in reading, math and science in 65 countries every three years.  The latest results are available for 2009.  The next will be administered in 2012.

Using scores from tests such as PISA or TIMSS to evaluate and assess science education misleads the public into thinking that science learning has been assessed in the first place.  For instance, in the United States there are more than 15,000 independent school systems, and to use a mean score on a science test, such as PISA, or TIMSS does not describe the qualities or inequalities inherent in the U.S.A.’s schools.  Furthermore, as we showed above, there are at least two visions for teaching science, and these tests seem to assess Vision I, ignoring perhaps more relevant and interesting science learning  that is taking place in many science classrooms.   That said, let’s look at two interpretations of data from these international tests.

Interpretation 1.

For example, take a look at these statistics that you can find here on the Broad Foundation website, most of which were based on PISA results from past years.

  1. American students rank 25th in math and 21st in science compared to students in 30 industrialized countries.
  2. America’s top math students rank 25th out of 30 countries when compared with
    top students elsewhere in the world. [PISA Math Assessment, 2006)]
  3. By the end of 8th grade, U.S. students are two years behind in the math being studied by peers in other countries. [Schmidt, W., 2003 at a presentation]
  4. Sixty eight percent of 8th graders can’t read at their grade level, and most will
    never catch up.

The Broad Foundation paints a picture of American education as a broken system, with little hope for many students, especially those who the Broad Foundation claims can not read at their grade level.

Interpretation 2.

Let’s take a look at another way to examine these data.  I have gone to the ED site that presents PISA data, and downloaded Highlights from PISA 2009 in reading, math and science to provide another view of the results.  Here is another interpretation, point by point.

  1. In mathematics, the only country of similar size and demographics that scored higher than the U.S. was Canada.  Most of the other countries that did score significantly higher were small European or Asian (Korea, Japan) countries.  The U.S. score was above the average score of OECD countries. Although there were 12 countries that scored significantly higher, there were only three that are similar to the U.S. in size and demographics.  We are not ranked 25th in math and 21st in science.   (source: PISA Data 2009)
  2. America’s top students’  performance place near the top of all students tested by PISA.  For example Dr. Gerold Tirozzi, Executive Director of the National Association of Secondary Schools, analyzed the PISA data from the lens of poverty, as measured by the percentage of students receiving government free or reduced lunches.  For example, Tirozzi found that in schools where less than 10% of the students get a free lunch, the reading score would place them number 2 in the ranking of countries.  This is very far from being 25th as reported by the Broad Foundation.
  3. Are we two years behind in the content of  math that is being studied by 8th graders?  There is no data that would support such a claim in the form of statistical analysis.  Curriculum differences have great variance from one country to another.  As in other countries, curriculum is implemented in American schools based now on the Common Core State Standards in mathematics, and the high-stakes tests that used in each state.
  4. It is not true that 68% of 8th graders can’t read at their grade level.  In the 2009 NAEP reading achievement-level results, 76% of American 8th graders were above  the basic level of performance.  The graph below shows 8th grade reading results, 1969 – 2011.  Yes, we have work to do, but the claim that 68% of 8th graders can not read is not justified.

NAEP Eighth-Grade Reading Achievement Results 1969 - 2011

 

 Trends in Performance

Here is the truth.

I have provided  graphs showing trends in science, mathematics and reading for American students as measured by National Assessment of Educational Progress (NAEP).  You will find that the trends reported by NAEP do not support the Broad Foundation’s opinions of American youth.

Science. U.S. students have significantly improved on the PISA test from 2006 to 2009, as shown in the graph below.  This trend is a positive sign, and disputes the claim that expectations for American students is low.  One of the ways in which data is perverted is to claim that American education, including science education is broken, and that the cause probably has to do with poor performance of “bad” teachers.  It is an unsubstantiated claim.

Average scores of 15-year olds in the U.S. and OECD countries in scienceSource: Fleischman, H.L., Hopstock, P.J., Pelczar, M.P., and Shelley, B.E. (2010). Highlights From PISA 2009: Performance of U.S. 15-YearOld Students in Reading, Mathematics, and Science Literacy in an International Context (NCES 2011-004). U.S. Department of Education, National Center for Education Statistics. Washington, DC: U.S. Government Printing Office.

 

Student performance is affected by a number of factors including gender, race/ethnicity, type of school, family income level.  The figure below shows Grade 4 results on the 2009 NAEP science assessment.  The graph shows relationship between family income (as measured by eligibility for reduced-price or free lunch).  Note that students of families with lower incomes perform lower than students from families with higher incomes.  This is an important factor when we interpret test scores, as Dr. Gerold Tirozzi found when he analyzed the PISA data from the lens of poverty.

Grade 4 Science Results, NAEP 2009 by Family Income. Click on the figure to explore this data in more detail.

Mathematics.  According to NAEP results, mathematics scores for 9- and 13-year olds were higher in 2008 when compared to previous years.  There was no significant change in the White – Black or White – Hispanic score gaps compared to 2004.  However, since 1973, Black and Hispanic students have made greater gains than White students.

Trend in Mathematics scores for 9- and 13-year olds 1973 – 2008.                                                                                                                                                    SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), various years, 1973–2008 Long-Term Trend Mathematics Assessments.


Reading. Overall, the national trend in reading showed improvement from 2004 to 2008 for students at three ages (9, 13, and 17). The average reading score for White, Black and Hispanic students was higher than in previous assessments.

Trend in fourth- and eighth-grade NAEP reading scores 1992 - 2011

Have you visited any of the educators in your community that teach science? Have you heard about any of the projects that they doing with their students? What do you think about the Broad Foundation’s crummy assessment of  American students’ performance in math and science and that we should have low expectations.  

What are they thinking?

Share on TwitterSave on DeliciousShare via email
Filed Under: Assessment, Humanistic Science, Reform, Standards Tagged With: , , , , , , ,
by on January 28, 2012 Leave a Comment

Quality Science Teaching

The No Child Left Behind Act is linked to the data that shows schools in California are teaching less science because teachers are pressured to prepare students for the required math and English high-stakes tests.

Valerie Strauss writes that Virginia is moving to require that students would only be required to take tests in math and English.  Students would not take tests in science and social studies.  On the one hand, this is a great idea because I believe high-stakes tests should be banned.  But on the other hand, there will be collateral effects on science and social studies because Virginia will put its emphasis on teaching math and English.  That is a bad idea.

Now, back to California.

High Hopes – Few Opportunities

Image attributed to http://www.tagxedo.com

In a recent research study entitled High Hopes – Few Opportunities: The Status of Elementary Science Education in California the authors reveal that intense pressure to meet accountability goals in mathematics and English has limited time for science, and teachers and schools do not have the infrastructure support needed to consistently provide students with quality science learning opportunities.

The research was conducted by administering 451 surveys to a sample randomly stratified  districts across the state of California.  One administrator from each district was asked to complete the survey.  Then from the 451 districts, 300 schools were randomly selected and their principals, and 3 teachers (selected randomly) from each school were asked to complete the survey.  The response rate was 62%, 56% and 70% respectively. Nine schools out the 300 were selected by a nomination project for in-depth case studies.

Among the findings:

  • 40 percent of elementary teachers say they spend 60 minutes or less teaching science each week.
  • Only One-third of elementary teachers say they feel prepared to teach science.
  • 85 percent of teachers say they have not received any professional development in science during the last three years.
  • Nine in ten principals say science education is very important and should start early.
  • Less than half of principals (44%) believe it is likely that a student would receive high-quality science instruction in his or her school.

According to the researchers, children rarely encounter high-quality science learning opportunities in California elementary schools because the conditions that would support them are rarely in place.  There is evidence in the study that the narrowing of the school curriculum to meet NCLB guidelines has created a curriculum that focuses primarily on math and English to the peril of other subjects.

Why is Quality Science Education Crucial?

In Taking Science to School, a report published by the National Research Council, the science education researchers concluded that all students have the intellectual ability and curiosity to learn science (and social studies).  The research base in science education supports the idea that young students are quite capable of becoming involved in science inquiry and can sort through various forms of knowledge to determine what is reliable and what is not, and indeed, engage in the practice of science.   Furthermore, when students are engaged in hands-on activities that are related to their lived experiences, students become more motivated to learn, and this of course will have positive effects on learning math and English.

When we de-emphasize science in the elementary school curriculum we loose opportunities for students to collaborate, to discuss and talk, and to explore their curiosities and interests—all of which under-gird our reasons for teaching science. Science inquiry should be fostered early in a child’s life, yet the California study showed that very few students are receiving a quality science education, and if Virginia commits schools to testing in math and English only, then schools will find it in their best interest to overexpose students to math and English at the expense of other subjects, especially science and social studies.

What is quality science education?

The researchers identified a number of conditions that influence the quality of science education in K-6 schools and these included teachers, instructional materials and resources, and assessing student progress.  Researchers found that schools with high poverty rates were likely to report a lack of facilities and also that limited help was provided to help them assess student progress.

Quality science education in the elementary classroom is dependent on the school district encouraging science in the curriculum, and providing the professional training for teachers to implement such a program.  According to recommendations from the National Research Council, students should be engaged actively in science activities to find how science works.  They need to collaborate with other students, and in so doing use the language of science to explore questions of interest to themselves and their peers.

Although hands-on science is an important mantra for science teaching, students need to have opportunities to build on their prior knowledge and interest by engaging in the practice of science—or as teachers would say, science inquiry.  Students need to learn science concepts, yes, but they should do this within a context that has meaning to them.

The amount of time devoted to science teaching in California elementary schools is about 90 minutes per week, and the amount of funds available to improving science education has diminished from about $10 million to about $1.3 million.  Lower numbers of teachers are involved in staff development and little support for new science.

Although the present economic state of the country, and California had led to diminished resources in science education, the NCLB Act probably has contributed more to the amount of time spent teaching science, and in some instances the pedagogy implemented to teach science.

Will Banning Science and Social Studies Tests Result in High Scores in Math and Reading?

NAEP test results are the only accurate performance measure that we have.  The tests are low-stakes.  The samples are large enough to produce reliable national and state results, but can not be used to rank schools, districts or students (please see Richard Rothstein’s article on this).  Even in a state like California where researchers indicated that schools were not teaching science because they had to spend more time on math and reading, NAEP reading scores did not change relative to the national scores, and increased very little.  With all of the effort since the passage of NCLB on math and reading, the overall change is very little, nationally or by state.

Average scale scores in Reading comparing California to the Nation, 1998 - 2011 (NAEP)

Average Scale scores in Reading comparing California to the Nation, Grade 8, 1999 - 2011 (NAEP)

High-stakes testing has created a conundrum in American education. Whether Virginia changes which subjects are tested, the evidence does not support the legislators who came up with this idea in the first place. It simply reinforces that NCLB has created a teach-to-the-test teaching style, and until legislators, and corporate leaders move to the side and listen to educators, we’ll see very little change in test scores.

Share on TwitterSave on DeliciousShare via email
Filed Under: Assessment, Reform, Standards Tagged With: , , , ,
« Older Posts