Progressive Science Education

I have been reading and have referenced on this weblog the October 2009 special issue of the Journal of Research in Science Teaching (JRST) on the topic/theme “Scientific Literacy and Contexts in PISA Science.”  The articles in the special issue provide a broad view of international testing as conceived in PISA, as well as the TIMSS.

One of the articles (by Sadler and Zeidler) which was focused on PISA and Socioscientific Discourse, used the term progressive science education as a way to describe a vision of science education that includes public understanding of science, humanistic science education, context-based science teaching, S-T-S, and socioscientific issues.  As pointed out by the authors, the term progressive science education was used by George DeBoer in 1991.

A bit of background.  In a paper written by Douglas A. Roberts (Scientific Literacy/Science Literacy) that appeared in the 2007 Handbook of Research on Science Education, the author introduced two visions to explore the notions of scientific and science literacy, namely Vision I and Vision II.  In Roberts view, Vision I gives meaning to scientific literacy by “looking inward at the canon of orthodox natural science, that is, the products and processes of science itself.”  As Roberts states, this approach envisions literacy (or, perhaps, thorough knowledgeability) within science.  He goes on to point out that the Benchmarks for Science Literacy by the AAAS approximates his view of Vision I.  I would add that the National Science Education Standards (NCES) imparts Vision I as well.

To Roberts, there is a contrasting and quite different vision of science, Vision II, which gets its meaning from “the character of situations with a scientific component, situations that students are likely to encounter as citizens.”  Roberts defines this vision as literacy (again, read thorough knowledgeability) about science-related situations.  In my view, a very good description and discussion of Vision II is by Glen Aikenhead in his book, Science for Everyday Life.

We might think of Vision I as traditional science education; Vision II as progressive science education.

In the JRST special issue on PISA Science, some of the authors suggest that most of the documents produced in the past 20 years under the “standards movement era” tend to support Vision I.  Indeed, we could also suggest that most state-standards are written as Vision I science literacy.  At the US national level, the NAEP assessments focus on Vision I.  At the international assessment level, we might identify TIMSS as a Vision I marker.

There is some suggestion in the JRST issue that PISA 2006 aligns very closely with the Vision II view of science literacy described by Roberts.  This view was suggested by the editors of this special issue, but Sadler and Zeidler wrote that they have serious concerns about the extent to which PISA supports progressive science education.  Can progressive science education, or Vision II science literacy be “measured” by the use of a standardized assessment such as PISA?

The answer to this question is probably not.  As much as the authors of PISA would like us to believe that the test measures contextualized and controversial topics, others argue that the items are really decontextualized.  I found the items on PISA to be quite complicated, and required a lot of reading, and in some cases, what the students were asked to read had little or nothing to do with the questions that were asked.   One thoughtful evaluation of the PISA assessment program was written by Svein Sjoberg (see PISA and “Real Life Challenges: Mission Impossible).  He suggests that, although PISA claims to test “real-life skills and competencies in authentic contexts,” such a goal is impossible in a traditional testing environment as described in the PISA documentation.

Progressive science education (humanistic science education) will require a different form of assessment, and one that will rely on the observations, and active assessment of learning in the context of classrooms by science teachers and researchers.  The most effective form of classroom assessments that contribute to our understanding of student learning, and indeed help students improve in their learning are formative assessments, not summative assessments in the form of PISA, or TIMSS.

Yet, in the USA, where science education has actually made a great deal of progress (see Lowell & Salzman), the winds of change are aimed at further standardizing teaching by the “common standards movement.”  This will be followed by the development of “common tests” which will be used to compare and contrast schools, school districts, states, and individuals, including teachers and principals.

More to come on this topic.

Should PISA Type Assessments be used to evaluate Teacher Performance?

There is a clear mandate to build “data systems that measure student success and inform teachers and principals how they can improve their practices.” This is one of the “reform areas” in the U.S. Department of Education’s Race to the Top Fund. A second area of reform is the adoption of internationally benchmarked standards and assessments that prepare students for success in college and the workplace. There are two other areas including the recruitment of effective teachers and principals, and an effort to turn around the lowest-performing schools.

In order for States to be eligible under this program (Race to the Top), a State must not have any legal, statutory, or regulatory barriers to linking student achievement or student growth data to teachers for the purpose of teacher and principal evaluation. Only a few States (California and New York) had a law that would restrict the use of student test data to evaluate individual teachers (see this article in the New York Times for details). California has since repealed the law, making it eligible to apply for $4.6 billion in Race to the Top Funds.

The second area of reform that I want to mention here is the adoption of “internationally benchmarked” standards. As you know there is a movement afoot called The Common Core State Standards Initiative under the auspices of the Council of Chief State School Officers, and the National Governors Association for Best Practices. Nearly all of the States are on board this initiative. The movement is to create a common set of standards (first in math and reading), and then to develop assessments based on these standards. All of this effort is designed to standardize curriculum and assessment, and in my opinion reduce the emphasis on innovation (and in science education this would mean less emphasis on inquiry-based teaching and learning, and increase in content-based instruction). But there is an interesting phrase in this reform effort, and that is “internationally benchmarked.” All of this has to do with the drive to “align” (line up) standards, assessment, curriculum, teaching materials, and so forth. When you read the documents upon which this thinking is based you will find words and phrases such as rigorous standards, true common standards, end-of-high school expectations, evidence-based learning.

The common standards movement, and the insistence by the Federal Government that States can have no law that would prevent student achievement scores being used to assess teachers and principles creates a non-innovative environment for education in the nation. And a recent report on American schools entitled Leaders and Laggards, suggested that schools lag in educational innovation. The report showcases examples of marque reformers, many private ventures into schooling (EdisonLearning, Wireless Generation), charter schools, and some teacher education reform efforts such as Teach for America. None of these are without their critics, yet according to the report, there is much to learn from these efforts. But overall, the report found much that “impedes innovation” such as rigid bureaucracies, finance systems of schooling and others. Yet, when you examine the details of the report, the movement of common standards, and the use of achievement tests to evaluate teacher performance are seen as principles in an innovative school culture. The report basically endorses the Race to the Top Fund strategy, and our overall penchant for driving schools toward a more regulated and common curriculum.

PISA type assessments could be seen as an example of an international benchmark of what students should be able to know and do in science, and could then be translated by educational bureaucrats into the framework for developing assessments that would be used to evaluate teacher performance. Very unfortunate if this happens.

More to come on this topic this week. In the meantime, what do think? Please add to this discussion.

The Race to the Top: Climbing Mt. PISA

In the last post, I referenced a recent special issue of the Journal of Research in Science Teaching which was devoted to an exploration of scientific literacy and context in PISA Science.  PISA (The Programme for International Student Assessment) is one of two (the other is TIMSS) major international assessments of science that has captured the attention of educators, politicians, policy makers, and the general public.  Since about 60 countries participate in these assessments, there is the general feeling that the results are important, and provide us with a glimmer of the nature of science education in these various nations.

With that said, it is interesting to note that the media gives special attention to the results of these assessments.  As Svein Sjoberg points out:

the main focus in the public reporting is in the form of simple ranking, often in the form of league tables for the participating countries. Here, the mean scores of the national samples in different countries are published. These league tables are nearly the only results that appear in the mass media. Although the PISA researchers take care to explain that many differences (say, between a mean national score of 567 and 572) are not statistically significant, the placement on the list gets most of the public attention. It is somewhat similar to sporting events: The winner takes it all. If you become no 8, no one asks how far you are from the winner, or how far you are from no 24 at any event. Moving up or down some places in this league table from PISA2000 to PISA2003 is awarded great importance in the public debate, although the differences may be non-significant statistically as well as educationally.

Whenever international results are reported, whether it be the results of TIMSS or PISA, the mean score of each country is reported in various charts.  I’ve provided a copy of the chart that is a rank ordered list of the mean scores by country on the 2006 PISA Science Assessment.  All of the countries shown in yellow are statistically above the PISA average; those in green are statistically below the average; and those in white are not statistically different from the average.  You’ll note that the United States is in the green section, with a mean score of 489.  At the top of the ranking is Finland, whose students attainted a mean score of 563.

PISA 2006 Assessment Results
PISA 2006 Assessment Results

I am not sure what the underlying rationale is for the U.S. Department of Education use of the phrase, “The Race to the Top,” but one interpretation is that there is a race to get to the top of “MT” PISA in the foreseeable future.

Using scores from tests such as PISA to evaluate and assess science education misleads the public into thinking that science learning has been assessed in the first place.  For instance, in the United States there are more than 15,000 independent school systems, and to use a mean score on science test, such as PISA, or TIMSS, or NAEP does not describe the qualities or inequalities inherent in the U.S.A.’s schools.

The Race to the Top is an unfortunate choice of words because it implies that moving up the mountainside of the PISA scorecard is the way to improvement and success in science education.

There is a need for the science education community to raise serious questions about implying that these assessment scores are valid assessments of science education.  I’ll talk more in the coming days about some of the movements going on to further standardize learning, and use high risk assessments of student achievement as a means to evaluate teacher effectiveness and school improvement.

PISA: Can this test measure the outcomes of progressive science education

Long title, sorry.  But, Volume 46, Issue 8 (October 2009) of the Journal of Research in Science Teaching was devoted to Scientific Literacy and Context in PISA Science.  The entire issue was devoted to this theme.  In one of the articles in this volume (Scientific Literacy, PISA, and Socioscientific Discourse: Assessment for Progressive Aims of Science Education), the authors used the term progressive science education in the way George DeBoer used in many years ago to summarize movements in science teaching that included public understanding of science, humanistic science education, context-based science teaching, STS, and socioscientific issues science education.

How can the aims of progressive science education be measured?

According to some, The Programme for International Student Assessment (PISA), which is coordinated by the Organization for Economic Co-operation and Development (OECD), suggests that their test, which assesses 15 year-old students in nearly 60 countries, can do so.   The PISA assessment in science (there are also PISA tests in Reading and Mathematics), which is described in the first article in this volume, purports to assess

  • scientific knowledge and use of that knowledge
  • understanding of the characteristic features of science
  • awareness of how science and technology shape
  • our world willingness to engage in science-related issues

The last administration of the science test was 2006, and more than 40 countries participated in the test. You can see sample test items here to get a feel for the nature of the test questions used on the PISA science test.

I was happy to see a little bit of criticism in two of the articles in the research journal, but overall I felt as if the journal was endorsing the PISA assessment.  It’s the criticism that I was interested in exploring, especially since science education around the world is very much structured around standards, and the resultant high stakes science achievement tests that are used to measure student progress, and now, teacher assessment.

Some of the authors pointed to an article written by Douglas Roberts, in which he differentiates between two different kinds or visions of science teaching.  Vision I emphasizes subject matter itself; Vision II emphasizes science in life situations in which science plays a key role.

According to many of the authors of this NARST issue, PISA has developed an assessment system that aligns “very well” with Vision II.  And indeed, if you go ahead and look at the sample of test items from the last PISA science test, there is the air of application, and use of science.   In this view, Vision II be seen as progressive science education, and if, indeed, PISA claims to be able to “measure” these kinds of outcomes, then it would indeed be an attractive instruments for science education.

But in my view, its simply another large scale test, that really does not assess how students use science in lived experiences.  Svein Sjøberg challenges the wisdom of PISA’s claim to measure students’ real life experiences and science.  Sjøberg is a professor of science education at the University of Oslo, and director of another large scale project that assesses students attitudes about science.  In his research on PISA, he points out that:

The main point of view is that the PISA ambitions of testing “real-life skills and competencies in authentic contexts” are by definition alone impossible to achieve. A test is never better than the items that constitute the test. Hence, a critique of PISA should not mainly address the official rationale, ambitions and definitions, but should scrutinize the test items and the realities around the data collection. The secrecy over PISA items makes detailed critique difficult, but I will illustrate thof the items with two examples from the released texts.

Sjøberg provides more details.   I’ll talk more about this in the days ahead.  I’ll also come back to some of the criticism that was included in two of research papers in the Journal of Research in Science Teaching.  In the meantime you might enjoy reading some of the abstracts in the JRST volume, and Sjoberg’s article.