Notes on ‘Scientific’ research in education and NCLB

While doing any research, I get incredibly distracted by side ideas that pop up. I’ve decided to use my blog as a place to take some of those notes for ideas that may be the start of future research, or may prompt someone else to go down that road. Today we started with an article called “School Reform 2007: Transforming Education into a Scientific Enterprise” by Barbara L. Schneider and ­Venessa A. Keesler College of Education and Department of Sociology, Michigan State University. Annual Review of Sociology Vol. 33: 197-217 (Volume publication date August 2007)
On scientific research and NCLB
It’s a very compelling review of how we got to a Scientific view of educational research and how “random assignment field trials” are de rigueur inside the educational field right now. I was particularly interested in how the US Federal educational people have released a report detailing how they like this to be done. (p. 3586) It goes on to say that one of the effects of this has been that
“NCLB (No Child Left Behind) places two key ideas at the center of school reform: performance and scientific evidence.”

The problem with this is that projects like this

emphasize incentives for outputs (student achievement) rather than the inputs that we know matter, such as raising teacher educational expectations, helping students spend more time on task, engaging students in challenging material, providing students with frequent and positive feedback, and helping students learn to be strategic in reaching their goals.

This is a critical difference for me. Focusing on the results of an activity sort of leave aside the process by which those results were achieved. I always think of the way calculus was taught to me. We never saw hide nor hair of a practical real world example of what made calculus so integral a part of the sciences. The work by which i ‘understood’ it, that is, could prove my mastery by doing problems had little or nothing to do with the way that those things might be achieved in an ‘applied’ way. Nor did I learn very much about how to learn something ‘like’ calculus for myself. I learned to remember a basic set of values, that did not really represent the base set that professional mathematicians would use, nor did it really make any contextualized sense to me. It was, in effect, an alienating activity.

Research Results from NCLB and possible explanations

The Harvard University Civil Rights Project also sponsored an extensive study on test score gains during the first years of NCLB. Comparing fourth and eighth graders’ National Assessment of Educational Progress (NAEP) reading and mathematics achievement test results prior to NCLB (1990–2001) to its initiation through 2005, Lee (2006) finds that achievement did not significantly improve. Moreover, racial and socioeconomic achievement gaps between advantaged whites and disadvantaged students have not changed significantly. State tests, however, appear to show greater gains, which suggest several alternative hypotheses: The states may be setting the bar on their tests too low; NAEP may not be as good an instrument as state tests for assessing year-to-year progress; or students may have no incentive to do well on NAEP tests so that trying to equate the tests, even if one could make the test items comparable, would be biased in favor of exams that are incentive based (Loveless 2006).

I particularly like the way this quote lays out some of the possible explanations for the results. It could be that they learned more, could be that the State reps have changed the rules to make the results look better.

As anyone who regularly reads this blog knows… i’m not super fond of this kind of research. I think that it has its place among the strata that represent the work done in education. I think that a drastic increase in test scores related to class size would be significant. (Interestingly, this article sites research that shows it has little to no effect) It should not, however, carry the day on anything. Any test, or any research, no matter how ‘randomized’ is immediately ‘framed’ by the person who has defined the research questions. The common response to this is that there are ‘rules’ governing the framing of research questions that reduce the risk of this. I agree, some research questions are awful. Some less so. Some very interesting.

The underlying assumption that truth is sitting there, waiting to be discovered, however, informs this kind of research. It assumes that THERE IS A PERFECT education system out there to be discovered. That there will be a solution if we look hard enough. This is highly unlikely. The unified educational theory that will support all students is a windmill, nothing more. It is an artifice that looks like universal education, and is mostly normative. It is designed by those in power, designed to serve the skills that those people value. This is not a statement about their intentions… just a necessary result of the way we are socialized. I think the things I think are important are important. You, likely, do the same. It is difficult, in a multicultural society, for that to work for everyone.

Any time that you use money as an ‘encouragement’ for schools to ‘perform’ you are running the risk of simply having the rules of the game change from under you. To quote the old expression… “teaching to the test creates great test takers”

Three things to remember when thinking about educational reform

1. Most educational reforms are about political grandstanding
2. Scale is important… what works on a small scale may not transfer to a larger scale
3. Few reforms receive continual support.

Hess (1999), in a study of school reform and school politics in a stratified random sample of 57 urban U.S. school districts from 1992–1995, offers another perspective on the failure of reforms of the 1980s and 1990s. First, most reforms, he contends, are not serious attempts to change teaching and learning but are more about political grandstanding, especially for those in high-level school decision-making positions. Second, some small-scale reforms fail when placed in larger settings, suggesting either that they work only in certain situations or that the requirements to bring specific interventions to scale remains unknown. Third, few reforms receive continual support. Those who engage in reforms are quickly churned in the process. Principals and teachers may invest in reforms that quickly fall out of fashion, and they are then left feeling defeated after having committed resources, time, and personal psychic energy to a wasted effort. Or some veteran teachers simply wait out the latest proposals, sticking to their own methods until the reform efforts fade. These teachers can become a source of inertia for change, leaving reforms to inexperienced professionals who often lack institutional memory on why certain interventions have or have not worked. Hess (1999) concludes that the professional and political interests of urban school leaders need to be linked to the long-term performance of their schools, a diagnosis that is consistent with the prescriptions offered in NCLB. Article quoted from Hess F. 1999. Spinning Wheels: The Politics of Urban School Reform. Washington, DC: Brookings Inst. Press

Author: dave

I run this site... among other things. View all posts by dave