Author: Stockard, Jean
Date published: July 15, 2011
Journal code: JRRE
I appreciate the opportunity to reply to Professor Eppley's comment on my paper that was recently published in the Journal of Research in Rural Education (Stockard, 2011b). Her paper contains numerous statements that misrepresent both the content of my original paper and the social science literature as well as a number of provocative philosophical comments. In the pages that follow I address each of these areas
Misrepresentations of Stockard, 2011
Readers of this exchange are urged to read my original manuscript rather than Eppley's summary of it, for her characterizations bear almost no resemblance to the actual content of the paper. The article is a quantitative analysis of data from over 800 students in three different districts in one rural state. All of the schools had proportionally more students at risk (measured as receipt of free or reduced lunch or minority status) than in the state as a whole. A cohort control group design (Cook & Campbell, 1979) and linear growth models were used to examine variations in growth in reading skills, comparing students with full exposure to Reading Mastery (RM) (defined as having had the curriculum from the beginning of kindergarten) and those with less exposure (beginning in later grades). Results indicated that students with full exposure had significantly higher reading skills, that these differences persisted through the primary grades, and that significant differences also appeared on state reading assessments given in the fourth grade.
First, Eppley contends that the intent and purpose of my study was "pedagogical development" (Eppley, p. 1). In fact, as clearly stated in the abstract as well as in the introduction (Stockard, 2011b, p. 1-2) and repeated in the summary and conclusion (p. 14), the purpose of the study was "to examine changes in reading skills through the primary grades of students in three rural, Midwestern districts that occurred after the implementation of a highly structured and explicit reading curriculum (Reading Mastery) with implementation-associated support and guidance" (p. 1). The mention of pedagogical skills reflects my citation of Arnold and associates' suggestion that identifying ways to help rural schools improve teachers' "pedagogical skills in ways that have the greatest impact on student achievement" should be a priority area of research in rural education (Stockard, 2011b, p. 1, citing Arnold, Newman, Gaddy, & Dean, 2005, p. 18). Thus, those interested in pedagogical development could use the findings, but this was not the stated purpose of the study.
Second, Eppley claims that I recommend teachers and school leaders' "purchase of a commercially produced 'curriculum'" (2011, p. 1). As noted above, my study found that students had significantly higher reading skills when they had full exposure to the Reading Mastery curriculum. While this result, coupled with that of many other studies of the curriculum, could prompt reasonable people to believe that the program could benefit children and should be used, there is no place within the paper where I make such explicit recommendations. She also claims that my "recommendation is that teachers intentionally avoid making connections between a child's life, background knowledge, and interests during the teaching of reading" (2011, p. 3). She gives no quotation from the paper to support this claim, and I have searched the article in vain for such a "recommendation." Her statements appear to be fabrications and included only to help support her polemics and cast aspersions.
Third, Eppley states that I "paint rural schools as places of lack," implying that I had low regard for the capabilities and skills of the rural teachers (2011, p. 2). Again, however, she provides no evidence from the paper to support this conclusion. In contrast, the concluding statement of my article describes the strengths of these schools and their efforts to help young people.
The results . . . demonstrated the ways in which three small districts, all in relatively isolated regions of the country, could implement structured
an explicit curricula and promote strong achievement gains that persisted through the early elementary years. I suggest that the example of these districts and the dedication and hard work of their teachers and administrators can begin to answer the call of Arnold and colleagues (2008) for ways to help rural schools improve teachers' "pedagogical skills in ways that have the greatest impact on student achievement." (Stockard, 2011b, p. 16).
This statement seems to be precisely the opposite of a portrayal of "lack" that Eppley describes. The results of the paper, including analyses that controlled for individual teacher characteristics, illustrated the power and strength of rural teachers and the ways in which they can do even better in helping their students, many of whom live in poverty and come from non-English speaking backgrounds. Again, Eppley's characterizations of my work and the article appear to have come from whole cloth.
Misrepresentations of the Scholarly Literature
Eppley's misrepresentations of the scholarly literature are also disturbing and certainly not worthy of a journal such as the Journal of Research in Rural Education. For instance, her description of Direct Instruction's academic foundations and related research bears virtually no resemblance to the extant scholarly literature, perhaps because she has relied on only secondary sources, websites, and polemical tracts for her information. Eppley states that "Project Follow Through is the research upon which Direct Instruction is based (Eppley, p. 2,). In actuality, the Direct Instruction curricula are based on a broad theoretical and empirical literature regarding the ways in which children learn and scientific studies of the most effective ways to transmit information. Each curricular element is field tested and revised based on these field tests (see Engelmann and Carnine, 1982b, 2011; Huitt, Monetti, & Hummel, 2009).
Project Follow Through did not provide the basis for the program's development. Instead, it was a very large empirical study of the relative efficacy of 21 different curricula, involving tens of thousands of students from communities in all parts of the United States. Extensive analyses of the results by independent researchers found that students receiving Direct Instruction had better academic (both basic skills and higher order cognitive skills) and affective outcomes (e.g. self esteem). No other program had such positive results (Becker, 1977, 1978; Kennedy, 1978; Stebbins, St. Pierre, Proper, Anderson, & Cerva, 1977; Watkins, 1995-6, 1997). Over the ensuing decades there have been many additional studies of Direct Instruction. The most succinct and recent summary comes from John Hattie's meta-analysis of meta-analyses, where he summarized the results of four meta-analyses that included Direct Instruction. His work incorporated 304 studies, 597 effects and over 42,000 students. He found that the average effect size associated with DI, when compared with other curricula, was .59 and noted that the positive results were "similar for regular (d=.99) and special education and lower ability students (d=0.86), [and] similar for the more low-level word-attack (d=.64) and also for high-level comprehension (d=.54)" (Hattie, 2009, p. 206-207; see Adams & Engelmann,1996; American Federation of Teachers, 1998; Beck & McCaslin, 1978; and Herman et al., 1999 for other meta-analyses incorporating Direct Instruction.) These effect sizes were substantially larger than those for any other curriculum that Hattie studied.
Eppley reserves some of her most vehement criticisms for the tightly structured nature of the Direct Instruction curriculum, using the phrase "pedagogy of erasure," defined as "the unintentional practice of erasing cultural identity through neglect, by not noticing and engaging the cultural presence of the other" (Eppley, p. 3 , citing Hicks, 1994). Although this discussion is somewhat obtuse, there appear to be three separate thoughts, which, again, are not supported by empirical evidence. First, Eppley appears to object to the "standardized" nature of the curriculum. As she puts it, "In order to erase difference, the standardized literacy instruction [Reading Mastery] is targeted toward a lowest common denominator, defined in this case as decoding speed and accuracy" (Eppley, p. 8). This statement appears to confuse one of the dependent measures used (see discussion below) with the curricular content. In fact, Reading Mastery includes extensive instruction related to both comprehension and fluency and, as noted immediately above, results in better outcomes on both of these measures. The program provides frequent readjustments of the pace of instruction and allows regrouping of students based on rate of skill acquisition. Because students can move more quickly through the curriculum they begin to study advanced materials sooner than they otherwise would, thus resulting in the higher comprehension scores noted above as well as the higher scores on state assessments, such as those reported in my article. (See Stockard, forthcoming, for other extensive analyses of state assessment scores.).
Second, Eppley states, "Everyone in the Reading Mastery classrooms gets the same sub-par instruction, but not everyone is unlucky enough to get Direct Instruction. It is instruction designed for the poor and at risk" (2011, p. 3). With these sentences she seems to be saying that Reading Mastery is inferior to other curricular programs and/or that it is only useful for poor or at risk students. She also seems to imply that it is fine for the poor and at risk to be "unlucky" and have this instruction. Of course, the literature cited above contradicts the notion that Reading Mastery instruction is "sub-par;" in fact, just the opposite is true. In addition, numerous studies indicate that all students - not just the poor and at risk, but also gifted and high achieving students - have stronger learning gains with the curriculum (e.g., Carnine & Kameenui, 1992; Engelmann & Carnine, 1982a; Fielding, Kameenui, & Gersten, 1983; Ginn, Keel, & Fredrick, 2002). Most important, results from numerous sites show that students from low income backgrounds who study with Reading Mastery and other Direct Instruction programs are more likely to develop higher order thinking skills and to catch up with their disadvantaged peers (Becker & Gersten, 1982; Stockard, 2010; Gersten & Keating, 1987; Gersten, Keating, & Becker, 1988; Meyer, 1984; Meyer, Gersten, & Gutkin, 1983).
Third, Eppley sees "the kind of teaching required by Reading Mastery as an example of an "'assimilative cultural pattern' that has the potential to over-determine or restrict possibilities for people and places" and that the curricula "erase the relevant knowledge the children bring to their classrooms by substituting it for homogenized knowledge sanctioned by a distant authority" (2011, p. 3). There is no indication that Eppley actually examined the curriculum or has any empirical evidence that it does not link to students' day-to-day knowledge - whether they are in rural or urban environments.
Eppley's lack of knowledge of the curriculum or of the dependent measures that were used is illustrated in other statements. One of the most important to address involves the notion that Direct Instruction teachers are "deskilled." In fact, a central point of my article was precisely the opposite. As stated, there,
The literature increasingly recognizes that teaching is a highly technical and involved process, and that training and support are crucial for developing and honing excellent instructional skills.Such support may be especially important for systematic and explicit curricula such as RM, which involve a broad array of behaviors and actions for complete implementation" (Stockard, 2011b, p. 3).
The statistical results confirmed this, with the strongest results occurring when the teachers had "fully learned the curriculum" (p. 15). The empirical evidence indicates that Direct Instruction is not deskilling, but instead helps teachers become much more skilled and proficient (Engelmann & Engelmann, 2004; Stockard, 2011a). Data regarding teacher attitudes indicate that this contributes to positive attitudes regarding teaching and the curriculum, with teachers believing that it helps them become more proficient and effective (e.g. Ogletree & DiPasalegne, 1975; Proctor, 1989; Schug, Tarver, & Western, 2001).
Eppley's characterization of the DIBELS assessment system also bears little resemblance to the extant literature. She implies that DIBELS measures only assess students' ability to decode nonsense words. Instead, the DIBELS assessments are carefully developed instruments, which can be quickly administered and capture students' reading skills as appropriate for their grade level. Thus, for the beginning levels (kindergarten and early first grade) nonsense words are used because they match the early learning of phonics. However, after that point grade level reading passages are used. In addition, contrary to Eppley's statement, a well-developed body of literature has found substantial correlations of DIBELS scores and other measures of reading achievement including comprehension (Fuchs, et al. 2001; Good et al. 2001). Interestingly, her fascination with "nonsense words" also appears in her discussion of the "pedagogy of erasure" where she describes Reading Mastery as involving "lots of practice in decoding nonsense words" (2011, p. 9), a statement with absolutely no basis in fact.
Finally, Eppley suggested that I made a "methodological error" by comparing "something with nothing" (p. 6). She is correct that we lack full details on the alternative curricula that were used. This reflects the fact that data were combined from three different districts and involved archival and longitudinal information. Still, to say that the comparison was "with nothing" is patent nonsense. If the comparison were truly "nothing," then there would have been no reading instruction occurring at all. Clearly that did not happen. Standard research practice with the cohort control group design is to compare a new program with the previous one (Cook and Campbell, 1979, pp. 126-133; Shadish, Cook, & Campbell, 2002, p. 137; Campbell & Stanley, 1963, pp. 57-60), and that was what was done.
Much of Eppley's discussion involves philosophical statements and broad generalizations. For instance, one of her comments involves the dismissal of "basic skills." She suggests that learning to read with Reading Mastery "indoctrinates children with the skills and dispositions needed for unskilled labor rather than preparing them to be engaged citizens of their rural communities, region, nation, and world." She then goes on to suggest that Direct Instruction involves an "emphasis on workforce training" and that the "Reading Mastery skill set is not basic, but inadequate and thereby reinforces the inequity for which the scripted instruction was purported to alleviate in the first place" (2011, p. 4). In this paragraph, Eppley seems to be saying that because Reading Mastery helps children develop basic skills, this is somehow detrimental to their future.
Basic skills are absolutely necessary for the development of higher order thinking skills. Students who do not possess these abilities fall further and further behind their peers. Moreover, as noted above, a very large literature, as well as the results of my paper with the state assessment scores, showed that students studying with Reading Mastery were more likely than other students to have higher level comprehension skills. Obtaining the basic skills at earlier ages and more quickly allows them to make greater progress and be more likely to escape disadvantage (see Becker & Gersten, 1982; Stockard, 2010). Eppley seems to imply that helping children learn to their full potential reinforces inequity. In fact, however, just the opposite is true. Denying children the opportunity to develop their full intellectual potential through using inadequate teaching and curriculum is what perpetuates social inequalities and class privilege. The "let them eat cake" philosophy that seems to be espoused by Eppley can only serve to deny opportunities and exacerbate existing inequalities.
Eppley talks about "rural life" (2011, p. 8), but fails to provide any type of definition. Instead, the commentary seems to reflect an overly romanticized and unrealistic notion of rural environments. In fact, the days of the small yeoman farmer, who read Cicero and wrote poetry while contemplating the wonders of his environment, are long gone, if they ever existed. Such people may still, of course, live in rural areas. However, today's rural schools, such as those in my sample, often have substantially more students at risk, including larger proportions of children in poverty and those from minority backgrounds, than do other schools in the nation. For many people in rural America, including the parents of the children in my sample their "rural life" involves backbreaking work in food processing factories or in agricultural fields, living with the constant fear of unemployment, having limited English skills with which to interact with the community, living in sub-standard housing, and never having enough money to feed or clothe their children. To deny their children the very best education possible is, in my view, morally reprehensible.
This brings me to the final paragraph of Eppley's comments. In this paragraph, she implies that the use of Reading Mastery "may be harmful and perhaps even unethical" (2011, p. 4). Again, however, she gives no explanation of this aspersion, but instead suggests that its use reflects "ideology and power." It appears to me that unethical actions would be those that would deny students the most effective curriculum and schooling experiences possible. A standard tenet of medical practice and social research is the notion of "beneficence," or doing good. Most people would agree that helping schools do a better job of teaching children is beneficent. If Eppley can provide evidence of other approaches that are as effective as Reading Mastery and Direct Instruction, she should do so.
Adams, G. L., & Engelmann, S. (1996). Research on Direct Instruction: 25 years beyond DISTAR. Seattle, WA: Educational Achievement Systems.
American Federation of Teachers. (1998). Building on the best, learning from what works: Seven promising reading and language arts programs. Washington, D.C.: AFT.
Arnold, M. L., Newman, J. H., Gaddy, B. B., & Dean, C. B. (2005). A look at the condition of rural education research: Setting a direction for future research. Journal of Research in Rural Education, 20(6). Retrieved from http://jrre.psu.edu/articles/20-6.pdf.
Beck, I. L., & McCaslin, E .S. (1978). An analysis of dimensions that affect the development of code-breaking ability in eight beginning reading programs (LRDC Report No. 1978/6). University of Pittsburgh.
Becker, W. C. (1977). Teaching reading and language to the disadvantaged-What we have learned from field research. Harvard Educational Review, 47, 518-543.
Becker, W. C. (1978). National evaluation of Follow Through: Behavior-theory-based programs come out on top. Education and Urban Society, 10, 431-458.
Becker, W. C., & Gersten, R. (1982). A follow-up of Follow Through: The later effects of the Direct Instruction model on children in fifth and sixth grades. American Educational Research Journal, 19, 75-92.
Campbell, D. T., & Stanley, J. C. (1963). Experimental and quasi-experimental designs for research. Chicago, IL: Rand McNally.
Carnine, D., & Kameenui, E. J. (1992). Higher order thinking: Designing curriculum for mainstreamed students. Austin, TX: Pro-ed.
Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design and analysis issues for field settings. Chicago, IL: Rand McNally.
Engelmann, S., Becker, W. C., & Carnine, D. (1988). Direct Instruction Follow Through model: Design and outcomes. Education and Treatment of Children, 11, 303-317.
Engelmann, S., & Carnine, D. (1982a). DI Outcomes with middle-class second graders, ADI News 1, 2-5.
Engelmann, S. E., & Carnine, D. (1982b). Theory of instruction: Principles and applications. New York: Irvington Publishers.
Engelmann, S., & Carnine, D. (2011). Could John Stuart Mill have saved our schools? Verona, WI: Full Court Press.
Engelmann, S. E., & Engelmann, K. E. (2004). Impediments to scaling up effective comprehensive school reform models. In T. K. Glennan, Jr., S. J. Bodilly, J. R. Galegher, K. A. Kerr (Eds.). Expanding the reach of education reforms: Perspectives from leaders in the scale-up of educational interventions (p. 107-133). Santa Monica, CA: Rand.
Fielding, G. D., Kameenui, E., & Gersten, R. (1983). A comparison of an inquiry and a direct instruction approach to teaching legal concepts and applications to secondary school students. Journal of Educational Research, 76, 287-293.
Fuchs, L. S., Fuchs, D., Hosp, M. K., & Jenkins, J. R. (2001). Oral reading fluency as an indicator of reading compentence: A theoretical, empirical, and historical analysis. Scientific Studies of Reading, 5(3), 239-256.
Gersten, R., & Keating, T. (1987). Long-term benefits from Direct Instruction. Educational Leadership, 44(6), 28-31.
Gersten, R., Keating, T., & Becker, W. C. (1988). The continued impact of the Direct Instruction model: Longitudinal studies of Follow Through students. Education and Treatment of Children, 11(4), 318-327.
Ginn, P. V., Keel, M. C., & Fredrick, L. D. (2002). Using reasoning and writing with gifted fifth-grade students. Journal of Direct Instruction, 2, 41-47.
Good, R. H., Simmons, D. C., & Kame'enui, E. J. (2001). The importance and decision-making utility of a continuum of fluency-based indicators of foundational reading skills for third-grade high-stakes outcomes. Scientific Studies of Reading, 5(3), 257-288.
Hattie, J. (2009). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. London and New York: Routledge.
Herman, R., Aladjam, D., McMahon, P., Masem, E., Mulligan, I., Smith, O., . . . Woodruff, D. (1999). An educator's guide to schoolwide reform. Washington, D.C.: American Institutes for Research.
Huitt, W. G., Monetti, D. M., & Hummel, J. H. (2009). Direct approach to instruction. In C. Reigeluth & A. Carr-Chellman (Eds.), Instructional-Design Theories and Models: Volume III, Building a Common Knowledge Base (p. 73-98). Mahwah, NJ: Lawrence Erlbaum.
Kennedy, M. M. (1978). Findings from the Follow Through planned variation study. Washington, D.C: U.S. Office of Education.
Meyer, L. A. (1984). Long-term academic effects of the direct instruction project follow-through. The Elementary School Journal, 84(4), 380-394.
Meyer, L., Gersten, R., & Gutkin, J. (1983). Direct Instruction: A project follow-through success story in an inner-city school. Elementary School Journal, 84, 241-252.
Ogletree, E. J., & DiPasalegne, R. W. (1975). Inner-city teachers evaluate DISTAR. Reading Teacher, 28, 633-637.
Proctor, T. J. (1989). Attitudes toward Direction Instruction. Teacher Education and Special Education, 12, 40-45.
Schug, Tarver, & Western (2001). Direct Instruction and the teaching of early reading, Wisconsin Policy Research Institute Report, 14, 1-29.
Shadish, W. R.,. Cook, T. D., & Campbell, D.T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston, MA: Houghton Mifflin.
Stebbins, L. B., St. Pierre, R. G., Proper, E. C., Anderson, R. B., & Cerva, T. R. (1977). Education as experimentation: A planned variation model (Vol IV-A). Cambridge, MA: Abt Associates.
Stockard, J. (2010). Promoting reading achievement and countering the "fourth-grade slump": The impact of Direct Instruction on reading achievement in fifth grade, Journal of Education for Students Placed at Risk, 15, 218-240.
Stockard, J. (2011a). Direct Instruction and first grade reading achievement: The role of technical support and time of implementation, Journal of Direct Instruction, 11, 31-50.
Stockard, J. (2011b). Increasing reading skills in rural districts: A case study of three schools, Journal of Research in Rural Education, 26, (8), 1-19.
Stockard, J. (forthcoming). Merging the accountability and scientific research requirements of the No Child Left Behind Act: Using cohort control groups," Quality and Quantity: International Journal of Methodology.
Watkins, C. L. (1995-6) Follow Through: Why didn't we? Effective School Practices, 15(1).
Watkins, C. L. (1997) Project Follow Through: A case study of contingencies influencing instructional practices of the educational establishment. Cambridge, MA: Cambridge Center for Behavioral Studies.
University of Oregon
Correspondence regarding this manuscript should be addressed to: Dr. Jean Stockard, Director of Research, National Institute for Direct Instruction and Professor Emerita, University of Oregon, P. O. Box 11248, 805 Lincoln Street, Eugene, OR 97440; Email: email@example.com.