Conducting Large-Scale Research in Adult ESL: Challenges and Approaches for the Explicit Literacy Impact Study

Background

According to the most recent statistics from the U.S. Department of Education (2004), more than 40% of the over 3 million students in the federally funded adult education program are English as a second language (ESL) students. These students—most of whom are immigrants and refugees— represent a wide range of nationalities and cultural backgrounds. Many students have had limited educational opportunities and have not developed the basic foundational reading and writing skills in their native language that are necessary for acquiring English literacy and language skills. They struggle with the dual challenge of acquiring literacy skills as they learn to communicate in English. Adult basic education (ABE) and ESL programs assist students in their efforts to acquire literacy and language skills by providing instruction through local education agencies, community colleges, and community-based organizations, but are often hampered with chronic under- and erratic-funding, lack of institutional support, and shifting student populations.

Teachers in these classes face their own challenges as they strive to provide effective instruction. Often poorly paid and working part-time, they usually receive little or no professional development and teach in crowded classrooms with limited resources. Furthermore, the open enrollment policies of many programs, along with the relatively low retention and attendance of adult ESL students, interfere with providing the continuous level of instruction students need to acquire literacy and language skills. As a result, instruction in community-based adult ESL classes is often eclectic and focused on functional or life skill topics without a clear curriculum scope and sequence.

Unfortunately, in ABE and adult ESL, there is very little research to help guide instruction or curriculum development. To help address the lack of research-based knowledge in adult ESL instruction for low literacy populations, the National Center for Education Evaluation and Regional Assistance of the Institute of Education Sciences (IES) funded a task to identify promising adult ESL literacy interventions and to design a rigorous large-scale study to test the impact of the intervention. In order to identify a promising intervention, Condelli and Wrigley (2004) first conducted a comprehensive review of all research studies on the effects of ABE and adult ESL instruction. They identified several widely used instructional approaches, but concluded that the research base was too limited and the studies’ methodologies were not sufficiently rigorous to warrant a large-scale, randomized study of any of the approaches identified in the research. Two separate panels of experts, one composed mostly of reading researchers and methodologists and another composed of adult ESL practitioners, concurred with this conclusion. However, after considering the broader research on literacy development, including research on elementary school children (e.g., National Reading Panel, 2000), the expert panels agreed that explicit literacy instruction was a promising intervention that would be valuable to study with low-literate adult ESL learners.

The Adult ESL Explicit Literacy Impact Study

To follow up on the results of the design task, IES is now funding an evaluation of the impact of explicit literacy instruction on adult ESL learners. AIR is leading the study, along with its partners at the Lewin Group, Berkeley Policy Associates, Mathematica Policy Research, and the Educational Testing Service. The purpose of the study is to test the impact of a curriculum-based explicit literacy intervention for low-literacy adult ESL learners. More specifically, the goal is to answer the following research questions:

• How effective is the explicit literacy intervention in improving the English reading, writing, and speaking skills of low-literate adult ESL learners?
• Does the explicit literacy intervention have different effects on subgroups of adult ESL learners?
• How well do instructors implement the intervention, and how does this affect learning?

To answer these questions, the study will employ an experimental design in 10 adult ESL programs, with teachers and students randomly assigned to condition within each program. Forty teachers (4 in each program) will be randomly assigned to teach either the explicit literacy intervention class or a regular ESL class offered by the program. Approximately 1,800 low-literacy (e.g., about third- to sixth-grade equivalent) students will also be randomly assigned to either an explicit literacy class or a regular ESL class.

After the experimental instruction is completed (about 16 weeks after enrollment and 60 hours of instruction with the explicit literacy intervention), both groups of students will be assessed on measures of reading, writing, and speaking. The design also includes an optional one-year followup, during which students would be assessed again a year after exit to determine the longer-term effects of the intervention.

This paper presents the research base which underlies the Adult ESL Explicit Literacy Impact Study, as well as some of the challenges inherent in community-based adult education programs to conducting a large-scale rigorous study, and the approaches the study team is taking to address those challenges.