Florida's Test-Based Promotion Policy. How Does Retention Affect Students' Academic Performance?

Table of Contents

List of Tables and Figures

List of Abbreviations

1. Introduction

2. Specification of Florida’s Retention Policy
2.1 Provision of Remedial Services
2.2 Promotion through Exemptions
2.3 Developmental Scale Scores

3. Evaluation Method: Fuzzy Regression Discontinuity Design
3.1 Application of the Fuzzy RDD in Florida
3.2 Graphical Illustration
3.3 Bandwidth Choice
3.4 Exclusion of the Manipulation of Test Scores

4. Estimation Setup and Treatment Effect
4.1 Linear Probability Model and Two Stage Least Squares
4.2 Composition of the Treatment Effect
4.2.1 Summer School Attendance
4.2.2 Assignment to a High-Quality Teacher
4.2.3 Incentive Effects
4.3 Comparison: Chicago’s Accountability Policy

5. Conclusion

References

List of Tables and Figures

Table 1: FCAT Reading Scale Scores and converted DSS

Figure 1: Discontinuity in the fraction of retained students

Figure 2: No Discontinuity before the introduction of the policy

List of Abbreviations

Abbildung in dieser Leseprobe nicht enthalten

1. Introduction

This paper aims to provide an overview of the outcomes of the test-based promotion policy, for the federal state Florida, introduced in 2002. In Florida, the policy is implemented via the Florida Comprehensive Assessment Test. The main task is to draw conclusions about the success of these policy interventions for students over time, as well as a comparison to the former social promotion policy. Success in this case is measured by the student’s academic performance. Can students improve their abilities from an additional year of schooling or is it a waste of time to attend the same class a second time? The views on this topic are widely mixed. Conservative beliefs based on research in the 1980s and 1990s express that the drawbacks of retention, like the deterioration of student’s social environment and exposedness to bullying, outweigh academic performance improvements. But more recent studies show that once applying proper research designs for evaluating retention policies, retention indeed does increase student’s outcomes over short and mid-term time periods.

The main task of the introduction of a test-based promotion policy is to end social promotion. Social promotion grants students to promote to the next grade regardless of the material they have learned in class, but focusing on individual characteristics like maturity. However, if a student’s overall performance is worse compared to other classmates, even under social promotion the teacher would draw the line to hold this student back. But this decision is mainly based on the subjective assessment of the teacher or whoever is responsible for this task. This is another factor why former research on this topic led to conclusions, that grade retention is more a harm than a benefit to students. But with the application of a standardized test, equal for all students, the assessment of the students’ performance is based on test scores rather than on subjective impressions of the assessor.

The main underlying study of this paper was written by Schwerdt and West (2013) and is focusing on a time span from 2002 to 2008. Thus, students between the third and ninth grade were observed and one major advantage in this study compared to studies without quasiexperimental research designs, is the true discontinuity in the probability of retention, generated by the test-based promotion policy of Florida. On the one hand it seems important to apply proper program evaluation methods, but these methods will only work if the setting, the testbased promotion policy, is carried out properly.

The first chapter will deal with specific features of Florida’s test-based promotion policy, followed by the research design that was used by Schwerdt and West (2013) to evaluate the program. Afterwards, estimation results and the treatment effect are discussed, including results of Chicago’s test-based promotion policy to compare the findings of different studies.

2. Specification of Florida’s Retention Policy

To get a better understanding of the whole test-based promotion policy topic, initially it will be discussed, what makes Florida’s retention policy different from other promotion policies in the United States, respectively which policy interventions are included for retained students.

2.1 Provision of Remedial Services

If a student is retained, consequently he is assigned to the treatment group, or in other words, the student will be subject to the test based promotion policy. The question now is, what this treatment or the treatment effect consists of. One view on this topic could lead to a conclusion that a retained student has to pass through the third grade again. Rehearse the topics addressed in the third grade curriculum one additional time, with the expectation that the student is able to deal with the covered academic content afterwards and scores at least on achievement level two in the FCAT after the retained year. The FCAT measures the performance of the students on five achievement levels ranging from little (level one) to high (level five) success (FDOE 2008).

But do students really increase their performance, just because of the fact that they are retained and thus have to keep in mind that they have to increase their academic effort, to get promoted to grade four? Certainly, even parents, who paid very less attention to their children’s school performance will now be more involved, at least through incentives from the responsible teacher, but there is a major need for services provided by the schools, to increase the student’s abilities. Those students, who show a great lack of academic performance need strong guidance during their retained year, to maximize their achievements in school.

Accordingly, Florida provides remedial services to retained students during the retained year, which is a common and needed mechanism for policy interventions. Namely these services consist of the opportunity of summer school participation in the student’s district, the assignment to a high performing teacher during the retained year, the development of an individual academic improvement plan for each student and furthermore the supply of intensive additional reading including 90 uninterrupted minutes per day, which is mandatory since the 2004/2005 school year. Furthermore, schools are also obliged to develop academic improvement plans that address the specific needs of these students (Schwerdt/West 2013).

These influence factors on the academic performance of the students depict the treatment effect. To which extent these services e.g. summer school participation or a high-quality teacher affect the treatment effect will be discussed in Chapter 4.2.

2.2 Promotion through Exemptions

Besides the five achievement levels of the FCAT, which are defined by the respective achieved test score, various exemptions exists, that give students the opportunity to promote although they failed in the FCAT. These exemptions should be applied in a sincere manner and the responsibility rests with the teachers or the principals. Exemptions, for instance, can be applied to limited English proficiency students, or students with disabilities, who are not appropriate for testing or have received prior intensive instructions (Greene/Winters 2007).

The 2014 data set¹ of District Scores for third grade reading achievement shows, that the Florida School for the Deaf and Blind has an 81% share of students scoring at achievement level one. Holding these students back for an additional year would not lead to adequate policy outcomes and therefore exemptions for students, who receive intensive instructions would apply. Thus, these exemptions are serving a meaningful purpose.

On the contrary, Greene and Winters (2009) suggest, that there might be the existence of institutional racism among students by assigning these exemptions. In their study, the share of Afro-American and Hispanic students, who did not pass the exam and were not considered for exemptions, is distinctly higher than the share of White students. This means White students with similar test scores below the threshold were rather assigned to an exemption category than students of a different ethnicity.

The next arising question is, whether promoted students benefited from the exemptions in academic regard. Promoted students who are unlikely to raise their effort in the next grade will not serve the exemption purpose. A suspicious fact in the Greene and Winters (2007, 2009) study is, that beside the regular exemption categories, students were promoted without a specific reason, which is called “no code listed”. This kind of arbitrariness, resulting through exemptions, implicates that retained students could have had higher scores below the cut-off than an exempted student, but were randomly or consciously excluded from the exemption process. Perhaps this is due to good parent-teacher relationships. If parents are able to convince teachers that their child should not be retained, because they will raise their own effort to be more involved in their children’s education process, the teacher might comply.

This thoughts should not query the overall exemption possibility, because certainly it is advised to promote students with disabilities, but it remains that a significantly high share of exempted students were granted an exemption without good reason.

2.3 Developmental Scale Scores

Another feature of Florida’s test-based promotion policy is the Developmental Scale Score. This score should provide an overview over a student’s academic progress in subsequent grades. Thus, not only the FCAT score is taken into account, but all scores a student reached, for instance in the reading exam, are converted into DSS providing an measurement of the overall reading achievement until graduation (FDOE 2012). For the 2008 FCAT a student who scored on achievement level one in grade 3 would have reached a reading score between 100 and 258 points. Table 1 shows the conversion of the FCAT reading test score into DSS, resulting in a span between 86 and 1045 points (FDOE 2008). This feature shall support the comparison of students across grades.

Abbildung in dieser Leseprobe nicht enthalten

Table 1: FCAT Reading Scale Scores and converted DSS

By taking a closer look at the DSS in subsequent grades, it appeals that in some grades average reading gains on the DSS are larger than in others. This is due to different curricula in adjacent grades and the importance of gaining reading knowledge in lower grades. Schwerdt and West (2013, p.24) provide a graphical illustration on DSS, depicting a very steep slow in reading gains in grade 4, which makes sense, because students in subsequent grades will have to deal with more complex topics and therefore fundamental reading skills should be acquired in early grades.

The DSS will have major use for the estimation setup in the Schwerdt and West (2013) study, because it provides a necessary foundation to compare DSS of those students, who were not retained in grade 3, to those who were subject to the test-based promotion policy. Again, this is another important policy specification to get rid of subjectivity in assessing a student’s performance in school. DSS are also provided for the FCAT in math, but since this test is not decisive for retention, it is just mentioned for completeness reasons at this point.

[...]

¹ The data set is available from: http://fcat.fldoe.org/mediapacket/2014/xls/GR03_Rdis_2014_v2.xls.