Technical Notes

 

Standard Conventions used in the 2006-07 ABCs Analyses

95R – Percent tested below 95% for the 2006-07 ABCs, a percentage of students tested was computed by combining information from EOG reading, EOG mathematics, EOC and alternate assessments across all grades in a school. The percentage must be greater than or equal to 95, when rounded to the nearest whole number. (To determine the 95% participation rate for meeting Adequate Yearly Progress (AYP) per the No Child Left Behind legislation, the cohort of 10th grade students on the first day of spring testing for the school were analyzed to determine the number of students who had taken the required tests.)  Schools identified as having fallen below the 95% threshold for the ABCs were asked to justify their rate. Their explanations were reviewed at DPI and if rejected, the school was assigned a 95R status. This assigned status means that the school was in violation of the rule and ineligible to receive incentive awards or recognition (other than low performing). A school in violation for two consecutive years may be identified as low-performing by the State Board of Education. 

 

Full Academic Year

Students must have been in membership in a school for at least 140 days as of the first day of spring testing in order for their test scores to be included in the growth calculations or AYP.  By contrast for ABCs in block scheduled high schools, the student must be in membership 70 days in the semester the course is taught.

 

Medical Exclusions

In compliance with NCLB and in light of the full availability of alternate assessments, only exclusions for serious medical emergencies and/or conditions were allowed in 2005-06. Examples included students who were (1) in the final stages of terminal or degenerative illnesses, (2) comatose, or (3) receiving extensive short-term medical treatment due to a medical emergency.   In response to a request from the student’s LEA the Director of Accountability Services provides a written statement of the decision to the LEA; any exception granted is limited to the testing period for the specific test(s) for which it was requested and does not carry forward to future test administrations, unless noted in the decision.  In the rare case that a student was still administered an assessment, the score from the assessment was used.

 

Algebra I Scores in the Performance Composite

Algebra I scores for current ninth graders who took Algebra I prior to grade 9 were included in the performance composite for the high school where they were currently enrolled. Algebra I scores of students in grades 6, 7, or 8 during the current school year were included in the K-8 performance composite of the middle school where they were currently enrolled. Algebra I scores of students currently enrolled in grade 10 in a senior high school (Grades 10-12) who took Algebra I while in earlier grades were included in the performance composite of the senior high school.

 

Confidence Interval Applied to the Performance Composite to Identify Low-Performing Schools

The performance composite is the percent of students’ scores at or above grade level (i.e., in Achievement Levels III or IV) on end-of-grade (and Computer Skills in grade 8 where applicable) and/or end-of-course tests. The performance composite is computed by adding all scores at or above Achievement Level III on each of the tests, and then dividing the sum by the total number of valid scores on the tests. If a school did not make expected growth and its performance composite was significantly less than 50, the school is given the status of low-performing.


The confidence interval is a way of taking into account the precision of the performance composite. By applying the confidence interval, the likelihood of incorrectly assigning a low-performing status to a school that does not deserve it is decreased while the likelihood of correctly assigning a status to schools is not hindered. The confidence interval itself will be narrow or wide depending on the size of the school and the variation of the student scores in the school. In general, the confidence interval is narrower when the number of students is larger, or the scores are more homogeneous; the confidence interval is wider when the number of students is smaller, or the scores are less homogeneous.

 

This means that a potentially low-performing school may have a performance composite that is considerably below 50% but when the confidence interval is applied (correcting for the precision of the performance composite), the school is not considered low-performing (because the confidence interval for that school is wide i.e., there is less confidence in the value of the performance composite). This situation would likely be true for a school that has few students or has wide variation in test scores. It is also possible for a school to have a performance composite that is fairly close to 50% and is considered low-performing because the confidence interval for that school is very narrow (i.e., there is high confidence in the performance composite). This situation would likely be true for a school that has a large number of students or students all have about the same test score.

 

As long as the value, 50, lies within or on the upper boundary of the confidence interval for an observed performance composite, then the performance composite is not significantly less than 50 and hence the school is not classified as low-performing.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Preliminary Analyses of the ABCs Model for 2005-06

The technical notes present preliminary analyses of the proposed ABCs model for 2005-06 using the c-scale. Eighteen tables are included and many compare findings from the proposed model and the original ABCs model.  Please note that academic change is calculated using either 0.9 or 0.8 in all table unless otherwise noted.

 

Table 1. Standard Setting Years, Means and Standard Deviations for End-of-Grade (EOG)  C-Scale Computation

EOG

Standard Setting Year

Mean

Standard Deviation

Reading (1st Edition)

 

 

 

Grade 3 Pretest

1997

137.7

8.57

Grade 3

1997

146.9

9.29

Grade 4

1997

150.3

9.34

Grade 5

1997

160.0

9.62

Grade 6

1997

156.7

9.61

Grade 7

1997

159.9

8.50

Grade 8

1997

163.1

8.04

Reading (2nd Edition)

 

 

 

Grade 3 Pretest

2003

238.7

9.94

Grade 3

2003

247.9

9.06

Grade 4

2003

252.3

8.68

Grade 5

2003

256.9

8.03

Grade 6

2003

258.7

8.55

Grade 7

2003

261.1

9.06

Grade 8

2003

263.9

9.05

Mathematics (1st Edition)

 

 

 

Grade 3 Pretest

1997

130.9

7.96

Grade 3

1997

142.9

11.09

Grade 4

1997

152.3

10.28

Grade 5

1997

159.3

9.99

Grade 6

1997

164.8

10.84

Grade 7

1997

170.8

10.58

Grade 8

1997

174.2

11.96

Mathematics (2nd Edition)

 

 

 

Grade 3 Pretest

2001

236.1

8.10

Grade 3

2001

250.6

7.75

Grade 4

2001

255.8

8.32

Grade 5

2001

260.0

9.62

Grade 6

2001

263.2

9.91

Grade 7

2001

267.1

10.63

Grade 8

2001

270.0

10.95


 

2nd Edition Math 2005

Special Transition Conversion Used when Comparing with 3rd Edition

 

 

 

Grade 3 Pretest

2005

237.9

7.7

Grade 3

2005

253.1

7.0

Grade 4

2005

258.6

8.0

Grade 5

2005

262.0

9.6

Grade 6

2005

266.1

9.6

Grade 7

2005

268.8

11.0

Grade 8

2005

272.1

10.9

3rd Edition Math 2006

 

 

 

Grade 3 Pretest

2006

329.7

11.35

Grade 3

2006

343.20

9.70

Grade 4

2006

348.90

9.46

Grade 5

2006

353.74

9.25

Grade 6

2006

354.91

9.70

Grade 7

2006

357.76

9.65

Grade 8

2006

359.15

9.21

 

All values are rounded. Full precision was used for actual calculations.

 

Table 2. Standard Setting Years, Means and Standard Deviations for End-of-Course (EOC) C-Scale Computation

 

EOC

Standard Setting Year

Mean

Standard Deviation

Algebra I

1994

55.1

9.12

 

2001

61.1

9.31

 

2006

63.3

10.1

 

2007

150.3

8.9

Algebra II

1997

58.5

10.26

 

2001

63.8

9.90

 

2007

150.2

9.3

Biology

1995

55.5

8.67

 

2002

57.9

7.61

 

2007

57.3

7.47

Chemistry

1997

56.8

8.53

 

2002

60.0

8.16

Civics and Economics

2006

150.2

9.19

 

2007

150.5

9.04

English I

2003

57.7

7.63

 

2007

150.36

8.93

Geometry

2001

59.8

8.85

 

2007

150.2

9.27

Physics

1997

56.5

8.65

 

2002

60.7

9.24

Physical Science

1997

54.0

9.41

 

2002

55.8

7.90

U.S. History

2006

150.0

9.14

 

 

 

Table 3. N Counts for EOG Proposed Formulas

 

Reading

Mathematics

Grade 3

545,120

545,862

Grade 4

534,515

536,147

Grade 5

587,547

589,558

Grade 6

525,760

453,806

Grade 7

450,087

450,838

Grade 8

397,382

397,852

 

•           The N counts for grades 3-8 for Reading and Mathematics correspond to the EOG correlations for the proposed formulas in Tables 4 and 5. 

 

Table 4. EOG Correlations Between Predicted Values and Actual Performance

Proposed Formulas

 

Original Formulas

 

Reading

Mathematics 

 

 

Reading

Mathematics

Grade 3

0.71

0.75

 

Grade 3

0.76

0.79

Grade 4

0.81

0.82

 

Grade 4

0.80

0.82

Grade 5

0.84

0.86

 

Grade 5

0.81

0.84

Grade 6

0.85

0.86

 

Grade 6

0.82

0.85

Grade 7

0.85

0.87

 

Grade 7

0.83

0.87

Grade 8

0.85

0.88

 

Grade 8

0.83

0.87

 

 

All proposed formulas’ correlations, except grade 3, are at least as high as those from the current method of calculation. This suggests comparable prediction accuracy.

 

Table 5. EOG Correlations Between Predicted Values and Residuals* 

Proposed Formulas

 

Reading

Mathematics

Grade 3

-0.13

-0.13

Grade 4

-0.04

-0.02

Grade 5

-0.19

-0.003

Grade 6

0.01

-0.04

Grade 7

0.04

0.06

Grade 8

-0.05

-0.02

 

Original Formulas

 

Reading

Mathematics

Grade 3

-0.15

-0.30

Grade 4

-0.03

-0.25

Grade 5

-0.30

-0.14

Grade 6

0.07

0.35

Grade 7

-0.15

0.06

Grade 8

-0.15

0.19

 

   All numbers are Pearson’s r.

*  A residual is the difference between the predicted value and the actual value. 


 

Table 6. N Counts and Equations for Proposed EOC Formulas

Algebra I

109,585

Grade 8 Mathematics EOG*0.8

Algebra II

265,165

Algebra I EOC*0.8

Biologya

 197,455

Grade 8 Reading EOG*0.8

Biologyb

 193,548

The average of English I EOC and grade 8 Reading EOG*0.9

Chemistry

84,620

Biology EOC*0.8

English I

280,400

Grade 8 Reading EOG*0.8

Geometrya

 172,713

Algebra I EOC*0.8

Geometryb

 149,943

The average of Algebra I EOC and grade 8 Mathematics EOG*0.9

Physical Science

110,722

Grade 8 Mathematics EOG*0.8

Physics

12,084

The average of Chemistry and Geometry EOCs*0.9

US History b

74,235

The average of English I and Biology EOC *0.92

US History a

75,174

Biology EOC * 0.82

Civics and Economics b

49,494

The average of English I and Biology EOC *0.92

Civics and Economics a

90,197

English I EOC * 0.82

 

a denotes only one predictor used. bdenotes two predictors used.

The N counts correspond to the EOC correlations for the proposed formulas in Tables 7 and 8. 

The predicted values for Algebra I and Physical Science are computed by multiplying the grade 8 Mathematics c-scale score by 0.8. 

The predicted value for Algebra II is computed by multiplying the Algebra I c-scale score by 0.8.

The predicted value for Biologya is computed by multiplying the grade 8 Reading c-scale score by 0.8, whereas the predicted value for Biologyb is computed by multiplying the average of the English c-scale score and the grade 8 Reading c-scale score by 0.9. 

The predicted value for Chemistry is computed by multiplying the Biology c-scale score by 0.8.

The predicted value for English I is computed by multiplying the grade 8 Reading c-scale score by 0.8.

The predicted value for Geometrya is computed by multiplying the Algebra I c-scale score by 0.8, whereas the predicted value for Geometryb is computed by multiplying the average of the Algebra I c-scale score and the grade 8 Mathematics c-scale score by 0.9.

The predicted value for Physics is computed by multiplying the average of the Chemistry  c-scale score and the Geometry c-scale score by 0.9.

 

Table 7. EOC Correlations Between Predicted Values and Actual Performance

Proposed Formulas                                                                 

Algebra I

0.78

Algebra II

0.73

Biologya

 0.73

Biologyb

0.78

Chemistry

0.71

English I

0.79

Geometrya

 0.77

Geometryb

 0.83

Physical Science

0.73

Physics

0.77

US Historyb

0.77

Civicsb and  Ecnomicsb

0.83

 

 

Original Formulas

Algebra I

0.75

Algebra II

0.76

Biology

0.03

Chemistry

0.76

English I

0.80

Geometry

0.82

Physical Science

0.72

Physics

0.72

US History a

0.75

Civicsa and

Economicsa

0.74

 

All numbers are Pearson’s r.

adenotes only one predictor used.

b denotes two predictors used.


Table 8. EOC Correlations Between Predicted Values and Residuals Proposed Formulas


 

Original Formulas

Algebra I

0.06

Algebra II

0.009

Biologya

 0.01

Biologyb

0.04

Chemistry

0.06

English I

-0.006

Geometrya

 0.02

Geometryb

 0.09

Physical Science

-0.11

Physics

0.02

 

New Formulas

Algebra I

-0.18

Algebra II

-0.28

Biology

-0.63

Chemistry

0.08

English I

-0.20

Geometry

-0.30

Physical Science

-0.40

Physics

-0.11

 

 


 

All numbers are Pearson’s r.

a denotes only one predictor used.

b denotes two predictors used.

            All correlations between predicted values and residuals are lower using the proposed formulas than the original formulas. A lower correlation of residuals suggests a weaker relationship between the predicted score and the amount of error in the prediction.  This implies less systematic bias in predicting student performance.

 

Table 9. Standard Error of Estimation (SEE) for EOG and EOC Predictions EOG EOC

 


Reading

Proposed Formulas

Original Formulas

Grade 3

0.663

0.635

Grade 4

0.571

0.579

Grade 5

0.530

0.505

Grade 6

0.513

0.569

Grade 7

0.503

0.438

Grade 8

0.489

0.439

 

 

 

Mathematics

Proposed Formulas

Original Formulas

Grade 3

0.565

0.564

Grade 4

0.530

0.640

Grade 5

0.444

0.432

Grade 6

0.489

0.604

Grade 7

0.469

0.467

Grade 8

0.446

0.506

 

 

Proposed Formulas

Original Formulas

Algebra I

0.620

0.774

Algebra II

0.699

0.704

Biologya

 0.685

1.370

Biologyb

 0.629

 

Chemistry

0.734

0.681

English I

0.586

0.657

Geometrya

 0.659

 

Geometryb

 0.596

0.630

Physical Science

0.634

1.083

Physics

0.922

1.284

 

 

 


 

a denotes only one predictor used. denotes two predictors used.

b denotes only one predictor used. denotes two predictors used.

 

 

 

Table 10. Percent of Students Meeting Expectations in the Lower 10% and 50% and Upper 50% and 10% Using the Proposed Formulas Compared to the Original Formulas

Reading

 

 

Proposed Formulas

 

 

 

Overall

Lower 10%

Lower 50%

Upper 50%

Upper 10%

Grade 3

46.6

53.1

51.8

41.7

30.4

Grade 4

47.5

45.5

50.4

44.7

39.7

Grade 5

49.9

64.3

56.6

43.4

35.1

Grade 6

60.4

60.1

58.4

62.4

59.6

Grade 7

58.6

55.4

56.5

60.8

57.7

Grade 8

52.8

58.5

54.5

51.1

48.6

 

 

 

Original Formulas

 

 

 

Overall

Lower 10%

Lower 50%

Upper 50%

Upper 10%

Grade 3

55.7

53.3

59.6

51.8

35.0

Grade 4

45.4

50.5

46.1

44.8

38.6

Grade 5

54.0

73.9

64.2

44.9

33.1

Grade 6

26.0

31.7

27.8

24.5

19.5

Grade 7

54.1

62.5

58.4

50.7

37.5

Grade 8

39.6

55.3

46.6

33.3

28.8

 

Mathematics

 

 

Proposed Formulas

 

 

 

Overall

Lower 10%

Lower 50%

Upper 50%

Upper 10%

Grade 3

56.4

65.4

59.6

53.5

44.8

Grade 4

57.1

59.3

57.0

57.2

56.6

Grade 5

54.3

59.7

53.5

55.1

58.7

Grade 6

65.2

70.4

64.5

66.0

64.2

Grade 7

57.2

63.9

54.9

59.8

68.4

Grade 8

60.6

64.7

60.0

61.3

60.0

 


 

 

 

Original Formulas

 

 

 

Overall

Lower 10%

Lower 50%

Upper 50%

Upper 10%

Grade 3

66.7

78.2

75.0

59.1

44.4

Grade 4

73.9

77.7

75.3

72.6

69.5

Grade 5

47.6

62.8

51.5

44.3

54.6

Grade 6

65.8

47.5

48.7

78.6

94.2

Grade 7

54.1

54.7

51.7

56.3

69.9

Grade 8

52.2

48.6

41.5

62.1

77.3

 

 

 

 

 

 

 

·         Using the proposed formulas, academic change is calculated by subtracting the predicted values from the posttests. The predicted values for grade 3 Reading and Mathematics are computed by multiplying 0.8 by the grade 3 pretest. The computation for grades 4-8 Reading and Mathematics is the product of 0.9 and the average of the two previous assessments (ATPA). This table shows the percent of students meeting expectations, where academic change is greater or equal to zero, at the lower 10%, lower 50%, upper 50% and upper 10% percentile of predicted values.

 

Table 11. Percent of Students Meeting Expectations by Quartiles Using the Proposed Formulas Compared to the Original Formulas

Reading

 

Proposed Formulas

 

 

1st Quartile

2nd Quartile

3rd Quartile

4th Quartile

Grade 3

52.2

51.6

46.2

36.3

Grade 4

49.6

51.3

46.4

43.1

Grade 5

61.4

51.4

46.7

40.4

Grade 6

58.8

57.9

62.0

62.7

Grade 7

56.0

57.0

61.2

60.2

Grade 8

56.9

52.0

51.9

50.4

Original Formulas

 

1st Quartile

2nd Quartile

3rd Quartile

4th Quartile

Grade 3

56.2

63.0

59.0

41.1

Grade 4

47.8

44.7

46.5

41.8

Grade 5

67.6

57.1

49.3

39.0

Grade 6

28.5

26.5

25.4

21.7

Grade 7

59.3

55.0

57.5

42.1

Grade 8

33.2

41.2

34.0

30.4

 


 

Mathematics

 

Proposed Formulas

 

 

 1st Quartile

2nd Quartile

3rd Quartile

4th Quartile

Grade 3

60.8

58.2

55.1

51.0

Grade 4

57.5

56.4

56.8

57.7

Grade 5

55.7

51.1

52.6

57.8

Grade 6

66.1

62.6

65.7

66.5

Grade 7

57.9

51.7

54.8

65.0

Grade 8

61.3

58.7

60.6

61.9

 

 

 

Table 12. Correlations Between Growth Composite Scores and Selected School Characteristics

at the School Level (Grades 3-8), 2003-04

                         Proposed Formulas

 

Total Number of Students

Percent of Minority Students

Academic Change

0.005

-0.32

 

Original Formulas

 

Total Number of Students

Percent of Minority Students

Expected Growth

-0.26

-0.19

 

            Correlation analyses for the proposed formulas were performed using the academic change composite for Reading and Mathematics for grades 3-8.

            Correlation analyses for the original formulas were performed using the expected standardized growth composite for Reading and Mathematics for grades 3-8.

 

 

Table 13. Correlations Between Growth Composite Scores and Percent of Minority Students by

School Size (Grades 3-8), 2003-04  

Proposed Formulas

 

Percent of Minority Students

Small Schools1

 Medium Schools2

 Large Schools3

Academic Change

-0.25

-0.37

-0.47

 

Original Formulas

 

Percent of Minority Students

Small Schools1

 Medium Schools2

 Large Schools3

Expected Growth

-0.16

-0.21

-0.34

School size range: 16 to 1,706

1 Less than 200 students: 28.8%

2 Between 200 and 400 students: 42.4%

3 More than 400 students: 28.8%

 

Correlation analyses for the proposed formulas were performed using the academic change composite for Reading and Mathematics for grades 3-8.

Correlation analyses for the original formulas were performed using the expected
standardized growth composite for Reading and Mathematics for grades 3-8.

 

 

Table 14. Percent of Schools (Grades 3-8) Meeting or Exceeding Growth Expectations by Quartiles of Percent Minority, 2003-2004

 

 

Proposed Formulas

 

 

1st Quartile

2nd Quartile

3rd Quartile

4th Quartile

Met Academic Change

89.6

82.6

63.2

41.3

Original Formulas

 

1st Quartile

2nd Quartile

3rd Quartile

4th Quartile

Met Expected Growth

76.9

71.0

57.7

55.8

 

 The results using the proposed formulas show less equity across the range of percent minority students compared to the original formulas.  9 The percent of schools meeting expected growth declines over the quartiles as the percent of minority students in the school increases. 9 For both formulas, the expectation is neutral toward demographic factors because the formulas use only student’s past achievement to predict future achievement.

Note: When viewing the results of analyses correlating demographic factors and school growth, as one demographic factor decreases in importance, others appear to increase in importance.


 

Table 15. Percent of Schools (Grades 3-8) Meeting or Exceeding Expectations by Quartiles of Number of Students, 2003-2004

 

 

Proposed Formulas

 

 

1st Quartile

2nd Quartile

3rd Quartile

4th Quartile

Met Academic Change

64.9

75.2

79.4

72.3

Original Formulas

 

1st Quartile

2nd Quartile

3rd Quartile

4th Quartile

Met Expected Growth

72.0

83.9

77.8

34.9

 

            In the proposed formulas, the percent of schools meeting academic change remains stable over the quartiles as the number of students in the school increases. A majority of schools are meeting academic change across all school sizes.

            In the original formulas, the percent of schools meeting expected growth declines after the second quartile as the number of students in the school increases.

Note: When viewing the results of analyses correlating demographic factors and school growth, as one demographic factor decreases in importance, others appear to increase in importance.

 

Table 16. Percent of Schools (Grades 3-8) Meeting or Exceeding Expectations by Average Pretest Score Quartiles

Reading

 

 

Proposed Formulas

 

Year

1st Quartile

 

2nd Quartile

3rd Quartile

4th Quartile

1999

27.6

 

44.9

52.3

62.5

2000

14.1

 

35.7

47.0

62.7

2001

15.7

 

38.8

53.1

73.3

2002

30.0

 

51.9

61.8

73.7

2003

17.5

 

36.0

47.5

72.3

2004

25.3

 

47.4

63.4

81.0

 

Original Formulas

Year

1st Quartile

 

2nd Quartile

3rd Quartile

4th Quartile

2002

29.8

 

44.1

55.1

68.4

2004

30.5

 

40.5

39.9

45.0

 

Mathematics

 

 

Proposed Formulas

 

Year

1st Quartile

 

2nd Quartile

3rd Quartile

4th Quartile

1999

41.6

 

64.0

72.6

86.0

2000

32.3

 

53.7

73.5

85.7

2001

19.0

 

25.3

38.9

68.2

2002

33.5

 

59.3

72.7

88.0

2003

82.5

 

96.9

98.7

98.9

2004

58.3

 

82.2

92.2

98.0

 

Original Formulas

Year

1st Quartile

 

2nd Quartile

3rd Quartile

4th Quartile

2004

68.8

 

83.4

94.3

98.5

 

 

 

Table 17. Trend in Percent of Schools with a C-Ratio of 1.5 or Greater by Reading Quartiles, Using the Proposed Formulas

Year

1st Quartile

2nd Quartile

3rd Quartile

4th Quartile

1999

11.8

15.8

20.7

19.9

2000

2.3

7.9

10.0

18.4

2001

2.5

6.9

11.7

22.0

2002

8.1

6.2

12.0

18.9

2003

4.6

3.5

4.8

16.7

2004

6.6

6.1

10.7

18.6

 

The c-ratio is computed by dividing the number of students who met their expectation for academic change (“0” or greater) by the number of students who did not meet their expectation.

The schools are classified in quartiles based on the average previous year’s Reading average c-scale score.

 

            A c-ratio is an indicator of school performance. Fundamentally, c-ratios greater than 1.5 mean that schools are helping a vast majority of their students meet individual growth standards. 

            In most years, schools in the lowest quartile have a lower ratio of students meeting the growth standard than the upper quartiles.

            Schools that have a lower ratio of students who are meeting the growth standard should also be less likely to meet the growth standard. This is reflected in Tables 15 and 16.


Table 18. Trend in Percent of Schools Meeting or Exceeding Expectations by EOGs

Reading

 

 

Proposed Formulas

 

 

Grade

1998-99

1999-00

2000-01

2001-02

2002-03

2003-04

3

29.3

22.8

29.1

38.2

50.1

51.2

4

51.1

33.8

38.8

43.3

22.0

49.4

5

19.2

55.0

59.1

56.8

46.5

45.0

6

90.5

76.8

72.1

75.8

63.9

68.4

7

71.7

58.4

71.8

65.6

57.3

66.0

8

39.1

36.0

44.9

64.7

58.4

52.1

 

 

 

Original Formulas

 

 

Grade

1998-99

1999-00

2000-01

2001-02

2002-03

2003-04

3

85.8

76.2

51.1

54.6

70.0

68.1

4

22.4

17.9

26.4

29.2

61.5

28.3

5

58.0

80.2

85.4

87.0

97.3

73.0

6

44.0

19.7

18.1

24.9

15.3

2.2

7

72.9

33.2

44.9

29.7

76.4

70.4

8

55.2

31.4

51.9

56.2

57.8

10.9

 

Mathematics

 

 

Proposed Formulas

 

 

Grade

1998-99

1999-00

2000-01

2001-02

2002-03

2003-04

3

51.1

50.0

49.3

49.0

91.2

86.5

4

54.0

59.1

31.4

62.4

94.6

82.2

5

61.5

55.8

36.8

56.6

90.5

62.5

6

79.0

60.0

49.3

75.9

90.9

78.5

7

68.3

60.8

40.5

62.2

67.5

47.0

8

71.9

74.8

40.4

65.3

78.0

73.6

 

 

 

Original Formulas

 

 

Grade

1998-99

1999-00

2000-01

2001-02

2002-03

2003-04

3

39.9

38.1

51.9

54.9

93.9

91.2

4

85.9

86.3

83.2

93.1

99.8

95.8

5

74.3

66.0

56.0

72.2

92.0

46.4

6

71.4

55.5

66.2

81.3

94.4

83.8

7

82.6

61.0

80.2

86.6

81.1

57.4

8

66.8

64.6

32.1

49.1

58.9

48.8

 

            The percent of schools making growth at any grade level in any year is expected to be different using the proposed formulas compared to the original formulas since the proposed formulas use a different standard of growth.

            Some of the notable differences are at the years when the posttest is the second edition and the pretest is the first edition. A fundamental strength of the proposed formulas is the ability to more accurately predict growth across test editions.

            Using the original formulas, the percent of schools meeting growth in grade 6 Reading is low across all years. Using the proposed formulas, the results show a higher percentage. This is a function of using a different method of setting the growth standards and more closely reflects the implementation of curriculum.

 

 

Table 19. Trend in Percent of Schools Meeting or Exceeding Expectations by EOCs

Proposed Formulas

 

2001-02

 

2002-03

2003-04

Algebra I

59.4

 

56.3

52.7

Algebra II

24.6

 

26.8

15.4

Biology

52.1

 

18.2

12.8

Chemistry

14.4

 

18.9

24.2

English I

92.8

 

39.3

42.0

Geometry

20.3

 

16.3

13.1

Physical Sciences

50.2

 

52.6

62.3

Physics

1.3

 

1.6

0.7

 

Original Formulas

 

2001-02

 

2002-03

2003-04

Algebra I

91.8

 

89.0

89.1

Algebra II

71.4

 

63.6

56.0

Biology

61.4

 

26.1

26.3

Chemistry

70.0

 

77.3

72.9

ELPS

44.5

 

39.6

 

English I

61.3

 

97.9

98.7

Geometry

34.0

 

30.1

26.8

Physical Sciences

58.4

 

61.8

67.6

Physics

76.2

 

69.4

70.2

US History

20.4

 

28.5

 

 

The percent of schools making growth across EOCs in any year is expected to be different using the proposed formulas compared to the original formulas since the proposed formulas use a different standard of growth.


 

Table 20. Percent of Schools Meeting Expected Growth and High Growth 

 

Proposed Formulas

 

2001-02

2002-03

2003-04

 

N

Percent

N

Percent

N

Percent

Met At Least Expected Academic Change

1,424

65.4

1,839

85.0

1,582

72.1

Met High Academic Change

395

18.1

751

34.6

536

24.4

 

Original Formulas

 

2001-02

2002-03

2003-04

 

N

Percent

N

Percent

N

Percent

 

Met At Least Expected Growth

1,642

74.8

2,092

94.3

1,676

75.1

 

Met High Growth

779

35.5

1,617

72.9

785

35.2

 

 

•           In the proposed formulas, high growth is met when Academic Change is greater or equal to “0” and the c-ratio is greater than 1.5.

Using the proposed formulas, the percent of schools meeting both the expected and high growth expectations in these three years is similar except for results for the high growth standard in 2002-03.

            The percent of schools meeting these standards is different between the two sets of formulas.

 

Table 21. Percent of Schools Meeting Expected Growth and High Growth by Grade Span*

Proposed Formulas

 

Grade Span

2001-02

2002-03

2003-04

Met At Least Expected Academic Change

K-5

577

52.8%

964

86.4%

842

75.0%

6-8

299

78.9%

327

85.8%

279

71.9%

9-12

319

96.4%

319

98.1%

246

73.6%

Met

K-5

125

11.4%

472

42.3%

359

32.0%

High Academic

6-8

139

36.7%

162

42.5%

104

26.8%

Change

9-12

70

21.1%

12

3.7%

8

2.4%

 


 

Original Formulas

 

Grade Span

2001-02

2002-03

2003-04

Met At Least

K-5

904

79.6%

1151

98.9%

988

85.1%

Expected Growth

6-8

222

59.4%

320

84.0%

124

32.0%

 

9-12

264

81.7%

277

85.7%

310

96.9%

 

K-5

486

42.8%

1095

94.1%

504

43.3%

Met High Growth

6-8

126

33.7%

214

56.2%

46

11.9%

9-12

63

19.5%

96

29.7%

139

43.4%

 

* Elementary (K-5) schools include schools with students no lower than kindergarten and no higher than grade 5.

Middle (6-8) schools are schools with students no lower than grade 6 and no higher than grade 8.

High schools (9-12) consist of schools with students no lower than grade 9. 

            The percent of schools making growth among the three grade spans in any year is expected to be different using the proposed formulas compared to the original formulas since the proposed formulas use a different standard of growth.

            Using the original formulas, the percent of middle (6-8) schools meeting growth is low in 2001-02 and 2003-04. Using the proposed formulas, the results show a higher percentage. 


 

 

Table.22 Correlations between Residuals and the Average of the Two Previous Assessment Scores (ATPA)

 

Reading

Mathematics

Grade 3*

-0.39

-0.41

Grade 4

-0.20

-0.18

Grade 5

-0.36

-0.19

Grade 6

-0.16

-0.23

Grade 7

-0.13

-0.13

Grade 8

-0.23

-0.23

 

*The grade 3 pretest c-scale score is used since two previous assessment scores are not available. 

 

√          In the original proposal, a factor of 0.1 was used to help offset the correlation between residuals and predicted values.  See Table 24 below for the results of this adjustment (including the adjustment for PA).  Since in Table 24 all correlations to residuals decreased to the hundredths place of the decimal, the decision was made to leave the factor at 0.1 since is reduced (along with the 0.2) a vast majority of the systematic error except for grade 5 reading. Additionally, the grade 3 systematic error was deemed inherent in the pretest post test system and other factors due to the age of the students involved.


 

Table.23 Correlations between Residuals and the Assessment Scores (PA)

 

Reading

Mathematics

Grade 3*

-0.39

-0.41

Grade 4

-0.17

-0.14

Grade 5

-0.32

-0.19

Grade 6

-0.05

-0.19

Grade 7

-0.20

-0.10

Grade 8

-0.20

-0.22

 

*The grade 3 pretest c-scale score is used since two previous assessment scores are not available. 

 

√            In the original proposal, a factor of 0.2 was used when only a single predictor was used (due to the similarity between those correlations and -0.2).  The results of incorporating the 0.1 factor are shown below.

 

 

Table 24. EOG Correlations Between Predicted Values and Residuals using 0.1 and 0.2 for Regression Coefficients*

Proposed Formulas

 

Reading

Mathematics

Grade 3

-0.13

-0.13

Grade 4

-0.04

-0.02

Grade 5

-0.19

-0.003

Grade 6

0.01

-0.04

Grade 7

0.04

0.06

Grade 8

-0.05

-0.02

 

*After careful consideration of the effects on traditionally high achieving students, the regression factor was adjusted to 0.08 for two previous assessments and 0.18 for a single previous assessment.

 

Table 25. Correlations Between Predicted Values and Residuals using 0.08 and 0.18 for Regression Coefficients*

 

Reading

Mathematics

Grade 3

-0.16

-0.16

Grade 4

-0.08

-0.05

Grade 5

-0.23

-0.04

Grade 6

-0.02

-0.07

Grade 7

0.01

0.02

Grade 8

-0.09

-0.06

*After careful consideration of the effects on traditionally high achieving students, the regression factor was adjusted to 0.08 for two previous assessments and 0.18 for a single previous assessment.

 

 

Scale Score Ranges

Subject/Grade

Level I

Level II

Level III

Level IV

Reading PT3

3

4

5

6

7

8

 

 

213-223

216-229

223-235

228-238

228-241

228-242

231-243

 

 

224-232 230-239 236-243 239-246 242-251 243-251 244-253

 

 

233-244 240-249 244-254 247-258 252-263 252-263 254-265

 

 

245-264

250-272

255-275

259-277

264-283

264-287

266-290

 

 

 

Achievement Level Descriptions:

Level I: Students performing at this level do not have sufficient mastery of knowledge and skills in this subject area to be successful at the next grade level.

Level II: Students performing at this level demonstrate inconsistent mastery of knowledge and skills in this subject are and are minimally prepared to be successful at the next grade level.

Level III: Students performing at this level consistently demonstrate mastery of grade level subject matter and skills and are well prepared for the next grade level.

Level IV: Students performing at this level consistently perform in a superior manner clearly beyond that required to be proficient at grade level work.

 

 

End-Of-Course (EOC) Achievement Levels (July 2004)

Description

Level I

Level II

Level III

Level IV

Algebra I

31-44

45-54

55-65

66-96

Algebra II

33-45

46-57

58-68

69-102

Biology

28-46

47-54

55-64

65-85

Chemistry

31-47

48-55

56-64

65-90

Civics and Economics

>139-139

140-149

150-159

160-<160

English I

28-42

43-51

52-60

61-82

Geometry

32-45

46-56

57-66

67-93

Physics

23-42

43-51

52-62

63-91

Physical Science

30-43

44-53

54-63

64-86

US History

>139 - 139

140-149

150-159

160- <160

 

2005-06 and 2006-07

Achievement Level Ranges for North Carolina

End-of-Course Tests

 

Test

Level I

Level II

Level III

Level IV

Algebra I

Less than or equal to 44

45-54

55-65

Greater than or equal to 66

Algebra I

(Interim for 2006-07)

Less than or equal to 139

140-147

148-157

Greater than or equal to  158

English I

Less than or equal to 42

43-51

52-60

Greater than or equal to 61

English I

(Interim for 2006-07)

Less than or equal to 137

138-145

146-156

Greater than or equal to 157

Biology

Less than or equal to 46

47-54

55-64

Greater than or equal to 65

U.S. History

(Interim for 2005-06)

Less than or equal to 139

140-149

150-159

Greater than or equal to 160

U.S. History

(for 2006-07 and beyond)

Less than or equal to 139

140-148

149-159

Greater than or equal to 160

Civics and Economics

(Interim for 2005-06)

Less than or equal to 139

140-148

149-158

Greater than or equal to 159

Civics and Economics

(for 2006-07 and beyond)

Less than or equal to 140

141-147

148-159

Greater than or equal to 160

Algebra II

Less than or equal to 45

46-57

58-68

Greater than or equal to 69

Algebra II

(Interim for 2006-07)

Less than or equal to 138

139-146

147-157

Greater than or equal to 158

Chemistry

Less than or equal to 47

48-55

56-64

Greater than or equal to 65

Geometry

Less than or equal to 45

46-56

57-66

Greater than or equal to 67

Geometry

(Interim for 2006-07)

Less than or equal 138

139-147

148-157

Greater than or equal to 158

Physics

Less than or equal to 42

43-51

52-62

Greater than or equal to 63

Physical Science

Less than or equal to 43

44-53

54-63

Greater than or equal to 64