To access Quick Links, visit our text-only version.

Division Home 
CCSA Conference 
Accountability and Testing Results 
NC Testing Program 
NC Final Exams 
Policies and Procedures 
Testing Students with Disabilities 
Testing Students Identified As Limited English Proficient 
Accountability Directory 
. Public Schools of North Carolina . . State Board of Education . . Department Of Public Instruction .

ABC PROGRAM INFORMATION

2001-02 :: EOC PREDICTION FORMULAS

EOC PREDICTION FORMULAS

Revised January, 2002
Vol. 1, No. 6

End-of-course prediction formulas were first used in 2000-2001to determine growth expectations on ten multiple choice End-of-Course (EOC) tests. EOC prediction formulas rely on previous test scores as predictors in the ABCs computations. This revision of the Accountability Brief reflects actions of the State Board of Education that include: (a) using the term growth in place of growth/gain; (b) using the term "High Growth" instead of Exemplary growth; (c) eliminating the North Carolina High School Comprehensive Tests in Reading and Mathematics at grade 10; and (d) removing English II from the growth model until the writing tests are revised.


Are the EOC prediction formulas applied only to high school EOC courses?

No. The formulas are used to compute growth wherever the EOC courses are offered, regardless of the grade configuration of the school. For example, a K-8 school that offers Algebra I to eighth graders would use the formulas (with seventh grade end-of-grade (EOG) math as predictor) to compute growth in that subject; the growth would be included in the composite growth for the school.


How do calculations with the prediction formulas differ from calculations using the index method that was used from 1997-98 through 1999-2000?

In the EOC prediction formulas, an equation is used to calculate an ABCs goal (or expected score) for each school on each EOC test. Each expected score is based on the proficiency of the students when they were in previous grades or courses. Proficiency is determined by students' performance (average scores) on the North Carolina tests, which serve as predictors of the same students' performance in the EOC course where they are currently enrolled. The EOG or EOC test(s) serving as predictor(s) are different for each EOC test based on the criteria of predictability, simplicity, and equity (i.e., predictors were chosen to produce the best prediction, with the simplest equation while maintaining equity for schools across the observed range of prior achievement.)

Let's look at an example of a calculation to find a school's ABCs goal (expected score) for Algebra I. The equation used to compute the target score for Algebra I is

  • Algebra I Expected Score = b0 + (bIMP x IMP), where
  • b0 is the state average performance for the EOC = 60.4; bIMP is the value used to estimate the effect of the school's average math proficiency on the predicted average EOC test score = 0.88; and
  • IMP is the index of mathematics proficiency [ equals the school's average EOG Grade 8 Mathematics scale score minus 176.1,(the state's average EOG Grade 8 Mathematics scale score)].

Substituting the values from the prediction formula parameters (b0 and bIMP) for Algebra I performance, the equation looks like this:

Algebra I Expected Score (Predicted Algebra I Average) = 60.4 + [0.88 x (Math ­ 176.1)]

  • Step 1: Identify a group of students currently enrolled in Algebra I with predictor scores. In this case, the group of students must have scores on EOG Mathematics from grade 8.
    Important considerations in selecting a matched set of students: Some students currently enrolled in Algebra I may not have a grade 8 EOG score in mathematics, for a number of reasons. There may be students who have transferred from other states; a student may have been absent and failed to complete a make-up. Enrollment can change daily due to the addition of new students, transfers, and withdrawals. This makes it impossible to compute a totally accurate expected score until the current enrollment is "captured" on the first day of EOC testing.

  • Step 2: Find the average EOG Mathematics (grade 8) score for the matched group of students; using this average in the equation, determine the expected Algebra I score. Let's say the average EOG Mathematics score in grade 8 for this group was 178.
    Algebra I Expected Score = 60.4 + [0.88 x (178 ­ 176.1)]
    Algebra I Expected Score = 60.4 + 1.672= 62.042*, or, 62.04 (rounded).
    This means that to reach the expected score for expected growth, the school must have an average Algebra I EOC score that equals or exceeds 62.04.
    *This sum reflects full precision carried throughout each computation, and does not reflect the sum of the rounded numbers shown in this example.


How is growth computed?

The ABCs accountability model was implemented on the assumption that the number of students in a school would be roughly equivalent from grade-to-grade. However, during the early years of the ABCs it was observed that a small number of students in a subject or grade occasionally had a disproportionate positive or negative influence on the ABCs growth of a school. Weighting of the ABCs growth composites was adopted to deal with such inconsistencies. (Expected Growth occurs when the actual EOC average score for a matched group of students in a school is equal to or greater than the predicted EOC target score, or ABCs goal for the school). The growth is computed much like growth is computed with the EOG tests. The school's expected score is subtracted from the actual EOC average of the group; the difference is divided by the standard deviation of differences (for schools in North Carolina), and the quotient is called the "standard expected growth" in a given subject. The standard expected growth for (e.g., Algebra I) is then multiplied by the number of scores for Algebra I and is then divided by the total number of scores for all ABCs growth components. This yields the weighted standard expected growth.

Let's continue with the Algebra I example from above to illustrate. The group's actual mean score for the current year is 63. The expected score computed for Algebra I was 62.0.

These values are placed in a worksheet and the expected growth for Algebra I is computed as follows:

 

  Components
A
B
C
E
F
G
H
I
Actual EOC Average   Predicted EOC Average Actual Minus Predicted       Standard Deviation Of Differences* Standard Expected Growth    
  Algebra I 63.0 62.0 1.0       3.3 +0.30**

*Standard Deviations of Differences are the same for both expected and high growth calculations for a given EOC course,

**Full precision, though not shown here, is used in all ABC Tools calculations; the final composite is rounded to one-hundredths.

In a high school, the weighted standard expected growth for each of the ten multiple-choice EOCs is added to the other growth components. Those components are:

  • standard expected growth in Reading and Mathematics from grades 8;
  • growth in the College/University Prep/College Tech Prep component;
  • competency passing rate; and
  • change in the ABCs dropout rate.


What about high growth?

For high growth, the same EOC prediction formulas are used. However, the state average performance is multiplied by 1.03. This means that the high growth standard is approximately 3% greater than the expected growth standard. The formula for high growth in Algebra I is:

  • Expected Score for Algebra I High Growth = (b0 x 1.03) + (bIMP x IMP)

From the earlier example for Algebra I, computations for determining high growth would follow the two steps below.

  • Step 1: Multiply b0 x 1.03.
    60.4 x1.03 = 62.21

    Step 2: Substitute the appropriate values in the formula and complete the calculations.
    62.21 + [0.88 x (178 ­ 176.1)]
    62.21 + 1.672 = 63.88*, or 63.90 (rounded).
  • This means that to reach the high growth expected score, the school's average performance on Algebra I EOC tests must equal or exceed 63.90
  • *This sum reflects full precision carried throughout each computation, and does not reflect the sum of the rounded numbers shown in this example.


What impact will be EOC prediction formula have on a school's ABCs status?

Implementation of the EOC prediction formulas in the ABCs calculations in 2000-2001 essentially "raised the bar" for high schools. This is because the formulas are based on a more recent population of students than was available when the index model was developed. Weighting the ABCs growth composites eliminates the concern over small groups of students having the same impact as large groups of students in determining whether a school has met growth standards.


Are there any other implications?

Since computations rely on previous scores for a group of students, there may be fewer students to include in the calculations. Statewide analyses of 1998-99 student test data indicated that nearly 90 percent of the students in North Carolina had the necessary data for using the EOC prediction formulas. It is critical that previous test data for students be included in the SIMS or NCWISE systems.

School administrators interested in improving the EOC average score in their school should focus on the achievement of all students; not just students who score at levels I and II. It is imperative for teachers to teach the North Carolina Standard of Course and monitor student work routinely.


Are there time requirements like the EOG 91-day rule for prediction?

No, there is no minimum number of days in membership requirement for use with the EOC prediction formulas. It is not currently feasible to collect data on days of membership in EOC test courses.


Are there data requirements for using the prediction formulas?

No, there is no minimum number of scores for each course in order to use the prediction formulas. However, there must be at least 30 scores across all courses taught contributing to the school's growth composite for the scores to be included in the ABCs.


Do testing requirements (95% rule, excessive exemptions) still apply?

Yes. High schools that test fewer than 95 percent of students subject to EOC tests may not receive recognition, rewards, and/or incentives. In addition, the 98 percent rule applies to schools that have grades that fall within the K-8 range; schools that test fewer than 98 percent of eligible students may not receive recognition, rewards, or incentives. A school comprised of grades 6 through 12, for example, must meet the 98 percent rule in grades 6 through 8, and the 95 percent rule in the high school grades. In addition, schools must not have excessive exclusions. Schools that violate the testing requirements for two consecutive years may be designated as low-performing by the State Board of Education. Please note that the only allowable exclusions involve LEP students who have been enrolled in the LEA for less than two years and who do not meet the cut scores on the language proficiency test.

Worksheet for Computing Weighted Expected Growth Composite for a High School During the 2001-2002 Accountability Cycle

A
B
C
D
E
F
G
H
I
J
K
Components
Actual EOC Average
Predicted EOC Average
Actual minus Predicted
Standard Deviation of Differences
Standard Expected Growth
# (n)
Weight (n) divided by (N)
Weighted Growth (H x J)
Algebra I             3.3        
Biology             2.6        
ELPS             3.1        
English I             1.8        
US History             2.2        
Algebra II             2.9        
Chemistry             2.5        
Physics             3.3        
Physical Science             2.5        
Geometry             2.5        
  Current % Year Two % Year One % Baseline Difference Subtract 0.1 Standard Deviation        
CUP/CTP             10.0        
  Grade 10 Passing Rate % Grade 8 Passing Rate %     Difference Subtract 0.1 Standard Deviation        
Competency                   12.8        
      Year 1 (%) Year 2 (%) Year 3 (%)              
Dropout1               2.02        
Total # (N)                        
Weighted Growth Composite3

1 Average the ABCs dropout rates (expressed as percentages) for Year One (1998-99) and Year Two (1999-2000) to form a baseline. Subtract the ABCs dropout rate from the ABCs dropout rate (percentage) in Year Three (difference in column (E) divided by standard deviation (G) yields the standard expected growth (H).

2 ABCs dropout rate standard deviation to be determined.

3 If the composite is greater than or equal to zero, the school has made expected growth.