200001 ABC PROGRAM INFORMATION
THE EOC PREDICTION FORMULAS
Revised February 6, 2001
Vol. 1, no. 5
The use of endofcourse prediction formulas to determine growth/gain on the ten multiple choice EndofCourse (EOC) tests is a recently approved modification to the ABCs. The formulas, designed for use with the EOC, were approved at the May 2000 State Board of Education (SBE) meeting for implementation beginning in the 200001 school year. In January 2001 the SBE approved finetuning of the EOC prediction formulas and weighting the ABCs growth/gain composites for the 200001 school year.
Why change the way gain
is computed for the high school EOC tests?
The ABCs of Public Education, first implemented in the 199697 school year, measured student growth by grade level and subject by comparing the endofgrade (EOG) test results from one year to the next, except in the case of grade 3, where a pretest was administered during the same school year. High schools expressed concerns about their participation in the ABCs, noting that there was no comparable mechanism in place at the high school that could measure developmental growth across a course or subject. Although attempts were made to develop a pretest to posttest model at the high school, such a model did not prove feasible. Continuing concerns about the ABCs at the high school level contributed to the postponement of implementation until the following year. During this time a Steering Committee for Assessment and Accountability (appointed by the SBE) collected input and studied the issue of high school accountability.
In 199798, a high school model based on an EOC index was ready for implementation. This model based EOC gain on an index that measured current year gains in EOC scores over twoyear historical baselines. Thus in the high school model, schools were not held accountable for change, or improvement, in the same group of students over time, but rather the gain of one group of students over previous groups of students. There was strong sentiment among stakeholders that a cohort model based on the same students' growth over time should become a part of the high school model. Research and planning in this direction continued, and one step was the development of the North Carolina High School Comprehensive Test (NCHSCT). This test was designed to measure growth in reading and mathematics from the time of the EOG administration at the end of grade 8 to grade 10; it was administered in 199798 to collect data for further analyses.
In 199899, a growth component based on the NCHSCT was added to the high school model. Growth from grade 8 to grade 10 in Reading and Mathematics would be measured using the pretest to posttest model. With this addition, the high school model had at least one component that was based on the same students' growth over time. The other five EOC tests continued to provide results for the index computations, while work towards a prediction model continued. In January 2001 the SBE approved finetuning of the EOC prediction formulas for the 200001 school year. These refinements include: 1) adjusting EOC prediction formulas exemplary growth/gain standard; and 2) weighting the ABCs growth/gain composites.Various statistical models of an EOC prediction formula were investigated. The formulas that were tested were measured against three criteria: predictability, simplicity and equity. After two years of research and analyses, the newly approved EOC prediction formulas met these criteria.
Are the EOC prediction formulas applied
only to high school EOC courses?
No. The formulas are used to compute growth/gain wherever the EOC courses are offered, regardless of the grade configuration of the school. For example, a K8 school that offers Algebra I to eighth graders would use the formulas (with seventh grade EOG math as predictor) to compute growth/gain in that subject; the growth/gain would be included in the composite growth/gain for the school.
How do calculations with the new
prediction formulas differ from calculations using the
index?
In the EOC prediction formulas, an equation is used to calculate an ABCs goal (or expected score) for each school on each EOC test. Each expected score is based on the proficiency of the students when they were in previous grades or courses. Proficiency is determined by students' performance (scores) on the North Carolina EndofGrade (EOG) or EndofCourse (EOC) tests, which serve as predictors of the same students' performance in the EOC course where they are currently enrolled. The EOG or EOC test(s) serving as predictor(s) are different for each EOC test based on the criteria of predictability, simplicity, and equity.
Let's look at an example of a calculation to find a school's ABCs goal (expected score) for Algebra I. The equation used to compute the target score for Algebra I is
 Algebra I Expected Score = b_{0} + (b_{IMP}
x IMP), where
b_{0} is the state average performance for the EOC = 60.4;
b_{IMP} is the value used to estimate the effect of the school's average math proficiency on the predicted average EOC test score = 0.88; and  IMP is the index of mathematics proficiency equals [school's average EOG Grade 8 Math scale score minus 176.1,(the state's average scale score)].
Substituting the values from the prediction formula parameters for EndofCourse performance, the equation looks like this:
 Algebra I Expected Score (Predicted Algebra I Mean) = 60.4 + [0.88 x (Math 176.1)]
Step 1: Identify a group of students currently enrolled in Algebra I with predictor scores. In this case, the group of students must have scores on EOG Math from grade 8.
 Important considerations in selecting a matched set of students: Some students currently enrolled in Algebra I may not have a grade 8 EOG score in math, for a number of reasons. There may be students who have transferred from other states; a student may have been absent and failed to complete a makeup. Enrollment can change daily due to the addition of new students, transfers, and withdrawals. This makes it impossible to compute a totally accurate expected score until the current enrollment is "captured" on the first day of EOC testing.
Step 2: Find the average EOG Math (grade 8) score for the matched group of students; using this average in the equation, determine the expected Algebra I score. Let's say the average EOG Math score in grade 8 for this group was 178.
 Algebra I Expected Score = 60.4 + [0.88 x (178
176.1)]
Algebra I Expected Score = 60.4 + 1.672= 62.042*, or, 62.0 (rounded).
This means that to reach the expected score for expected growth/gain, the school must have an average Algebra I EOC score that equals or exceeds 62.0.
*This sum reflects full precision carried throughout each computation, and does not reflect the sum of the rounded numbers shown in this example.
How is gain computed?
The ABCs accountability model was implemented on the assumption that the number of students in a school would be roughly equivalent from gradetograde. Every year since the implementation of the ABCs a small number of students in a subject or grade have had a positive or negative influence on the ABCs growth/gain of a school. Weighting of the ABCs growth gain composites was adopted to deal with such inconsistencies. Gain occurs when the actual EOC average score for a matched group of students in a school is equal to or greater than the predicted EOC target score, or ABCs goal for the school. The gain is computed much like growth is computed with the EOG tests. The school's expected score (ABCs goal) is subtracted from the actual EOC average of the group; the difference is divided by the standard deviation of differences (for schools in North Carolina), and the quotient is called the "standard EOC gain" in a given subject. The standard expected growth/gain is then multiplied by the number of scores for Algebra I and is then divided by the total number of scores for all ABCs growth/gain components. This yields a weighted standard expected growth/gain.
Let's continue with the Algebra I example from above to illustrate. The group's actual mean score for the current year is 63. The expected score computed for Algebra I was 62.0.
These values are placed in a worksheet and the expected growth/gain for Algebra I is computed as follows:
Components

A

B

C

E

F

G

H

I


Actual EOC Average

Predicted EOC Average

Actual Minus Predicted

Standard Deviation Of Differences*

Standard Expected Growth/Gain


Algebra I  63.0  62.0  1.0  3.3  +0.3** 
*Standard Deviations of Differences are the same for both expected and exemplary gain calculations for a given EOC course, EXCEPT English II (where gain is based on the index model). See page 7, column G.
**Full precision, though not shown here, is used in all ABC Tools calculations; the final composite is rounded to tenths.
In a high school, the weighted standard expected growth/gain for each of the ten multiplechoice EOCs is added to the other growth/gain components. Those components are:
 standard expected growth in Reading and Mathematics from grades 8 to10;
 gain in the competency passing rate;
 gain in the College/University Prep/College Tech Prep component;
 index gain in English II, and
 change in the ABCs dropout rate.
What about exemplary
growth/gain?
For exemplary gain, the same EOC prediction formulas are used. However, the state average performance is multiplied by 1.03. This means that the exemplary standard is approximately 3% greater than the expected EOC expected score. The formula for exemplary growth/gain in Algebra I is:
 Expected Score for Algebra I Exemplary Growth/Gain = (b_{0} x 1.03) + (b_{IMP} x IMP)
From the earlier example for Algebra I, computations for determining exemplary growth/gain would follow the two steps below.
 Step 1: Multiply b_{0} x 1.03.
60.4 x1.03 = 62.21
 Step 2: Substitute the appropriate values in the formula
and complete the calculations.
62.21 + [0.88 x (178 176.1)]
62.21 + 1.672 = 63.88*, or 63.9 (rounded).
This means that to reach the exemplary expected score, the school's average performance on Algebra I EOC tests must equal or exceed 63.9
*This sum reflects full precision carried throughout each computation, and does not reflect the sum of the rounded numbers shown in this example.
What impact will the EOC prediction
formula have on a school's ABCs status?
Implementation of the EOC prediction formulas in the ABCs calculations will essentially "raise the bar" for high schools. This is because the formulas are based on a more recent population of students than was available when the index model was developed. Weighting the ABCs growth/gain composites will eliminate the concern over small groups of students having the same impact as large groups of students in determining whether a school has met growth standards.
Are there any other
implications?
Since computations rely on previous scores for a group of students, there may be fewer students to include in the calculations. Statewide analyses of 199899 student test data indicated that nearly 90 percent of the students in North Carolina had the necessary data for using the EOC prediction formulas. It is critical that previous test data for students be included in the SIMS or NCWISE systems. School administrators interested in improving the EOC average score in their school should focus on the achievement of all students; not just students who score at levels I and II. It is imperative for teachers to teach the Standard of Course and monitor student work routinely.
Are there time requirements like the EOG
91day rule for prediction?
No, there is no minimum number of days in membership requirement for use with the EOC prediction formulas. It is not currently feasible to collect data on days of membership in EOC test courses. Keep in mind, however, that the growth computations for reading and mathematics from grade 8 to grade 10, which are based on the North Carolina High School Comprehensive Test, require 160 days in membership.
Are there data requirements for using the
prediction formulas?
No, there is no minimum number of scores for each course in order to use the prediction formulas. However, there must be at least 30 scores contributing to the school's growth/gain composite for the scores to be included in the ABCs.
Do testing requirements (95% rule,
excessive exemptions) still apply?
Yes. High schools that test fewer than 95 percent of students subject to EOC tests and the NC Comprehensive Test may not receive recognition, rewards, and/or incentives. In addition, the 98 percent rule applies to schools that have grades that fall within the K8 range; schools that test fewer than 98 percent of eligible students may not receive recognition, rewards, or incentives. A school comprised of grades 6 through 12, for example, must meet the 98 percent rule in grades 6 through 8, and the 95 percent rule in the high school grades. Schools that violate the testing requirements for two consecutive years may be designated as lowperforming by the State Board of Education. Please note that the only allowable exemptions involve LEP students who have been enrolled in school for less than two years.
Worksheet for Computing Weighted Expected
Growth/Gain Composite for a High School Starting with the
20002001 Accountability Cycle
A

B

C

D

E

F

G

H

I

J

K



Components

Actual EOC Average

Predicted EOC Average

Actual minus Predicted

Standard Deviation of Differences

Standard Expected Growth/ Gain

# (n)

Weight (n) divided by (N)

Weighted Growth/ Gain (H x J)


Algebra I  3.3  
Biology  2.6  
ELPS  3.1  
English I  1.8  
US History  2.2  
Algebra II  2.9  
Chemistry  2.5  
Physics  3.3  
Physical Science  2.5  
Geometry  2.5  
Current Index  Year Two Index  Year One Index  Baseline  Difference  Subtract 0.1  Standard Deviation of Expected Gain^{1}  
English II  7.6  
Current %  Year Two %  Year One %  Baseline  Difference  Subtract 0.1  Standard Deviation  
CUP/CTP  10.0  
CompTest (Gr. 10)  Actual Growth  Expected Growth  Actual Minus Expected  Standard Deviation  
Reading  1.6  
Math  2.0  
Grade 10 Passing Rate %  Grade 8 Passing Rate %  Difference  Subtract 0.1  Standard Deviation  
Competency  Year One(%)  Year Two (%)  12.8  
Dropout^{2}  2.0  
Total # (N)  
Weighted Growth /Gain Composite 