---

Appendix
 
 

Special Conditions

2000-2001

Adjustments for ABCs Special Conditions

Technical Notes: Standard Conventions used in the 2000-2001 ABCs Analyses

98% Rule. K-8 schools must test at least 98% of eligible students. Eligible students are those who are not excluded from an End-of-Grade test according to the rules and procedures governing the testing program. A composite percentage of eligibles tested is computed by combining information from reading, mathematics and alternate assessments across all grades in a school. (Note. Students who registered for the North Carolina Computer Adaptive Testing System (NCCATS), or the North Carolina Alternate Assessment Portfolio (NCAAP), or the North Carolina Alternate Assessment Academic Inventory (NCAAAI) are credited as having been tested). The final composite must be greater than or equal to 98, when rounded to the nearest whole number. Schools identified as having violated the 98% rule are asked to justify their rate. The explanations are reviewed by DPI, and either accepted or rejected. If rejected, the school is assigned a 98R status, which means the school is in violation of the rule and ineligible to receive incentives or recognition. A school in violation for two consecutive years may be identified as low-performing by the State Board of Education. In this report, schools that violate testing requirements are assigned a violation status and cannot receive another ABCs status, except low-performing.

95% Rule. High schools must test at least 95% of the students in membership in grade 10 and enrolled in courses for which there are End-of-Course (EOC) tests. A composite percentage of students is computed by combining information from grade 10 membership, alternate assessments and enrollment in EOC courses. (Note. Students who registered for the North Carolina Computer Adaptive Testing System (NCCATS), or the North Carolina Alternate Assessment Portfolio (NCAAP), or the North Carolina Alternate Assessment Academic Inventory (NCAAAI) are credited as having been tested). The composite must be greater than or equal to 95, when rounded to the nearest whole number. Schools in violation are asked to justify their rate. The explanations are reviewed by DPI; and accepted or rejected. If rejected, the school is assigned a 95R status, which means the school is in violation of the rule and ineligible to receive incentives or recognition. A school in violation for two consecutive years may be identified as low-performing by the State Board of Education. In this report, schools that violate testing requirements are assigned a violation status and cannot receive another ABCs status, except low-performing.

Algebra I Scores in the Performance Composite. Algebra I scores for current ninth graders who took Algebra I prior to their ninth grade year are included in the performance composite for the high school where they are currently enrolled. Algebra I scores of students in the middle school grades (6, 7, or 8) during the current school year are included in the K-8 performance composite of the middle school where they are currently enrolled. Algebra I scores of students currently enrolled in grade 10 in a senior high school (Grades 10-12) who took Algebra I while in grades 7, 8, or 9 are included in the performance composite of the senior high school.

Confidence Interval Applied to the Performance Composite in Low-Performing Schools. The performance composite is the percent (proportion) of students in a school who score at or above grade level (i.e., in Achievement Levels III or IV) on end-of-grade and/or end-of-course tests. The performance composite is computed by adding all scores at or above Achievement Level III on each of the tests, and then dividing the sum by the total number of valid scores on the tests. The result is one of the factors used in determining low-performing school status. If a school does not make expected growth and less than half its students are performing at or above grade level, the school is low-performing.

The confidence interval is a way of taking into account the statistical fluctuations that occur from year to year in small schools or schools with highly variable scores. The confidence interval itself will be narrow or wide depending on the size of the school and the variation in the scores that make up the performance composite. In general, the confidence interval is narrower when the number of students is larger, or the scores are more homogeneous; conversely, the confidence interval is wider when the number of students is smaller, or the scores are less homogeneous.

This means it is possible that a potentially low-performing school has a performance composite that is considerably below 50% but that school is not considered low-performing because the confidence interval for that school is wide (i.e., there is less confidence in the value of the performance composite). This situation would likely be true for a school that has few students or has wide variation in test scores. Conversely, it is possible that a school has a performance composite that is fairly close to 50% and is considered low-performing because the confidence interval for that school is very narrow (i.e., there is high confidence in the performance composite). This situation would likely be true for a school that has a large number of students or students all have about the same test score.

As long as the value, 50, lies within or on the boundary of the confidence interval for an observed performance composite, then the performance composite is not significantly less than 50 and hence the school is not classified as low-performing.

Excessive Exclusions. Another testing requirement of schools in the ABCs is that they must not have excessive exclusions. Student exclusions from testing must be consistent with federal and state guidelines for students with disabilities and students appropriately identified as Limited English Proficient. Please refer to relevant testing program guidelines specified in Testing Modifications and Accommodations for Students with Disabilities and Guidelines for Testing Students with Limited English Proficiency, and the updates to these guidelines, which were disseminated during 2000-2001 through the following memorandums to LEA superintendents:
 
 

• Changes to Guidelines for Testing Students with Disabilities (September 19, 2000),
• Changes to Guidelines for Testing Students with Limited English Proficiency Effective 2000-2001 (February 16, 2001), and
• End-of-Grade Testing for Students with Disabilities; Use of End-of-Grade and End-of-Course Tests in Promotion Decisions or Course Credit Decisions for Students with Disabilities; Procedures for Review of Requests for Waiver of Promotion Standards (March 29, 2001).

Exclusions are determined at the school on a case by case basis and they must be documented in Individualized Education Programs (IEPs). A composite index for exclusions is computed for every school; when this index is greater than the state average exclusion rate plus two standard deviations (determined from the distribution of school exclusion rates), they are considered excessive. (Note. Students who registered for the North Carolina Computer Adaptive Testing System (NCCATS), or the North Carolina Alternate Assessment Portfolio (NCAAP), or the North Carolina Alternate Assessment Academic Inventory (NCAAAI) are credited as having been tested).
 
 

To compute the exclusion rate in a K-8 school, information from reading and mathematics EOG, alternate assessments and EOC information if appropriate across all grades in a school are combined in a composite index. For high schools, EOC data, North Carolina High School Comprehensive Test (NCHSCT) information and alternate assessment information if appropriate are combined to form a composite index.

Schools identified as having excessive exclusions are asked to justify their exclusion rate; explanations are reviewed by DPI, and accepted or rejected. If rejected, the school is assigned an EE status, and cannot receive incentives or recognition. A school that has excessive exclusions for two consecutive years may be identified as low-performing by the State Board of Education.

Equating Study. In May of 1998 the State Board of Education (SBE) adopted a new K-12 mathematics curriculum which necessitated development of new mathematics tests. The Testing Section of the Division of Accountability Services developed EOG field test items for mathematics in 2000, and the "new" mathematics tests at grades 3-8 were administered in the Spring of 2001. As early as January, 1999, the Department of Public Instruction informed the SBE of the need to conduct an equating study to link the old mathematics test (1^st edition) and the new mathematics test (2^nd edition) to enable use of the ABCs formulas for the 2000-2001 school year.
 
 

Two groups of analysts worked independently analyzing the equating data during the summer of 2001 for the EOG mathematics tests. The groups were Dr. David Thissen and his staff at the L.L. Thurstone Psychometric Laboratory (UNC-CH) and, the Reporting Section of the Division of Accountability Services. After determining the adequacy of the equating samples, equipercentile equating was used. This method calculates the percentiles of each distribution of the old mathematics test and the new mathematics test and equating scores corresponding to the same percentile ranks.

The groups conferred and compared results in early August. An audit panel of outside technical experts appointed by the SBE reviewed the methodology and results on August 16 -17 and reported to the SBE Ad Hoc Committee on September 7. The panel concluded that the technical process used to compute the equating study was sound. The Ad Hoc Committee of the SBE accepted the report and directed the Department of Public Instruction Accountability Services Division to proceed with the compilation of the ABCs report. With the sign-off of the technical process to equate the new mathematics test scores with the old ones, achievement level cut scores also were revised and the new achievement levels are used in the ABCs results for 2000-2001.
 
 

Component of Growth Composite	Standard Deviation of change for expected growth composite	Standard Deviation of change for exemplary growth composite
English II	7.6	7.5
Competency Passing Rate	12.8	12.8
College Prep/College Tech Prep %	10.0	10.0
ABCs Dropout Rate	2.0	2.0

 
 
 
 
 

The NCAAP has four domains: Communication (C), Personal Home Management (PHM), Community (COM), and Career/Vocational (C/V). Each domain receives a score of .25 for a total possible score of 1.00 if a student is proficient in all four domains. A student must receive a 3 or 4 in a domain to be counted as proficient in that area. The student’s proficiency score is derived from the following scale:

1=Novice; 2=Apprentice; 3=Proficient; and 4=Distinguished

The resulting sum of scores across the four domains of NCAAP contributes to the school’s performance composite score for the ABCs. Table 1 shows possible NCAAP data for two students in a school.

 

	A Communications	B Personal Home Management	C Community	D Career/Vocational	E Number of Domains Proficient	F Proficient Score E/4(# of Domains)
Student (1)	*NP	**P	P	NP	2	.50
Student (2)	NP	NP	NP	NP	0	.00

Total # Of (P)

0

1

1

0

2

.50

*NP = Not Proficient

**P = Proficient

Example: For a K-5 Elementary School Performance Composite at grade 4 for Reading, Math, Writing and NCAAP.
 
 

The performance composite for any school is the total number of scores at or above Level III (or at or above grade level) in each subject included in the ABCs model, divided by the total number of valid scores. The performance composite is reported as a percentage. In Table 2, the NCAAP performance for the two students in Table 1 is included in the school’s performance composite.

The first row represents the total number of scores at or above Level III (numerator) as reported for each subject area and for the NCAAP. In the second row, the total number of valid scores (denominator) or number of students who were tested in each subject area or participated in the NCAAP at this particular school are included. To obtain the performance composite, divide the total number scores at or above Level III by the total number of valid scores or number of students who took the test and participated in the NCAAP. Multiply the product by 100 to yield the performance composite score for this school.
 
 

 

Subject/Grade			Level I	Level II	Level III	Level IV
Reading	PT3		119-127	128-132	133-144	145-162
	3		115-130	131-140	141-150	151-172
	4		119-134	135-144	145-155	156-174
	5		124-138	139-148	149-158	159-178
	6		128-140	141-151	152-161	162-180
	7		130-144	145-154	155-163	164-183
	8		132-144	145-155	156-165	166-184
Mathematics	PT3		211-219	220-229	230-239	240-260
	3		218-237	238-245	246-254	255-276
	4		221-239	240-246	247-257	258-285
	5		221-242	243-249	250-259	260-295
	6		228-246	247-253	254-264	265-296
	7		231-249	250-257	258-266	267-307
	8		235-253	254-260	261-271	272-310

 
 

Subject/Grade			Level I	Level II	Level III	Level IV
Writing 4 & 7			0.0-1.0	1.5-2.0	2.5-3.0	3.5-4.0

 

Achievement Level Descriptions:
 
 

Level I: Students performing at this level do not have sufficient mastery of knowledge and skills in this subject area to be successful at the next grade level.
 
 

Level II: Students performing at this level demonstrate inconsistent mastery of knowledge and skills in this subject are and are minimally prepared to be successful at the next grade level.
 
 

Level III: Students performing at this level consistently demonstrate mastery of grade level subject matter and skills and are well prepared for the next grade level.
 
 

Level IV: Students performing at this level consistently perform in a superior manner clearly beyond that required to be proficient at grade level work.

Copyright and Legal Notices