|
|
 |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
 |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
Appendix The ABCs of Public Education: 2001-2002 Growth and Performance of North Carolina Schools Special Conditions
Technical Notes Technical Notes: 98% Rule. K-8 schools must test at least 98% of eligible students. Eligible students are those who are not excluded from an End-of-Grade test according to the rules and procedures governing the testing program. A composite percentage of eligible students tested is computed by combining information from reading, mathematics and alternate assessments across all grades in a school. (Note. Students who registered for the North Carolina Computer Adaptive Testing System (NCCATS), or the North Carolina Alternate Assessment Portfolio (NCAAP), or the North Carolina Alternate Assessment Academic Inventory (NCAAAI) are credited as having been tested). The final composite must be greater than or equal to 98, when rounded to the nearest whole number. Schools identified as having violated the 98% rule are asked to justify their rate. The explanations are reviewed at DPI, and either accepted or rejected. If rejected, the school is assigned a 98R status, which means the school is in violation of the rule and ineligible to receive incentive awards or recognition. A school in violation for two consecutive years may be identified as low-performing by the State Board of Education. In this report, schools that violate testing requirements are assigned a violation status and cannot receive another ABCs status, except low-performing. 95% Rule. High schools must test at least 95% of the students enrolled in courses for which there are End-of-Course (EOC) tests. A composite percentage of students is computed by combining information from alternate assessments and enrollment in EOC courses. (Note. Students who registered for the North Carolina Computer Adaptive Testing System (NCCATS), or the North Carolina Alternate Assessment Portfolio (NCAAP), or the North Carolina Alternate Assessment Academic Inventory (NCAAAI) are credited as having been tested). The composite must be greater than or equal to 95, when rounded to the nearest whole number. Schools in violation are asked to justify their rate. The explanations are reviewed at DPI; and accepted or rejected. If rejected, the school is assigned a 95R status, which means the school is in violation of the rule and ineligible to receive incentive awards or recognition. A school in violation for two consecutive years may be identified as low-performing by the State Board of Education. In this report, schools that violate testing requirements are assigned a violation status and cannot receive another ABCs status, except low-performing. Excessive Exclusions. Another testing requirement of schools in the 2001-02 ABCs is that they must not have excessive exclusions. Student exclusions from testing must be consistent with federal and state guidelines for students with disabilities and students appropriately identified as Limited English Proficient. Please refer to relevant testing program guidelines specified in Testing Modifications and Accommodations for Students with Disabilities and Guidelines for Testing Students with Limited English Proficiency, and the updates to these guidelines, which were disseminated during 2000-2001 through the following memorandums to LEA superintendents:
Exclusions are determined at the school on a case-by-case basis and they must be documented in Individualized Education Programs (IEPs). A composite index for exclusions is computed for every school; when this index is greater than the state average exclusion rate plus two standard deviations (determined from the distribution of school exclusion rates), they are considered excessive. (Note. Students who registered for the North Carolina Computer Adaptive Testing System (NCCATS), or the North Carolina Alternate Assessment Portfolio (NCAAP), or the North Carolina Alternate Assessment Academic Inventory (NCAAAI) are credited as having been tested). To compute the exclusion rate in a school, information from EOG reading, EOG mathematics, alternate assessments and EOC information are combined as appropriate, across all grades in a school to form a composite index.
Schools identified as having excessive exclusions are asked to justify their exclusion rate; explanations are reviewed by DPI, and accepted or rejected. If rejected, the school is assigned an EE status, and cannot receive incentive awards or recognition. A school that has excessive exclusions for two consecutive years may be identified as low-performing by the State Board of Education.
Algebra I Scores in the Performance Composite. Algebra I scores for current ninth graders who took Algebra I prior to their ninth grade year are included in the performance composite for the high school where they are currently enrolled. Algebra I scores of students in the middle school grades (6, 7, or 8) during the current school year are included in the K-8 performance composite of the middle school where they are currently enrolled. Algebra I scores of students currently enrolled in grade 10 in a senior high school (Grades 10-12) who took Algebra I while in earlier grades are included in the performance composite of the senior high school.
Confidence Interval Applied to the Performance Composite in Low-Performing Schools. The performance composite is the percent of students’ scores at or above grade level (i.e., in Achievement Levels III or IV) on end-of-grade and/or end-of-course tests in the school. The performance composite is computed by adding all scores at or above Achievement Level III on each of the tests, and then dividing the sum by the total number of valid scores on the tests. The result is one of the factors used in determining low-performing school status. If a school does not make expected growth and its performance composite is less than 50, the school is low-performing.
The confidence interval is a way of taking into account the statistical fluctuations that occur from year to year in small schools or schools with highly variable scores. The confidence interval itself will be narrow or wide depending on the size of the school and the variation in the scores that make up the performance composite. In general, the confidence interval is narrower when the number of students is larger, or the scores are more homogeneous; conversely, the confidence interval is wider when the number of students is smaller, or the scores are less homogeneous.
This means it is possible that a potentially low-performing school has a performance composite that is considerably below 50% but that school is not considered low-performing because the confidence interval for that school is wide (i.e., there is less confidence in the value of the performance composite). This situation would likely be true for a school that has few students or has wide variation in test scores. Conversely, it is possible that a school has a performance composite that is fairly close to 50% and is considered low-performing because the confidence interval for that school is very narrow (i.e., there is high confidence in the performance composite). This situation would likely be true for a school that has a large number of students or students all have about the same test score.
As long as the value, 50, lies within or on the upper boundary of the confidence interval for an observed performance composite, then the performance composite is not significantly less than 50 and hence the school is not classified as low-performing. Equating Study. In May of 1998 the State Board of Education (SBE) adopted a new K-12 mathematics curriculum which necessitated development of new mathematics tests. The Testing Section of the Division of Accountability Services developed EOG field test items for mathematics in 2000, and the "new" mathematics tests at grades 3-8 were administered in the Spring of 2001. As early as January, 1999, the Department of Public Instruction informed the SBE of the need to conduct an equating study to link the old mathematics test (1st edition) and the new mathematics test (2nd edition) to enable use of the ABCs formulas for the 2000-2001 school year.
Two groups of analysts worked independently analyzing the equating data during the summer of 2001 for the EOG mathematics tests. The groups were Dr. David Thissen and his staff at the L.L. Thurstone Psychometric Laboratory (UNC-CH) and, the Reporting Section of the Division of Accountability Services. After determining the adequacy of the equating samples, equipercentile equating was used. This method calculates the percentiles of each distribution of the old mathematics test and the new mathematics test and equates scores corresponding to the same percentile ranks.
The groups conferred and compared results in early August. An audit panel of outside technical experts appointed by the SBE reviewed the methodology and results on August 16 –17, 2001 and reported to the SBE Ad Hoc Committee on September 7, 2001. The panel concluded that the technical process used to compute the equating study was sound. The Ad Hoc Committee of the SBE accepted the report and directed the Department of Public Instruction Accountability Services Division to proceed with the compilation of the ABCs report. With the sign-off of the technical process to equate the new mathematics test scores with the old ones, achievement level cut scores also were revised and the new achievement levels were used in the ABCs results for 2000-2001.
On January 9, 2002, the State Board of Education voted to continue to use the equating results for another year. Therefore, both pre and post mathematics scores were converted from the 2nd edition scale to the 1st edition mathematics scale in order to implement the existing growth model for the 2001-02 ABCs.
Parameters and Other Constants Used in the ABCs End-of-Grade Growth Model for 2001-2002
Parameters for End-of-Course
Prediction Formulas
IRP = Index of Reading Proficiency
Centering Means for Proficiency Indexes*
Constants Used in the ABCs Growth Model for Other Components
How to Include the North Carolina Alternate Assessment Portfolio (NCAAP) and the North Carolina Alternate Assessment Academic Inventory (NCAAAI) Results in a School Performance Composite NCAAP. The NCAAP has four domains: Communication, Personal Home Management, Community, and Career/Vocational. Each domain contains student tasks that are individually read and scored (0-4) by two independent readers. The Domain Score is the sum of the task scores received from two readers, or in the case of multiple tasks within a domain, the average of the task scores each equaling 8 possible points. The sum of the domain scores is the total portfolio score. Using the total portfolio score, a student is given an achievement level. This achievement level is used in calculating the performance composite. The student will count in the denominator for the performance composite and if the student attains a level III or IV on the portfolio, then the student counts in the numerator. NCAAAI. The North Carolina Alternate Assessment Academic Inventory is an alternate assessment for students with disabilities who are following the North Carolina Standard Course of Study. It provides a mechanism for teachers to evaluate a student based on student performance on competencies in the specific content areas (reading, mathematics, and writing). If a student is assessed on grade level (that is, at the assigned grade level designated in SIMS/NCWISE), the student receives an Achievement Level of I, II, III, or IV as appropriate. If a student is assessed off-grade level (that is, any grade level below the assigned grade level designated in SIMS/NCWISE), the student receives an Achievement Level of I or II as appropriate. The student cannot receive an achievement level higher than II. Students who participate in the ABCs using the NCAAAI, are counted in the Performance Composite at their respective school. Example: For a K-5 Elementary School Performance Composite with NCAAP and NCAAAI Scores The performance composite for any school is the total of the number of scores at or above Level III (or at or above grade level) in each subject included in the ABCs model, divided by the total number of valid scores. The performance composite is reported as a percentage. As in the example below, a K-5 school will typically include Reading, Mathematics, the NCAAP and the NCAAAI in the performance composite. The first row represents the total number of scores at or above Level III (numerator) as reported for each subject area and for the NCAAP and NCAAAI. In the second row, the total number of valid scores (denominator) in each subject area or the number of students who participated in the NCAAP or NCAAAI at this particular school are included. To obtain the performance composite, divide the total number scores at or above Level III by the total number of valid scores. Multiply the product by 100 and round to the nearest tenth to yield the performance composite score for this school. Example.
Complete details on the NCAAAI
are located at the following web address: The web address for the NCAAP:
North Carolina End-of-Grade
Tests
Achievement Level Descriptions: Level I: Students performing at this level do not have sufficient mastery of knowledge and skills in this subject area to be successful at the next grade level. Level II: Students performing at this level demonstrate inconsistent mastery of knowledge and skills in this subject are and are minimally prepared to be successful at the next grade level. Level III: Students performing at this level consistently demonstrate mastery of grade level subject matter and skills and are well prepared for the next grade level. Level IV: Students performing at this level consistently perform in a superior manner clearly beyond that required to be proficient at grade level work.
Top of Page
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
North Carolina Department of
Public Instruction 301 N. Wilmington St. Raleigh, NC 27601 Phone: 919-807-3300 |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
 |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
