[National Assessment Governing Board (NAGB) Achievement Levels. Interim Letter Report.

N

20

Academic Achievement; Academic Standards; Data Analysis; Government Role; Letters (Correspondence); National Programs; Program Evaluation; Reliability; Research Methodology; Research Reports; Standard Setting; Testing Programs; Validity

This letter is an interim response to the October 7, 1991 request from the Committee on Education and Labor and the Subcommittee on Elementary, Secondary, and Vocational Education of the House of Representatives asking for a review of the National Assessment Governing Board (NAGB) achievement levels for the National Assessment of Educational Progress (NAEP). It summarizes findings and conclusions to date. The General Accounting Office (GAO) reviewed the record of the development and results of the NAGB's level-setting approach from early 1989 to the present and interviewed government and testing contractor (Educational Testing Service) staff. In examining the achievement levels, the GAO found problems of procedure, reliability, validity, and reporting. Problems seem sufficiently serious to preclude continuing the application of the achievement levels. It is contended that too much is uncertain to support the NAGB's decision to organize the reporting and analysis of the 1992 NAEP results around achievement levels. Even with continued improvement in the item judgment method of setting standards, the approach does not seem suitable. The NAGB apparently conducted only a cursory review of alternative methods for setting standards. Recommendations are made for realigning the functions of the NAGB and the National Center for Education Statistics or for strengthening the capacity of the NAGB to make sound technical decisions. Three enclosures provide additional information about the NAEP and the achievement levels. (SLD)

1992

ED342821