Assessing Accomplished Teaching: Advanced-Level Certification Programs

339 Pages·2008·1.46 MB·English

by  Committee on Evaluation of Teacher Certification by the National Board for Professional Teaching Standards (Milton D. Hakel

Assessing Accomplished Teaching
Advanced-Level Certifi cation Programs
Committee on Evaluation of Teacher Certification by the National Board for
Professional Teaching Standards
Milton D. Hakel, Judith Anderson Koenig, and Stuart W. Elliott,, Editors
Board on Testing and Assessment
Center for Education
Division of Behavioral and Social Sciences and Education
THE NATIONAL ACADEMIES PRESS 500 Fifth Street, N.W. Washington, DC 20001
NOTICE: The project that is the subject of this report was approved by the Govern-
ing Board of the National Research Council, whose members are drawn from the
councils of the National Academy of Sciences, the National Academy of Engineer-
ing, and the Institute of Medicine. The members of the committee responsible for
the report were chosen for their special competences and with regard for appropri-
ate balance.
This study was supported by Contract No. ED-04-CO-0139 between the National
Academy of Sciences and the U.S. Department of Education. Any opinions, find-
ings, conclusions, or recommendations expressed in this publication are those of
the authors and do not necessarily reflect the views of the organizations or agencies
that provided support for the project.
Library of Congress Cataloging-in-Publication Data
Assessing accomplished teaching : advanced-level certification programs :
Committee on Evaluation of Teacher Certification by the National Board for
Professional Teaching Standards / Milton D. Hakel, Judith Anderson Koenig, and
Stuart W. Elliott, editors.
p. cm.
ISBN 978-0-309-12118-7 (pbk.) — ISBN 978-0-309-12119-4 (pdf) 1.
Teachers—Certification—United States. I. Hakel, Milton D. II. Koenig, Judith
A. III. Elliott, Stuart W. IV. National Board for Professional Teaching Standards
(U.S.)
LB1771.A77 2008
371.120973—dc22
2008026490
Additional copies of this report are available from the National Academies Press,
500 Fifth Street, N.W., Lockbox 285, Washington, DC 20055; [phone] or
[phone] (in the Washington metropolitan area); Internet, http://www.nap.
edu.
Copyright 2008 by the National Academy of Sciences. All rights reserved.
Printed in the United States of America
Suggested citation: National Research Council. (2008). Assessing Accomplished
Teaching: Advanced-Level Certification Programs. Committee on Evaluation of
Teacher Certification by the National Board for Professional Teaching Standards.
Milton D. Hakel, Judith Anderson Koenig, and Stuart W. Elliott, editors. Board on
Testing and Assessment, Center for Education, Division of Behavioral and Social
Sciences and Education. Washington, DC: The National Academies Press.
The National Academy of Sciences is a private, nonprofit, self-perpetuating society
of distinguished scholars engaged in scientific and engineering research, dedicated to
the furtherance of science and technology and to their use for the general welfare.
Upon the authority of the charter granted to it by the Congress in 1863, the Acad-
emy has a mandate that requires it to advise the federal government on scientific
and technical matters. Dr. Ralph J. Cicerone is president of the National Academy
of Sciences.
The National Academy of Engineering was established in 1964, under the charter
of the National Academy of Sciences, as a parallel organization of outstanding en-
gineers. It is autonomous in its administration and in the selection of its members,
sharing with the National Academy of Sciences the responsibility for advising the
federal government. The National Academy of Engineering also sponsors engineer-
ing programs aimed at meeting national needs, encourages education and research,
and recognizes the superior achievements of engineers. Dr. Charles M. Vest is presi-
dent of the National Academy of Engineering.
The Institute of Medicine was established in 1970 by the National Academy of
Sciences to secure the services of eminent members of appropriate professions in
the examination of policy matters pertaining to the health of the public. The Insti-
tute acts under the responsibility given to the National Academy of Sciences by its
congressional charter to be an adviser to the federal government and, upon its own
initiative, to identify issues of medical care, research, and education. Dr. Harvey V.
Fineberg is president of the Institute of Medicine.
The National Research Council was organized by the National Academy of Sci-
ences in 1916 to associate the broad community of science and technology with the
Academy’s purposes of furthering knowledge and advising the federal government.
Functioning in accordance with general policies determined by the Academy, the
Council has become the principal operating agency of both the National Academy
of Sciences and the National Academy of Engineering in providing services to the
government, the public, and the scientific and engineering communities. The Coun-
cil is administered jointly by both Academies and the Institute of Medicine. Dr.
Ralph J. Cicerone and Dr. Charles M. Vest are chair and vice chair, respectively, of
the National Research Council.
www.national-academies.org

COMMITTEE ON EVALUATION OF TEACHER
CERTIFICATION BY THE NATIONAL BOARD FOR
PROFESSIONAL TEACHING STANDARDS
MILTON D. HAKEL (Chair), Department of Psychology, Bowling Green
State University
JULIAN BETTS, Department of Economics, University of California, San
Diego
MARK DYNARSKI, Mathematica Policy Research, Inc., Princeton, New
Jersey
ADAM GAMORAN, Department of Sociology, University of Wisconsin,
Madison
JANE HANNAWAY, The Urban Institute, Washington, DC
RICHARD INGERSOLL, Graduate School of Education, University of
Pennsylvania
MICHAEL T. KANE, National Conference of Bar Examiners, Madison,
Wisconsin
DEIRDRE J. KNAPP, Human Resources Research Organization,
Alexandria, Virginia
SUSANNA LOEB, School of Education, Stanford University
JAMES (TORCH) H. LYTLE, Graduate School of Education, University
of Pennsylvania
C. FORD MORISHITA, Clackamas High School, Clackamas, Oregon
LYNN W. PAINE, College of Education, Michigan State University
NEIL J. SMELSER (Emeritus), University of California, Berkeley
BRIAN STECHER, RAND Corporation, Santa Monica, California
ANA MARIA VILLEGAS, College of Education and Human Services,
Montclair State University
DOROTHY Y. WHITE, Department of Mathematics and Science
Education, University of Georgia
KAREN K. WIXSON, School of Education, University of Michigan
JUDITH A. KOENIG, Study Director
STUART W. ELLIOTT, Senior Program Officer
ALIX BEATTY, Senior Program Officer
VIOLA HOREK, Manager, Program Operations
TERESIA WILMORE, Senior Program Assistant
v
BOARD ON TESTING AND ASSESSMENT
LYLE F. BACHMAN, Department of Applied Linguistics, University of
California, Los Angeles
STEPHEN B. DUNBAR, College of Education, University of Iowa
DAVID J. FRANCIS, Department of Psychology, University of Houston
ANDREW J. HARTMAN, Independent Consultant, Denver, Colorado
MICHAEL NETTLES, Educational Testing Service, Princeton, New
Jersey
JAMES W. PELLEGRINO, Department of Psychology, University of
Illinois, Chicago
DIANA C. PULLIN, School of Education, Boston College
STUART W. ELLIOTT, Director
vi
Acknowledgments
The Committee on Evaluation of Teacher Certification by the Na-
tional Board for Professional Teaching Standards (NBPTS) was formed in
response to legislation passed by the U.S. Congress asking the National
Academies to develop a framework for evaluating programs that offer
advanced-level certification to teachers and to apply that framework in an
evaluation of the impacts of the NBPTS. The committee began its work in
September 2005 and, over the course of the next 30 months, held six meet-
ings, commissioned numerous papers and analyses, and collected a variety
of information as part of its evaluation. The committee’s work benefited
tremendously from the contributions of many people, and the committee is
grateful for their assistance and support.
The work was overseen by the U.S. Department of Education with
management handled by Susan Sanchez. We thank Susan for her expert
guidance and quick response to our many questions.
Over the course of the project, we made many requests of NBPTS staff,
including numerous requests for information, documentation, research re-
ports, and data files; visits to their office and meetings with their staff; and
presentations at our committee meetings. We extend our heartfelt thanks to
Mary Dilworth for her diligence and patience in responding to our exten-
sive inquiries. Mary gave generously of her time, and we thank her for all
the information she provided in response to the committee’s questions. We
also thank NBPTS president Joseph Aguerrebere and staff members Lillie
Saunders and Joan Auchter for the valuable information they provided.
vii
viii ACKNOWLEDGMENTS
During the course of this evaluation, the Educational Testing Service
(ETS) served as the test development contractor to NBPTS, and we wish to
acknowledge the assistance of several ETS staff members. We are grateful
to Mari Pearlman for recapturing for us the history of the development of
the assessment program and to Drew Gitomer for providing his review of
research on the NBPTS program. We especially thank Steve Schreiner for
providing an overview of the assessment scoring process and for guiding
us through a review of sample portfolio materials and videotapes submit-
ted by teachers. This review contributed greatly to our understanding of
the assessment and the requirements that teachers must meet to earn board
certification.
Our evaluation drew on work conducted for us by several researchers.
Our initial review of the research base revealed a large number of studies
focused on the impact of board-certified teachers on their students’ achieve-
ment. The findings from these studies presented a complex set of somewhat
conflicting results. We thank Henry Braun and Paul Holland, both at ETS,
for their insights about these studies, which helped us to sort out details
of research methodologies and statistical analyses. We decided to conduct
a full-scale review of these studies combined with work to identify and
conduct additional analyses to help resolve the differing findings. The re-
searchers who assisted us in this endeavor greatly contributed to our evalu-
ation, and we are indebted to each of them, including Doug Harris at the
University of Wisconsin–Madison, Helen Ladd at Duke University, Daniel
McCaffrey at the RAND Corporation, Steven Rivkin at Amherst College,
and Tim Sass at Florida State University. We are especially grateful to Tim
Sass for the many additional analyses he conducted for us. We also thank
Jon Fullerton and Tom Kane at Harvard University, Steve Raudenbush at
the University of Chicago, and Bill Sanders at the SAS Institute, Inc., for
their presentations at our third committee meeting and their insights about
this body of work.
We also commissioned a psychometric review of the NBPTS assess-
ments. This was an extensive task, given that NBPTS awards certification
in 25 areas, each using different assessments. We are indebted to Teresa
Russell at the Human Resources Research Organization (HumRRO), who
led this work for us. Her thorough and painstaking efforts to review a
multitude of documents and summarize the psychometric characteristics of
the assessments were a tremendous resource to the committee.
In addition, we commissioned analyses of NBPTS participants and
comparisons of their characteristics with those of nonparticipating teachers.
David Perda, a doctoral student at the University of Pennsylvania, assisted
us with these analyses. We sincerely appreciate his thorough and careful
ACKNOWLEDGMENTS ix
analyses and are particularly grateful for his responsiveness to our numer-
ous follow-up requests for additional analyses.
To learn more about the history of the NBPTS program, we conducted
interviews of a number of key people involved with its early development.
We are very grateful for the time these individuals spent with us and
the wealth of information they provided. We thank former NBPTS staff
members Joan Baratz-Snowden at the American Federation of Teachers
(former vice president of assessment and research); Chuck Cascio at ETS
(former director of test development); Ann Harman at Harman and Asso-
ciates (former director of research); Jim Kelly, retired, who served as the
first president of NBPTS; and David Mandel at Carnegie-IAS Commission
on Mathematics and Science Education (former vice president for policy
development at NBPTS). We are also grateful for the information provided
by Sally Mernissi (former vice president and corporate secretary), who died
in January 2006. We also thank the following individuals who assisted the
NBPTS staff with development of the program: Lloyd Bond at the Carnegie
Foundation (former director of the NBPTS Technical Analysis Group);
Emerson Elliott at the National Council for Accreditation of Teacher Edu-
cation; Mary Futrell at George Washington University (former president of
the National Education Association and member of the NBPTS board of
directors); Lee Shulman at the Carnegie Foundation (former consultant to
the NBPTS board of directors); Gary Sykes at Michigan State University
(former consultant to the NBPTS board of directors); and Suzanne Wilson
at Michigan State University (former consultant to the NBPTS board of
directors). We also thank Joshua Boots at the American Board for Cer-
tification of Teacher Excellence (ABCTE) and Kathy Madigan, formerly
at the ABCTE, for the information they provided about their assessment
program.
We sought to gather information from teachers and teacher educa-
tors with regard to their experiences with and perceptions of the NBPTS
program. Members of the Teacher Advisory Council of the National
Research Council (NRC) spoke with us on numerous occasions. In addi-
tion four teachers and teacher educators attended our third meeting: Sara
Eisenhardt with the Cincinnati school system, Maxine Freund and Mary
Futrell at George Washington University, and Carol Matern with the
Indianapolis public schools and Indiana University–Purdue University,
Indianapolis. We thank these individuals for their insightful comments
about the program.
During the course of this project, we spoke with and heard presenta-
tions from numerous individuals who conducted research on the NBPTS
program. We are particularly grateful to the following researchers for their