research designs for the 17 prOects along with an overview of - Eric

214 Pages·2007·4.37 MB·English

DOCUMENT RESUME,
CG 016 303
ED 222 835
Gottfredson, Gary D., Ed.
AUTHOR
First Interim
The School Action Effectiveness Study:
TITLE
Report. Report No. 325.
Social
Johns Hopkins Univ., Baltimore, Md. Center for
INSTITUTION
Organization of Schools.
National Inst. for Juvenile Justice and Delinquency
SPONS AGENCY
Prevention (Dept. of Justice/LEAA), Washington, D.C.;
National Inst. of Education (ED), Washington, DC.
Apr 82
PUB DATE
80-JN-AX-0005; NIE-G-80-0113
GRANT
219p.
NOTE
Reports - Research/Technical (143)
PUB TYPE
MF01/PC09 Plus Postage.
EDRS PRICE
Behavior Change; *Delinquency prevention;
DESCRIPTORS
*Demonstration Programs; Educational Objectives;
*Evaluation Methods; *Federal Programs; Formative
Evaluation; Models; *Nontraditional Education;
Program
Program Development; *Program Effectiveness;
Evaluation; Secondary Education
ABSTRACT
This report focuse-s on the School Action
of the Office of
Effectiveness Study (SAES), the national evaluation
Alternative Education
Juvenile Justice.and Delinquency Prevention's
activities, successes, and
Program. It presents the history, start-up
17 action projects.
problems encountered during the first year of the
discusses general topics that
In the first section, the report
discusses the record of
summarize the entire evaluation. Chapter 2
and delinquency
research in creating organizational change
lead to rigorOus
prevention. Chapter 3 summarizes the conditions that
the program
summative evaluation, while Chapter 4 describes
Chapter 5 presents
development evaluation model used in the study.
in SAES to provide diagnostic information
some of the measures used
provides a brief sketch of the
and to measure delinquency. Chapter 6
with an overview of the
research designs for the 17 prOects along
provides a narrative
formative evaluation results. The second section
index, and a
description of each project. A subject index, a name
project descriptions
section containing abstracts of the narrative
are appended. (JAC)
********************************************************************
*
be made
Reproductions supplied by EDRS are the best that can
*
*
from the original document.
*
***********************************************************************
EFFECTIVENBSS STUDY:
THE SCHOOL ACTION
FIRST INTERIM REPORT
Editor
Gary D. Gottfredson,
Report No. 325
April 1982
"PERMISSION TO REPRODUCE THIS
MATERIAL AS BEEN GRANTED BY
U.S. DEPARTMENT OF EDUCATION
NATIONAL INSTITUTE OF EDUCATION
/
I
INFORMATION
EDUCATIONAL RESOURCES
CENTER (ERIC)
reproduced as
Li This document has been
organization
received from the person or
originating it.
X' Minor changes have been made to improve
TO THE EDUCATIONAL RESOURCES
reproduction quality.
INFORMATION CENTER (ERIC)."
in this docu-
Points of view or opinions stated
official NIE
ment do not necessarily represent
position or policy.
from
Grant No. 80-JN-AX-0005
in large part by
This report is sponsored
Prevention,
and Delinquency
for Juvenile Justice
the National Institute
NIE-G-80-0113 from
by Grant No.
Justice, and in part
U.S. Department of
The
Department of Education.
of Education, U.S.
the National Institute
position or policy
necessarily represent the
opinions expressed do not
of
for Social Organization
Published by the Center
of either agency.
center by funds
research and development
Schools, supported in part as a
of
Education, Department
National Institute of
from the,United Stated
Education.
Organization of Schools
Center for Social
University
The Johns Hopkins
3505 North Charles Street
21218
Baltimore, MD
for the Handicapped
assembled by the Centers
Printed and
Silver Spring, MD
2
Introductory Statement
a
primary objectives:
Organization of Schools has two
The Center for Social
affect their students,
knowledge of how schools
to develop a scientific
and organi-
develop better school practices
and to use this knowledge to
zation.
The
objectives.
five programs to achieve its
The Center works through
of
applies the basic theories
Studies in School Desegregation program
of
study the internal conditions
social organization of schools to
policies,
of alternative desegregation
desegregated schools, the feasibility
other equity issues
school desegregation with
and the interrelations of
The School Organization program
desegregation.
such as housing and job
task structures,
authority-control structures,
is currently concerned with
It has produced a
in schools.
reward systems, and peer group processes
Student
schools, has developed
large-scale study of the effects of open
in
for teaching various subjects
Team Learning Instructional processes
computerized system
and has produced a
elementary and secondary schools,
Career
The $chool Process and
monitoring.
for school-wide attendance
post
transition& from high school to
Development program is studying
development of
role of schooling in the
secondary institutions and the
The
actualization of labor market outcomes.
career plans and the
examining
Environments program is
Studies in Delinquency and School
and individual
environments, school experiences,
the interaction of school
and later-life delinquency.
characteristics ln relation to in-school
that
yielloclicationResearcigstl program
The Center also supports 4
conduct and publish
talented young reeearchers to
provides opportunities for
and
the participation of women
significant research, and to encourage
minorities in research on education,
School Environ-
in Delinquency and
This report, prepared by the Studies
program's national
the interim results of the
ment& program, describes
Preventionts
Justice and Delinquency
evaluation of the Office. of Juvenile
(QJJDP's) Alternative Education Program,
Staff
10221_AstionAttLissiteatuAsvgx_isteff
19.21.1EJAL-12Sill_gt2211121i20-2L-1022i1
Project Director
Gary D. Gottfredson,
Scientist
Associate Research
Denise C. Gottfredson,
Research Scientist*
Michael S. Cook, Associate
Analyst
Deborah K. Ogawa, Research
Research Analyst
Donald E. Rickert, Jr.,
Technician
Helene Kapinos, Data
Assistant
Claire Skarda, Editorial
Assistant
Stuart Gavurin, Research
Assistant
Laura Gugerty, Research
Assistant
Richard D. Joffe, Research
Assistant
Robert Kirchner, Research
Hazel Kennedy, Secretary
Barbara Hucksoll, Secretary
Lois Hybl, Secretary
Iglial Action Research Center
of Formative Evaluation
J. Douglas Grant. Director
of Formative Evaluation**
Carol Yamasaki. Coordinator
Associate***
Richard Carlton, Research
Associate
Deborah Daniels, Research
Associate
Jane S. John, Research
Eeggui_ftglitilAffiggrs
Justice and
National Institag_for Juvenile
Delinguengx Pleyeation
Barbara Tatem* Project Officer
laSiAl-Entali1AW11201-Qttic2-91-AgInill-Agnic2
and Delinguencv Ptetention
Monserrate Diaz, Project Officer
Cecilia Smith, Project Officer
1110 OM.
* Joined the staff January. 1982.
** Left the staff January. 1982.
1981.
*** Left the staff September.
Staff
Ttarling_RiviSi2a/ALLiE2_21
Technkcal Assilitagg and
4uvenile_Junice Ang_Delinggincl_triltnnin
a
Officer
Connie Walton, Project
National Institgle_ILEggcation
Officer
Bruce Mckenzie-Haslam, Project
Technical Assistance
Polaris Researn_ang_Develoompt
Noel Day.- Project Co-Director
David Sheppard* Project Co-Director
Dian Overbey* Project Associate
Preface
Preface
divergent interests com-
The School Action Effec-
The approach taken
pl.etely.
tiveness Study (SAES) is the
in this project is to
national evaluation of the
involve as many parties as
Office for Juvenile Justice
possible in the evaluation
and Delinquency Prevention's
When the sub-
enterprise.
(0JJOP's) Alternative Educa-
jects of evaluation are col-
The study is
tion-Program.
laborators in its develop-
rooted in the perception
ment, the tensions may be
that neither social action
evalua-
reduced somewhat and
nor research will make prom-
tion feedback may be more
gress without the collabora-
helpful in project develop-
tive effort of project
ment.
implementers and research-
Together, these two
ers.
We have been partially
and
groups can create change
successful with this
examine its consequences in
approach, but complete suc-
settings where answers are
cess has escaped our graspo
needed and problems are
The reasons are many, and
real.
include limits on time,
talent
moneys and our own
The study is also rooted
Some of the
and energy.
in the notion that theory is
how-
more important reasons,
of
an essential ingredient
in the
ever, are differences
both program development and
perspectives of the Federal
Conse-
evaluation research.
action
sponsor, the various
quently, SAES aims to imple-
projects, and ourselves.
ment an action research
below
The Venn diagram shown
model, in which project
illustrates the problem.
implementers work together
this
The various actors in
with researchers, specifying
project have sometimes over-
theory-based research ques-
and sometimes u-
lapping
tions and designing their
nique, goals or outlooks.
aid to
Own evaluations as an
organizational self-study
and continued project devel-
opment.
Any large evaluation pro-
Mul-
ject creates tension.
tiple stakeholders, a col-
lection of agencies and
actors, and varied audiences
with only partially overlap-
pingand sometimes diver-
gentinterests present what
willf always be a challenge
Because
for evaluators.
evaluation necessarily
involves critical and skep-
tical examination, there may
be no way to mesh these
Preface
At the
for each project.
The primary sponsor
Ls the
root of this bias
(0JJDP) is first and fore-
conviction that the public
most interested in delin-
deserves educational and
It wants
quency prevention.
delinquency prevehtion
to evaluate demonstration
efforts whose effectiveness
projects--involving youth
has been or can be demon-
participation, organiza-
This is especially
strated.
tional innovations, alterna-
demonstra-
true in a Federal
and
tive reward structures,
the
tion program, in which
individualized instruction
expenditure of public funds
(among other interven-
evalua-
is justified by the
tions)--creating institu-
effort
tion of the resulting
tional changes that may pre-
develop and
to learn how to
The
vent delinquency.
implement similar projects
action projects sometimes
In short. we
effectively.
place less emphasis on de-
the
have little sympathy for
They
linquency prevention.
arguments that evaluation
in educa-
may be interested
diverts effort from or
tional achievement or
detracts from programmatic
truancy. or in -.ontinuing a
work in these circumstances.
exis-
program already in
of
Because the effectiveness
Delinquency preven-
tence.
is un-
these action projects
tion is sometimes related to
known, the only ethical
these interests even when
it
evalua-
course is systematic
As
is not a primary aim.
In addition, we
tion.
evaluators, we are inter-
evaluation
assume that the
ested in assisting in the
the
of a project's efforts,
development of effective
generation of knowledge
critically assess-
projects
of a
about the consequences
ing project effectiveness,
is
project's interventions,
and contributing to know-
project
a part of sound
Our critical per-
ledge.
administration and implemen-
spective often results in
tation,
approaches that diverge from
the methods action projects
We have not assumed that
find most comfortable.
will
evaluating this program
These projects often assume
be easy, and we eve grati-
their interventions to be
fied that we have been as
effective and prefer not to
successful as we have been
devote energy to the criti-
ambitions
in translating our
cal and sometimes painful
The excellent
into reality.
scrutiny of those activi-
rapport and cooperation we
ties--especially in areas
have with the Federal agen-
that are of interest to the
with most
cies involved, and
sponsor and to science but
have
of the action projectst
that may be seen as tangen-
in this suc-
been critical
tial or even irrelevant by
cess.
action project managers.
This interim report sum-
Our bias when encounter-
have
marizes some of what we
ing this tension has been to
of
learned in the first year
push as much as possible for
(For the most
the SAES.
evalua-
a sound and thorough
tion, in ways appropriate
Preface
be--authoritative and con-
the
part the report covers
They
clusive statements.
period August 1980-August
provide
are intended to
Most action projects
1981.)
for pro-
information useful
their
began implementing
ject development.
dur-
interventions sometime
and all
ing the past year,
Effective projects
out of
17 projects are now
incorpo-
develop over time,
their planning phases.
from their
rating feedback
Start-up activities are now
and those
own observations
Action projects
behind us:
become
of evaluators to
to
have had_Up to a year
Provided that a
stronger.
become-arccustomed to our
a suf-
project operates over
expectations and methods,
time, with
ficient period of
and we have had up to a year
of goals and
a stable set
with
to acquaint ourselves
with
guiding values, and
We are
the action projects.
guide
evidential pressure to
is
pleased that evaluation
alternative
the choice among
becoming routinized as an
evolution
activities, an
well-understood
expected and
eventu-
that approaches an
of
part of the activities
program
ally stable "climax"
most projects.
(Tharp C
may be expected
This
Gallimore. tied.).
second
We are entering a
directed
report is therefore
with
year of interaction
imple-
primarily at project
In many
these 17 projects.
includ-
menters and others.
designs
cases, evaluation
its technical
ing OJJOP and
those
that are stronger than
who
assistance contractor,
possible in the first year
fostering
have a stake in
implemented.
are now being
in
project development or
eval-
The next phase of this
planning new initiatives.
uation should be more pro-
how-
It is also directed,
ductive in assessing project
community of
ever, to the
effects on students and
in the
scientists engaged
schools.
evaluation of social pro-
meth-
The evaluation
grams.
Everyone is impatient for
in the
ods being developed
information about a proj-
Effectiveness
School Action
This report
ect's "impact."
interest
Study should be of
look for
is not the place to
and students
to evaluators
Here
impact assessments.
of evaluation.
information
you will find
about a project's history,
Acknowledgments
and
its start-up activities,
problems
its successes and
is
This interim report
its
in implementation during
people's
the product of many
You will find
first year.
it is
contributions, aril
diag-
some organizational
only
possible to describe
about
noses, and some ideas
contributions
some of these
Occa-
improving projects.
Initial discussions
here.
sionally, you will find pre-
proposal
leading to the
liminary attempts to assess
J.
involved a small group:
These pre-
effectiveness.
suggested
Douglas Grant
liminary attempts are not--
-nor are they intended to
Preface
.
Yamasaki
Jr.. and Carol
development
using a program
hard with
worked long and
incorporating
approach and
personnel in
action project
a
critical benchmarks as
visits, and
workshops, site
model, Den-
component of the
for
to prepare
on the phone
conceptual-
ise Gottfredson
evolve Pro-
tree surveys, to
system.
ized the information
Evaluation
gram Development
prepared a
Lenore Campbell
project
plans, and to draft
very
preliminary workplan on
This report is
narratives.
Gary Gott-
short notice. and
of their
possible because
theor-
fredson provided the
wisdom.
help and practical
etical and methodological
project.
rationale for the
Doris Coa-
Ann Birdseye,
heavily
This rationale was
Dilligard,
xum, Barbara
of or
influenced by the work
Roland Pat-
Hilda Gutierrez,
discussions with LaMar
N. Whitney,
terson, Edward
Joseph
Empey, Lee Sechrest,
Irwin,
Glen Bader, Hilda
and
Weis, David Hawkins,
Charles Almo,
Richard Smith,
oth-
Travis Hirschi, among
Thomas Cor-
Herman S,teptoe,
The general strategy
ers.
Lewis, Preston
coran, Mary
the intellec-
owes much to
Vactor, Anita
Elrod, Vanita
action
tual father of
Bailey, Ana-
Batisti, David
and is
research. Kurt Lewin,
Tyrone &aals.
dia Andrews.
example of
influenced by the
Ciorah Montes,
Phyllis BeIz,
in his
Don M. Gottfredson
Chardon-Zavala,
Ivelissi M.
justice
work with criminal
Phillip
Craig Overbeck,
of
The evolution
agencies.
Lopez,
Cano, Cristobal
much to
the POE model owes
Cohen,
Roberto Duran, Nancy
Carol Yama-
discussions with
Prentice
Marilyn McKnight,
reactions of
saki, and the
Aaron, Dave
Deadrick, Melvin
Research Cen-
Social Action
Joe Nathan,
Reiss, Lia Tepp,
in early staff
ter personnel
Nic
Mark Gilbert-Cougar,
well
training sessions, as
Wisotzkey,
Cooper, Sally
persuasive
as to the the
William Kott-
Sonny Luster,
ecology of
account of the
Leighty, Darlene
man, Tom
development and
program
Pat
Amyotte, Roy Mahoney,
by Roland
research provided
Andrea
Kenney, Marvin Dunn,
Gallimore.
Tharp and Ronald
Dunn,
Loring, Dolores
Pat Gahl.
Mizhael Coleman,
based in
This project is
Martin were cri-
and Dorothy
contributions of
part on the
the real
tical links between
and
thousands of students
evaluation.
world and the
their
teachers who shared
would be
whithout whom there
schools
views about their
Action Effective-
no School
information
and provided
They provided
ness Study.
We hope
about themselves.
guided
the theories that
rewarued for
they will be
instrument and
much of the
their
their help by having
devel-
scale construction,
acted upon.
views heeded and
plans using
oped project
and
evaluation terminology,
Deborah
Richard Carlton,
projects and
made the action
Gottfred-
Daniels, Denise C.
data collection go.
John, Deborah
son, Jane St.
E. Rickert.
K. Ogawa, Donald
Preface
the way for thiF project to
Denise C. Gottfredson
proceed, and helped to
performed the superhuman
resolve nearly countless
task of coordinating and
problems along the way.
managing all of the data.
Gary Gottfredson and Donald
This report is edited by
E. Rickert designed the sur-
Gary D. Gottfredson, who was
vey instruments. and Deborah
assisted by Claire Skarda.
K. Ogawa and Donald E. Rick-
Ms. Skarda devoted many
ert put in many 18-hour days
weeks to turning our nearly
analyzing data under incred-
hopeless original manu-
Compu-
ible time pressure.
scripts into grammatical and
tation at the Center for
Gary D.
readable chapters.
Social Organization of
Gottfredson wrote the chap-
Schools is possible because
ters not_attributed to oth-
of the work of Carol Wein-
Michael Cook, Deborah
ers.
reich.
Daniels, Denise C. Gottfred-
son. Lee Sechrest. and Jane
Delbert Elliott,LaMar
-
St. John made useful com-
Enipey, Joyce Epstein, J.
ments on a dr7fft of several
Douglas Grant, -Joan Grant.
Opinions
chapters.
Travis Hirschi. and John
expresssed are the authors'
Holland gave valuable advice
or editor's, end do not
on the development of the
necessarily reflect the
student questionnaire, not
Position or policy of any
all of it taken.
Raul
agency or institution.
\
Romero translated the stu-
dent questionnaire into
Organization of this Regort
Spanish. and Dennis Dillon
of Intran Corporation vo-
The remainder of this
duced the optically scanna-
report is organized into two
ble instruments.
The first of
sections.
these discusses general top-
We were ably assisted in
ics that undergird or summa-
managing and processing the
rize the entire evaluation.
data by Stuart Gavurin,
Chapter 2 discusses the
Laura Gugerty, Richard 0.
record of research in creat-
Joffe, Helene Kapinos. and
ing organizational change
Helene
Robert Kirschner.
and delinquency prevention;
Kapinos coordinated the flow
it summarizes the weak
of an incredible amount of
nature of foregoing efforts,
Lois Hybl
information.
and argues that the defects
helped keep the project ord-
of these earlier efforts
erly despite what often
must be overcome to increase
seemed to be incessant new
the value of research and
demands; she maintained pro-
demonstration programs.
ject files, typed manu-
Chapter 3 summarizes the
scripts, and helped us to
conditions that lead to rig-
sthedule our time.
orous summative evaluation
and argues that sone common
Monserrate Diaz and Bar-
objections to creating these
bara Tatem of the Office for
conditions can be overcome.
Juvenile Justice and Delin-
Chapter 4 describes the
quency Prevention cleared
10