Table Of ContentDOCUMENT RESUME
CE 078 594
ED 448 264
McGrail, Loren, Ed.; Purdom, Laura, Ed.; Schwartz, Rick,
AUTHOR
Ed.; Simmons, Alison, Ed.
Adventures in Assessment: Learner-Centered Approaches to
TITLE
Assessment and Evaluation in Adult Literacy, 1991-1998.
INSTITUTION
System for Adult Basic Education" Support, Boston, MA.
Massachusetts State Dept. of Education, Boston. Bureau of
SPONS AGENCY
Adult Education.
PUB DATE
1998-00-00
881p.; Published annually. This document consists of the
NOTE
first eleven issues of this journal.
SABES/World Education, 44 Farnsworth Street, Boston, MA
AVAILABLE FROM
02210-1211. Tel: 617-482-9485. Web site:
http://sabes.org/resource.htm.
Collected Works
Serials (022)
PUB TYPE
Adventures in Assessment; v1-11 Nov 1999-Win 1998
JOURNAL CIT
MF06/PC36 Plus Postage.
EDRS PRICE
Adult Basic Education; *Adult Literacy; Adult Students;
DESCRIPTORS
*Alternative Assessment; Annotated Bibliographies; Check
Lists; Diagnostic Tests; Educational Practices; Educational
Research; *Evaluation Methods; LeaLiiiily Activities;
*Literacy Education; Miscue Analysis; Nursing Homes; Program
Evaluation; Questionnaires; Records (Forms); Screening
Tests; Student Educational Objectives; *Student Evaluation;
Theory Practice Relationship; Workplace Literacy; Writing
Skills
Massachusetts; *Student Centered Assessment
IDENTIFIERS
ABSTRACT
"Adventures in Assessment" is an annual journal of the
system for Adult Basic Education Support (SABES), a comprehensive training
and technical assistance initiative for adult literacy educators and
programs. SABES accomplishes this goal through staff and program development
workshops, consultation, mini-courses, mentoring, peer coaching, and other
training activities provided by five Regional Support Centers located at
community colleges throughout Massachusetts. The first three volumes present
a comprehensive view of the state of practice in Massachusetts through
articles written by adult literacy practitioners. Volume 1,
"Getting
Started," includes start-up and intake activities; volume 2,
"Ongoing,"
shares tools for ongoing assessment as part of the learning process; volume
3, "Looking Back, Starting Again," focuses on tools and procedures used at
the end of a cycle or term, including self, class, and group evaluation by
both teachers and learners. Volume 4 covers a range of interests, and volume
"The Tale of the Tools," is dedicated to reflecting on Component 3 tools
5,
of alternative assessment. Volume 6, "Responding to the Dream Conference," is
dedicated to responses to volumes 1-5. Volume 7, "The Partnership Project,"
highlights writings from a memtoring project for practitioners intersted in
learning about participatory assessment. Volumes 8-11 cover a range of
topics, including educational reform, workplace education, learner
involvement in asseessment, etc.
(MN)
Reproductions supplied by EDRS are the best that can be made
from the original document.
Adventures in Assessment
Learner-Centered Approaches to Assessment
and Evaluation in Adult Literacy
1991-1998
-C A,
rvrf
A -1
+
1:10
ysLem 101 fiuulL 1_3 aw.1/4., Lauticadon
Boston, Massachusetts
Funded by the Massachusetts State Department
of Education
U.S. DEPARTMENT OF EDUCATION
PERMISSION TO REPRODUCE AND
Office of Educational Research and Improvement
DISSEMINATE THIS MATERIAL HAS
EDUCATIONAL RESOURCES INFORMATION
BEEN GRANTED BY
CENTER (ERIC)
This document has been reproduced as
received from the person or organization
L. \Ntolltab
originating it.
Minor changes have been made to
improve reproduction quality.
TO THE EDUCATIONAL RESOURCES
Points of view or opinions stated in this
INFORMATION CENTER (ERIC)
document do not necessarily represent
1
official OERI position or policy.
SABES
BEST COPY AVAILABLE
0
Learner-centered
approaches
to assessment &
evaluation
in adult literacy
Volume 1
Getting Started
November 1991
Funded by the Massachusetts Department of Education
3
,
s 8315
.1..
SABES is the System for Adult Basic Education Support, a comprehensive training and technical
assistance initiative for adult literacy educators and programs. Its goal is to strengthen and enhance
literacy services and thus to enable adult learners to attain literacy skills. SABES accomplishes this
goal through staff and program development workshops, consultation, mini-courses, mentoring and
peer coaching, and other training activities provided by five Regional Support Centers located at
community colleges throughout Massachusetts.
L.
SABES also maintains an adult literacy Clearinghouse to collect, evaluate, and disseminate ABE
materials, curricula, methodologies, and program models. Each of the five SABES Regional Support
L
Centers similarly offers program support and a lending library. SABES maintains an Adult Literacy
Hotline, a statewide referral service which responds to calls from new learners and volunteers. The
Hotline number is 1-800-447-8844.
The SABES Central Resource Center, located at World Education, publishes a statewide quarterly
newsletter, "Bright Ideas," and journals on topics of interest to adult literacy prufessiunuiS, .yucli
this volume of "Adventures in Assessment."
The first three volumes of "Adventures in Assessment" present a comprehensive view of the state of
practice in Massachusetts through articles written by adult literacy practitioners. Volume 1, Getting
Started, included start-up and intake activities. Volume 2, Ongoing, shares tools for ongoing
assessment as part of the learning process; Volume 3, Looking Back, Starting Again, focused on tools
and procedures used at the end of a cycle or term, including self class, and group evaluation by both
teachers and learners. Volume 4 covered a range of interests. Volume 5, Tale of the Tools was
dedicated to reflecting on component 3 tools of alternative assessment. Volume 6, Responding to the
Dream Conference, was dedicated to responses to Volumes 1-5. Volume 7, The Partnership Project,
L.
highlighted writings from a mentoring project for practitioners interested in learning about
participatory assessment. Volume 8 covered a range of topics from education reform to learner
involvement in assessment. Volume 9 looked at assessment in a volunteer program, native language
literacy, ESOL programs, and evaluations in workplace education programs.
We would like to see your contribution, especially if you are doing native language literacy or low-
level ESOL literacy. Contact editor, Alison Simmons, to discuss your submission.
Permission is granted to reproduce portions of this journal. We request appropriate credit be given
to "Adventures in Assessment" and to the author.
"Adventures in Assessment" is free to DOE funded Massachusetts programs; out-of-state requests
will be charged a nominal fee, at cost. Please write to, or call:
Alison Simmons
L _
SABES Central Resource Center
World Education
44 Farnsworth Street
Boston, MA 02210-1211
617-482-9485
4
Learner-centered
approaches
to assessment &
111.
evaluation
in adult literacy
Volume 1
Editor: Loren Mc Grail
Assistant Editor: Laura Purdom
ASSESSMENT
November 1991
Table of Contents
page ii
Introduction
Assessment Issues: Research and Practice
page 1
by Loren Mc Grail
Partners in Evaluation: Evaluating the South Cove Manor
Nursing Home Workplace Education Program with Participants
by Johan Uvin
page 5
Getting in Touch: Participants' Goals and Issues
by Lucille Fandel
page 11
Read/Write/Now Adult Learning Center Assessment Adventures
by Janet Kelly
page 17
"Down and Dirty" Miscue Analysis
by Lindy Whiton
page 31
The Education Goals Assessment Packet
by Martha Germanowski
page 33
Alternative Assessment: An Annotated Bibliography (excerpt)
by Don Robishaw, ed.
page 37
U
Introduction
However, a tool kit of assessment procedures as
"Constructing new images of adults--im-
ages built on assumptions of dignity and
suggested by Hemmendinger" and Auerbach' might
competence, of literacy as reflective and
include activities around the following three com-
self-critical practice, and of learning as
ponents:
U
participatory--requires that we rethink or
- Start-up or Intake Activities
reconceptualize not only our notions of
U
- Along the Way or On-going Activities
what counts as literacy but also our methods
of inquiry--the processes we use to docu-
- Looking Back or End of Cycle Activities
ment and assess learning."'
U
The first issue of Adventures in Assessment
Across Massachusetts, literacy programs are
looks at "Start-up" activities. Some of the tools
struggling with a common concern -- learner assess-
included here aim to get a sense of student interests
To assess levels of literacy and English
ment.
and goals as they enter a program. As Auerbach
language proficiency and to document progress,
says, "They provide base-line data about what
programs likely use one or more of these methods:
students can already do with language and literacy,
.standardized testing, materials-based assessments,
how they think about it and what they may want to
U
competency-based assessment, and participatory
do as a result of instruction."'
assessment.'
Volume two of Adventures in Assessment will
The primary purpose of this guidebook is to
focus primarily on-going assessment activities, the
explore participatory assessment or learner-cen-
ways we document learning as it takes place and the
U
tered approaches to assessment and evaluation.
ways we integrate it into instruction. However, we
Participatory literacy education is "based on the
are open to receiving submissions on any activities
belief that learners, their characteristics, aspira-
or tools that you have developed that may be
tions, backgrounds, and needs should be at the
defined as participatory or learner-centered. We
center of literacy education. "'Participatory assess-
are also interested in readers responding to articles,
ment means a collaborative relationship among
especially if the article is used to experiment with
learners and program staff in determining the
the ideas or tools described.
goals, texts and contexts of assessment, as well as
The first piece, "Assessment Issues" synthe-
in judging its outcomes.'
sizes the various articles I have read about alterna-
A few key principles guide alternative forms of
tive assessment. It was first published in 1990 as a
assessment. The most important principle is that
working paper for SABES as an attempt to put forth
assessment must be done with the learner and not to
for readers why the topic of evaluation and assess-
Second, it must be integrated into
the learner.
ment has become so critical to the field of adult
instruction--it should inform curriculum develop-
education.
ment which inturn provides feedback that will
In "Partners in Evaluation," Johan Uvin, a
U
make programs more effective.'
workplace ESOL teacher, emphasizes that evalu-
By definition, participatory assessment or learner-
ation is an integral part of curriculum development.
S
centered assessment, differs from program to pro-
He describes the challenge of striking a balance be-
gram, teacher to teacher, and learner to learner.
tween evaluation needs and preferences of all who
S
I
have vested interests, from learners to fenders. He
education and English as a second language practi-
believes evaluation encompasses three
levels:
tioners. Though the ABE/ESL lens may be helpful,
you might want to also pay attention to what a
assessment of participants, evaluation of the pro-
gram, and evaluation of organizational change.
whole language approach (Kelly) and a participa-
Lucille Fandel describes the importance of un-
tory approach (Fandel and Uvin) have in common
covering themes and issues as an integral part of
and how they differ. You might want to look at how
initial assessment in "Getting in Touch."
She
these practitioners have adapted others ideas, forms
relates how she adapted and created tools that help
and methods for their own purposes, and how they
her and her Latino students make the link between
have made the tools their own (Germanowski).
lesson content and their goals for learning English.
It is our hope that this guidebook will become
Janet Kelly, in her article describes her own
a resource by and for practictioners to select and
"Assessment Adventures" and illustrates what as-
adapt tools for their own contexts. The develop-
sessment looks like in a whole language ABE
ment of this resource depends upon program-based
program. She uses a portfolio approach to show
practitioner research, the results of which will help
how initial and early assessment information even-
develop the field of adult education.
tually becomes a comprehensive record of on-
In future volumes, Janet will
Loren Mc Grail, editor
going assessment.
share her work in on-going reading and writing
assessment. Following Janet's piece is a related
Resources/References
1 Lytle,Susan, "Dimensions of Adult Literacy: Toward Re-
article by Lindy Whiton. Her "Down and Dirty
conceptualizing Assessment," in "Living Literacy: Re-
Miscue Analysis," elaborates on one of the evalu-
thinking Development in Adulthood." (paper).
ation tools Janet describes.
Martha Germanowski documents the devel-
2 Lytle,S. and Wolfe,M. "Adult Literacy Education: Pro-
opment of "The Education Goals Assessment
gram Evaluation and Learner Assessment. " (Columbus:
ERIC Clearinghouse on Adult Career and Vocational
Packet," how it started and how it looks now. Her
Education, 1989).
model illustrates how we can use and adapt other
peoples ideas and tailor them to fit our needs. .Her.
Fingeret, Arlene and-Jurmo, Paul. eds..- "Participatory Lit-
packet also illustrates how skills can be integrated
eracy Education. " New Directions for Continuing Edu-
cation, No. 42. (San Francisco: Jossey Bass. 1989).
with goals.
In his introduction to "Alternative Assessment:
4 Lytle, S. "Dimensions of Adult Literacy: Toward Recon-
An Annotated Bibliography," Don Robishaw ar-
ceptualizing Assessment," ibid.
gues against traditional standardized quantitative
approaches in favor of alternative approaches which
Their Use and Misuse," BCEL
"Standardized Tests:
5
Newsletter (22); January 1990.
help practitioners "set up structures for improving
learning opportunities and "inform the develop-
Hemmendinger, Anna: A Tool Kit:
Self Evaluation
6
ment of curriculum which is based on the learners
Exercises for Students and Literacy Workers (Ontario,
evolving progress."
Canada: East End Literacy, 1988).
While each of these writings falls (loosely)
7 Auerbach, Elsa, Making Meaning, Making Change: (Bos-
under the theme of start-up activities, there are
ton: University of Massachusetts, 1990).
other lenses through which to view them. The most
obvious is that they are written by adult basic
8 ibid.
iii
1\11*/..11.111jx`111.".\-S7MI:/\*P-1:-v1:,:-/-\\:0-/vz,-/\ --cf/wF
Assessment
Issues:
Research and Practice
by Loren McGrail
n recent years the subject of testing and as-
gram effectiveness and that at least a third of local
sessment in adult literacy and education has
programs which apply for federal ABE funds (ad-
become a focus of growing concern, both
ministered in Massachusetts by the DOE) should be
nationally and locally. Decisions about the nature
evaluated through the use of standardized tests. (see
and purpose of assessment are critically important to
BCEL Newsletter, No.22, January 1990)
literacy policy and practice at all levels.
Adult
While the federal mandate for standardized as-
learners; teachers, administrators, and funders all
sessment in ABE and ESL is objectionable to many,
agree that what programs choose to assess and the
it is also a sign that adult education has come of age,
methods they employ affect the quality of teaching
for the need for accountability often accompanies
and learning. Assessment in adult literacy is a central
the investment of significant resources.
issue with high stakes:
The mandate, however, also highlights the
"The authority vested in
these tests can determine the way programs are de-
growing concern of literacy practitioners, theorists,
and test designers about what standardized tests
veloped, what is taught, and the climate of teaching
It shapes legislation and funding
and learning.
actually test and for what purposes.
policies of public and private agencies. It is tied to
As standardized tests have come into sweeping
welfare eligibility for young parents. It drives gov-
use, so have complaints about them and their valid-
ernment job training programs. It can deny entry into
ity. For many the most important question is, do the
military, or crucial access to a diploma or a job."
test results tell us anything of real value either about
(Business Council for Effective Literacy Newsletter
learner progress or program effectiveness?
No. 22,January 1990)
The Mandate
The teachers evaluated me to find out
In April of 1988 Congress passed legislation
how much I knew and compared me
which called for the use of standardized tests to
with the rest of the class. It made me
feel small.
evaluate all adult basic education (ABE) and English
--Carmen Ferino
as a second Language (ESL) programs funded under
the Adult Education Act. These Adult Education
The Debate
Amendments (Public law 100-97) and the imple-
menting regulations of the U.S. Department of
The use of standardized testing most likely de-
Education (August 1989) require that the results of
rives in part from their relative ease of administration
standardized tests be used as one indicator of pro-
and what is viewed by some as their reliable and
1
wi'v>11-."4"*Ii"1\1.1.\\7MP\l''':11.1P.W/7-1\ql./7).C(/WF-;
ri\\.-- 7 NIj\
1\:-:%1-41 1\-\N
1\:%.17
4-4
-7
objective, quantitative data on the achievement,
Lytle and Johnston (1986) that by assessing only
decontextualized subskills like word recognition,
abilities, and skills of studentsdata that is free from
individual judgement. Because the tests and the
paragraph comprehension, and vocabulary, we fail
conditions are (theoretically) constant, they are
to measure the ways in which adult learners already
thought to be useful indicators of individual progress
can and do use literacy in their daily lives.
Still others
over time (pre- and post- testing). Thus, by exten-
argue that because the tests focus on product rather
sion, they are thought to be useful indicators of
than process, they tell us very little about the affec-
program effectiveness.
tive or metacognitive factors in literacy acquisition.
According to Susan Lytle of the University of
Even Dr. Thomas Sticht in his report to the U.S.
Pennsylvania, the chief spokesperson and advocate
Department of Education, "Testing and Assessment
for alternative forms of assessments, these charges
in Adult Basic Education and English as a Second
of "quantifiable and reliable data" for program
Language Programs," advocates that, "It may be de-
evaluation are questionable at best.
For Lytle,
sirable to separate testing for program accountability
("From the Inside Out: Reinventing Assessment" in
from testing for instructional decision making." His
Focus on Basics, Volume 2, Number 1, Fall 1988),
advice should provide some guidance to literacy and
the equation of learner progress assessment--de-
ABE programs caught in the eternal conflict of how
fined as increased reading levelswith program ef-
to be accountable to the funders and the public in a
fectiveness is problematic for two reasons: 1) It ig-
quantifiable way and how to create ways for teachers
to use assessment tools and processes that are an
nores other legitimate criteria for evaluating a liter-
integral part of teaching and learning and that can
acy program like the quality of the curriculum, teach-
ing, or its connection to significant social issues
inform curricula.
The call for alternative forms of assessment is
relevant to students lives and interests; and, 2) it fails
to recognize that increases in reading scores have
coming from all directions, including many main-
little to do with the way adults live and use literacy in
stream and professional organizations and educators.
the real world.
Elsa Auerbach of the University of Massachusetts in
her book, Making Meaning, Making Change,
So, despite the relative ease of administrating
tests, few adult educators are satisfied with the
quotes the Board of Directors of the International
quality of information revealed and many more are
Reading Association's its 1988 statement: "Reading
assessment must reflect recent advances in the
deeply dissatisfied with the effects such testing has
understanding of the reading process. The IRA is
had on teaching and learning. In a special 1989 issue
on alternative assessment from the Literacy Assis-
concerned that instructional decisions are too often
made from assessments which define reading as a
tance Center in New York, this grievance was articu-
"There seems to be considerable
lated clearly:
sequence of discrete skills that students must master
to become readers. Such assessments foster inap-
agreement among adult literacy practitioners that the
propriate instruction."
TABE (whether it be the 1976 or 1987 edition) is not
only an inappropriate instrument for individual as-
Alternative
sessment, but it does not inform the teaching and
Assessment
Approaches/Tools
learning process, and it in fact may act to discourage
students as they re-involve themselves in the educa-
How is
student assessment conducted?
tional process."
- oral interview
Other practitioners and theorists agree with
- tests (what kind? standardized/program-developed?)
2
10
