Program Evaluation: Pragmatic Methods for Social Work and Human Service Agencies

508 Pages·2020·3.291 MB·English

by  Allen Rubin

P RO G RAM E VAL UAT I O N
Pragmatic Methods for Social Work and Human Service Agencies
Be prepared for your future role in a service-oriented agency. This textbook
provides practical guidance on program evaluation while avoiding replicat-
ing other course material. Drawing on over forty years of subject knowl-
edge, Allen Rubin describes evaluation methods and designs that are feasi-
ble for service-oriented agencies and that match the degree of certainty
needed by key users of outcome evaluations. The utility and easy calcula-
tion of within-group effect sizes are outlined, which enhance the value of
outcome evaluations that lack control groups.
Instructions are also given on how to write and disseminate an evaluation
report in a way that maximizes its chances of being used. Conducting focus
group interviews and capitalizing on the value of non-probability samples
will become second nature after following the effective and pragmatic ad-
vice mapped out chapter-by-chapter.
Allen Rubin has been teaching courses on program evaluation for over
forty years. He is the Kantambu Latting College Professor of Leadership
and Change at the University of Houston’s Graduate College of Social
Work, past president of the Society for Social Work and Research, and a fel-
low in the American Academy of Social Work and Social Welfare.
PROGRAM EVALUATION
Pragmatic Methods for Social Work and Human
Service Agencies
Allen Rubin
University of Houston
University Printing House, Cambridge CB2 8BS, United Kingdom
One Liberty Plaza, 20th Floor, New York, NY 10006, USA
477 Williamstown Road, Port Melbourne, VIC 3207, Australia
314–321, 3rd Floor, Plot 3, Splendor Forum, Jasola District Centre, New
Delhi – 110025, India
79 Anson Road, #06–04/06, Singapore 079906
Cambridge University Press is part of the University of Cambridge.
It furthers the University’s mission by disseminating knowledge in the
pursuit of education, learning, and research at the highest international levels
of excellence.
www.cambridge.org
Information on this title: www.cambridge.org/rubin
DOI: 10.1017/9781108870016
© Allen Rubin 2020
This publication is in copyright. Subject to statutory exception and to the
provisions of relevant collective licensing agreements, no reproduction of any
part may take place without the written permission of Cambridge University
Press.
First published 2020
Printed in the United Kingdom by TJ International Ltd, Padstow, Cornwall
A catalogue record for this publication is available from the British Library.
Library of Congress Cataloging-in-Publication Data
Names: Rubin, Allen, author.
Title: Pragmatic program evaluation for social work : an introduction /Allen
Rubin, University of Houston.
Description: Cambridge, United Kingdom ; New York, NY : Cambridge
University Press, 2020. | Includes bibliographical references and index.
Identifiers: LCCN 2019059898 (print) | LCCN 2019059899 (ebook) | ISBN
9781108835992 (hardback) | ISBN 9781108870016 (ebook)
Subjects: LCSH: Social service — Evaluation. | Evaluation research (Social
action programs)
Classification: LCC HV11 .R824 2020 (print) | LCC HV11 (ebook) | DDC
361.3—dc23
LC record available at https://lccn.loc.gov/2019059898
LC ebook record available at https://lccn.loc.gov/2019059899
ISBN 978-1-108-83599-2 Hardback
ISBN 978-1-108-79909-6 Paperback
Additional resources for this publication at www.cambridge.org/rubin
Cambridge University Press has no responsibility for the persistence or accuracy
of URLs for external or third-party internet websites referred to in this publication
and does not guarantee that any content on such websites is, or will remain,
accurate or appropriate.
Contents in Brief
Part I Introduction
1 Introduction and Overview
2 Ethical and Cultural Issues in Program Evaluation
Part II Quantitative and Qualitative Methods for
Formative and Process Evaluations
3 Needs Assessment
4 Survey Methods for Program Planning and Monitoring
Part III Evaluating Outcome in Service-Oriented
Agencies
5 Selecting and Measuring Outcome Objectives
6 Inference and Logic in Pragmatic Outcome Evaluation: Don’t
Let the Perfect Become the Enemy of the Good
7 Feasible Outcome Evaluation Designs
8 Single-Case Designs for Evaluating Programs and Practice
9 Practical and Political Pitfalls in Outcome Evaluations
Part IV Analyzing and Presenting Data
10 Analyzing and Presenting Data from Formative and Process
Evaluations
11 Analyzing Data from Outcome Evaluations
12 Writing and Disseminating Evaluation Reports
Epilogue: More Tips for Becoming a Successful Evaluator
References
Index
Contents in Detail
List of Figures
List of Tables
Preface
Acknowledgments
Part I Introduction
1 Introduction and Overview
1.1 Introduction
1.2 Why Evaluate?
1.3 Some Programs are Ineffective or Harmful
Critical Incident Stress Debriefing
Scared Straight Programs
1.4 Historical Overview of Program Evaluation
1.5 Evidence-Informed Practice
1.6 Philosophical Issues: What Makes Some Types of
Evidence Better Than Other Types?
Contemporary Positivism
Interpretivism
Empowerment
Constructivism
1.7 Qualitative versus Quantitative Evaluations: A False
Dichotomy
1.8 Definitions
1.9 Different Evaluation Purposes
1.10 Types of Evaluation
Summative Evaluation
Formative Evaluation
Process Evaluation
Performance Measurement Systems
Evaluating One’s Own Practice
Accreditation
1.11 Chapter Main Points
1.12 Exercises
1.13 Additional Reading
2 Ethical and Cultural Issues in Program Evaluation
2.1 Introduction
2.2 Ethical Issues
2.3 Institutional Review Boards (IRBs)
2.4 Culturally Sensitive Program Evaluation
2.4.1 Recruitment
2.4.2 Retention
2.4.3 Data Collection
2.4.4 Analyzing and Interpreting Evaluation Findings
2.4.5 Measurement Equivalence
2.5 Developing Cultural Competence
2.5.1 Acculturation and Immigration
2.5.2 Subgroup Differences
2.5.3 Culturally Sensitive Data Analysis and Interpretation
2.6 Chapter Main Points
2.7 Exercises
2.8 Additional Reading
Part II Quantitative and Qualitative Methods for
Formative and Process Evaluations
3 Needs Assessment
3.1 Introduction
3.2 Defining Needs: Normative Need versus Felt Need
3.3 Felt Need versus Service Utilization
3.4 Needs Assessment Approaches
3.4.1 Social Indicators
Advantages/Disadvantages
3.4.2 Rates under Treatment
Advantages/Disadvantages
3.4.3 Key Informants
Advantages/Disadvantages
3.4.4 Community Forums
Advantages/Disadvantages
3.4.5 Focus Groups
How to Conduct a Focus Group
Types and Sequence of Focus Group Questions
Advantages/Disadvantages
3.4.6 Community Surveys
Advantages/Disadvantages
3.5 Triangulation
3.6 Chapter Main Points
3.7 Exercises
3.8 Additional Reading
4 Survey Methods for Program Planning and Monitoring
4.1 Introduction