Handbook of Practical Program Evaluation

907 Pages·2015·12.6 MB·English

by  Hatry

HANDBOOK OF PRACTICAL PROGRAM
EVALUATION

HANDBOOK OF
PRACTICAL PROGRAM
EVALUATION
FOURTH EDITION
Kathryn E. Newcomer
Harry P. Hatry
Joseph S. Wholey
JOSSEY-BASS
A Wiley Imprint
www.josseybass.com
Coverimage:©iStock.com/naqiewei
Coverdesign:Wiley
Thisbookisprintedonacid-freepaper.
Copyright©2015byKathrynE.NewcomerandHarryP.Hatry,andJosephS.Wholey.Allrights
reserved.
PublishedbyJohnWiley&Sons,Inc.,Hoboken,NewJersey
PublishedsimultaneouslyinCanada
Nopartofthispublicationmaybereproduced,storedinaretrievalsystem,ortransmittedinanyform
orbyanymeans,electronic,mechanical,photocopying,recording,scanning,orotherwise,exceptas
permittedunderSection107or108ofthe1976UnitedStatesCopyrightAct,withouteithertheprior
writtenpermissionofthePublisher,orauthorizationthroughpaymentoftheappropriateper-copyfee
totheCopyrightClearanceCenter,222RosewoodDrive,Danvers,MA01923,[phone],fax(978)
646-8600,oronthewebatwww.copyright.com.RequeststothePublisherforpermissionshouldbe
addressedtothePermissionsDepartment,JohnWiley&Sons,Inc.,111RiverStreet,Hoboken,NJ
07030,[phone],fax[phone],oronlineatwww.wiley.com/go/permissions.
LimitofLiability/DisclaimerofWarranty:Whilethepublisherandauthorhaveusedtheirbestefforts
inpreparingthisbook,theymakenorepresentationsorwarrantieswiththerespecttotheaccuracyor
completenessofthecontentsofthisbookandspecificallydisclaimanyimpliedwarrantiesof
merchantabilityorfitnessforaparticularpurpose.Nowarrantymaybecreatedorextendedbysales
representativesorwrittensalesmaterials.Theadviceandstrategiescontainedhereinmaynotbe
suitableforyoursituation.Youshouldconsultwithaprofessionalwhereappropriate.Neitherthe
publishernortheauthorshallbeliablefordamagesarisingherefrom.
Forgeneralinformationaboutourotherproductsandservices,pleasecontactourCustomerCare
DepartmentwithintheUnitedStatesat(800)762-2974,outsidetheUnitedStatesat(317)572-3993or
fax[phone].
Wileypublishesinavarietyofprintandelectronicformatsandbyprint-on-demand.Somematerial
includedwithstandardprintversionsofthisbookmaynotbeincludedine-booksorin
print-on-demand.IfthisbookreferstomediasuchasaCDorDVDthatisnotincludedintheversion
youpurchased,youmaydownloadthismaterialathttp://booksupport.wiley.com.Formore
informationaboutWileyproducts,visitwww.wiley.com.
LibraryofCongressCataloging-in-PublicationData:
Handbookofpracticalprogramevaluation/KathrynE.Newcomer,HarryP.Hatry,JosephS.Wholey,
editors.—Fourthedition.
pagescm.—(Essentialtextsfornonprofitandpublicleadershipandmanagement)Includes
index.
ISBN978-1-118-89360-9(hardback:alk.paper);
ISBN978-1-118-89361-6(pdf);ISBN978-1-118-89369-2(epub)
1.Policysciences.2.Politicalplanning—Evaluation.I.Newcomer,KathrynE.,editor.II.Hatry,
HarryP.,editor.III.Wholey,JosephS.,editor.
H97.H3582015
658.4—dc23
2015009201
PrintedintheUnitedStatesofAmerica
10 9 8 7 6 5 4 3 2 1
CONTENTS
Figures, Tables, and Exhibits ix
Preface xv
The Editors xxi
The Contributors xxv
PART ONE: EVALUATION PLANNING AND DESIGN 1
1.Planning and Designing Useful Evaluations 7
KathrynE.Newcomer,HarryP.Hatry,JosephS.Wholey
2.Analyzing and EngagingStakeholders 36
JohnM.Bryson,MichaelQuinnPatton
3.Using LogicModels 62
JohnA.McLaughlin,GretchenB.Jordan
4.Exploratory Evaluation 88
JosephS.Wholey
v
vi Contents
5.Performance Measurement 108
TheodoreH.Poister
6.Comparison Group Designs 137
GaryT.Henry
7.Randomized Controlled Trials 158
CaroleJ.Torgerson,DavidJ.Torgerson,CeliaA.Taylor
8.ConductingCase Studies 177
KarinMartinson,CarolynO’Brien
9.Recruitment and Retention ofStudy Participants 197
ScottC.Cook,SharaGodiwalla,KeeshawnaS.Brooks,ChristopherV.
Powers,PriyaJohn
10.Designing, Managing,and Analyzing MultisiteEvaluations 225
DebraJ.Rog
11.EvaluatingCommunityChange Programs 259
BrettTheodos,JosephFirschein
12.CulturallyResponsive Evaluation 281
StaffordHood,RodneyK.Hopson,KarenE.Kirkhart
PART TWO: PRACTICAL DATA COLLECTION PROCEDURES 319
13.Using Agency Records 325
HarryP.Hatry
14.Using Surveys 344
KathrynE.Newcomer,TimothyTriplett
15.Role Playing 383
ClaudiaL.Aranda,DianeK.Levy,SierraStoney
16.Using Ratingsby Trained Observers 412
BarbaraJ.CohnBerman,VernaVasquez
Contents vii
17.CollectingData inthe Field 445
DemetraSmithNightingale,ShelliBalterRossman
18.Using the Internet 474
WilliamC.Adams
19.ConductingSemi-Structured Interviews 492
WilliamC.Adams
20.Focus GroupInterviewing 506
RichardA.Krueger,MaryAnneCasey
21.Using Stories in Evaluation 535
RichardA.Krueger
PART THREE: DATA ANALYSIS 557
22.QualitativeData Analysis 561
DelwynGoodrick,PatriciaJ.Rogers
23.Using Statistics inEvaluation 596
KathrynE.Newcomer,DylanConger
24.Cost-Effectiveness and Cost-Benefit Analysis 636
StephanieRieggCellini,JamesEdwinKee
25.Meta-Analyses, Systematic Reviews, and EvaluationSyntheses 673
RobertBoruch,AnthonyPetrosino,ClaireMorgan
PART FOUR: USE OF EVALUATION 699
26.PitfallsinEvaluations 701
HarryP.Hatry,KathrynE.Newcomer
viii Contents
27.ProvidingRecommendations, Suggestions, and Optionsfor
Improvement 725
GeorgeF.Grob
28.Writing for Impact 739
GeorgeF.Grob
29.Contracting for EvaluationProductsand Services 765
JamesB.Bell
30.Use of EvaluationinGovernment 798
JosephS.Wholey
31.EvaluationChallenges, Issues, and Trends 816
HarryP.Hatry,KathrynE.Newcomer,JosephS.Wholey
Name Index 833
SubjectIndex 841