Monitoring and evaluation: tools for biodiversity conservation and development projects

2008·5.1 MB·English

by  Cape Action for People and the Environment

tools for
biodiversity conservation
and
development
projects
2008
SANBI
Biodiversity Series
11

SANBI Biodiversity Series 11
Monitoring and evaluation:
tools for biodiversity
conservation and development
projects
The second in a series of project management handbooks, dealing with
monitoring and evaluation as a pathway to learning in a people-centred
development context
compiled by
CapeAction for People and the Environment
national
biodiversity
institute
SANBI
Pretoria
2008
SANBI Biodiversity Series
The South African National Biodiversity Institute (SANBI) was established on 1 Sep-
tember 2004 through the signing into force ofthe National Environmental Manage-
ment: BiodiversityAct (NEMBA) No. 10 of 2004 by PresidentThabo Mbeki. The Act
expandsthe mandate ofthe former National Botanical Institute to include respon-
sibilities relating to the full diversity ofSouth Africa's fauna and flora, and builds on
the internationally respected programmes in conservation, research, education and
visitor services developed bythe National Botanical Institute and its predecessors
over the past century.
The vision ofSANBI: Biodiversity richness for all South Africans.
SANBI's mission is to champion the exploration, conservation, sustainable use,
appreciation and enjoyment of South Africa's exceptionally rich biodiversityfor all
people.
SANBI BiodiversitySeries publishes occasional reports on projects, technologies,
workshops, symposia and other activities initiated by or executed in partnership with
SANBI.
Technical editor: Emsie du Plessis
Design & layout: Daleen Maree
Cover design: SANBI Graphics
Howto citethis publication
CAPE ACTION FOR PEOPLE AND THE ENVIRONMENT 2008. Monitoring and
evaluation: toolsfor biodiversity conservation and development projects. SANBI
BiodiversitySeries 1 1. South African National Biodiversity Institute, Pretoria.
ISBN 978-1-919976-47-1
© Published by: South African National Biodiversity Institute
Obtainable from: SANBI Bookshop, Private Bag XI01, Pretoria, 0001 South Africa.
Tel.: [phone]. E-mail: [email protected]. Website: www.sanbi.org.
Printed by: Credo Communications (Pty) Ltd, RO. Box 9403, Johannesburg, 2000.
Contents
Preface v
Message v
Acknowledgements vi
Acronyms vii
Jargon ... the insider’s language vii
INTRODUCTION 1
WHYTHIS HANDBOOK?
1. I
a) Recenttrends in M&E 2
b) M&Ewithin the biodiversity conservation sector 3
2. WHO ISTHIS HANDBOOK FOR? 6
3. HOWYOU CAN USETHIS HANDBOOK 7
4. HOWTHE HANDBOOK IS STRUCTURED 8
Section1:DEVELOPINGANM&EPLAN 11
1.1 WHATARETHE OBJECTIVES OFTHE PROJECT? 12
1.2 WHOWILL USETHE INFORMATION COLLECTED? 17
1.3 HOWARETHE INDICATORS USEDAND DEVELOPED? 20
1.4 WHAT METHODSWILL BE USEDTO GATHER INFORMATION? 28
WHO
1.5 WILL PARTICIPATE IN M&E? 30
1.6 WHENWILL M&ETAKE PLACE? 33
1.7 HOWWILLTHE M&E SYSTEM BE MANAGED? 35
1.8 HOWWILL LEARNING BE BUILT INTOTHE M&E PROCESS? 37
Section2:LEARNING 40
2.1 HOW DOES LEARNING HAPPEN? 41
2.2 LEARNING INTHE PROJECTTEAM 43
2.3 BUILDINGA LEARNING ENVIRONMENT 50
Section3:GATHERINGTHEINFORMATION 52
3.1 UNDERSTANDINGYOUR INFORMATION NEEDS 53
3.2 SELECTINGAND USINGAPPROPRIATE METHODS 57
3.3 MANAGINGTHE INFORMATION GATHERING SYSTEM 63
Section4:ANALYSINGTHEINFORMATION 67
4.1 ESTABLISHING IFTHEREARE SIGNIFICANT DIFFERENCES 68
4.2 CLARIFYINGANDAGREEING ONTHE REASONS FORANY DIFFERENCES 74
4.3 CONDUCTING FORMAL EVALUATIONS 76
Section5:ACTINGONTHEANALYSIS 63
5.1 DRAWING OUT RECOMMENDATIONS 84
5.2 EFFECTIVE DECISION-MAKING 87
iii
Section6:SHARINGTHELEARNING(REPORTING) 93
6.1 WHAT ISACCOUNTABILITY? 94
6.2 REPORTING FORMATS 99
6.3 FINDING COMMON GROUND 102
CONCLUSION 107
FURTHERREADING 109
BIBLIOGRAPHY 111
APPENDIX1:GUIDELINESFORTRAINERS 113
INTRODUCTION 113
Section DEVELOPINGAN M&E PLAN 114
I:
Section 2:LEARNING 117
Section 3:GATHERINGTHE INFORMATION 120
Section 4.ANALYSINGTHE INFORMATION 123
Section 5:ACTING ONTHEANALYSIS 125
Section 6:SHARINGTHE LEARNING 126
IV
Preface
We are pleased to presentthe second in a series ofthree handbooks thatform part ofthe C.A.P.E.Part-
nersToolbox,followingthe first publication,Projectplanning:tools forbiodiversityconservation and development
projects,published lastyear.
We hope thatthis second publication,focusing on project-level monitoring and evaluation (M&E) and re-
porting,will be ofuse to organisations and projects across the C.A.P.E.partnership.Although some projects
have M&E systems in place,there is always room f—or improvement in tacklingthe big questions around the
real impact ofour biodiversity conservation work in terms ofboth the state ofbiodiversity and the deliv-
ery ofsocio-economic benefits.
This resource sets out practical tips for developingan M&E plan,buildinga learning environment,setting up
an information gathering system,analysingthe information,drawing out recommendations and sharingthe
learning. I believe itwill be an asset to anyone who is designing,implementing or improving an M&E system.
Guidelines fortrainers are also provided to enable you to use the material in the bookfurther within your
organisation.
I wish you the best ofluck in usingthis resource and hope itwill contribute to the development ofexcel-
lence in monitoring and evaluation throughoutthe C.A.P.E.programme.
Dipolelo Elford
Chairperson:C.A.P.E. Implementation Committee
June 2008
Message
The Critical Ecosystem Partnership Fund has supported the production ofthis handbook as a resource
that adds value to our investment in the Cape Floristic Region biodiversity hotspotand to the C.A.P.E.
programme.
CEPF believes that it is critical for all ofus in the conservation sectorto analyse the overall impact ofour
work in making progress towards our intended conservation outcomes.This involves setting clear and
coherent objectives and strategies thatgive our projects focus and direction,an expected results chain or
logical framework,measurable indicators that meet our monitoringand reporting needs,and systematic and
regular processes for collecting and managing data.
All these topics are covered in this handbook,using real case studies that bringthe theory ofmonitoring
and evaluation to life and make itaccessible.We welcome the addition ofthis handbookto the C.A.P.E.
PartnersToolbox and encourage readers to apply itto their own projects and programmes.
Nina Marshall
Grant Director
Critical Ecosystem Partnership Fund
June 2008
v
Acknowledgements
This handbook is an adaptation ofthe Planning formonitoring and evaluation handbook by Michael Randel,
published by Olive Publications in 2002.CapeAction for People and the Environment (C.A.PE.) obtained
permission forthe adaptation,and commissioned this handbook in 2007-2008.
It is preceded by and builds on the material contained in the Project planning:tools forbiodiversityconservation
and developmentprojects handbook prepared by C.A.P.E.and published by the SouthAfrican National
Biodiversity Institute in 2007.
In the course ofdevelopingthis handbook,a number ofpeople made valuable contributions.In the early
stages,the User Reference Group (comprisingTanya Goldman and Onno Huyser) and the Client Reference
Group (comprising Mandy Barnett,Azisa Parker,Caroline Petersen and Monique Damons) provided a clear
foundation forthe development ofthe handbook.
Project managers from three projects and a small grants funding agencygenerously provided time and
insightforthe purpose ofgenerating case study material.They are:
• Cape Flats Nature.
• The C.A.P.E.Conservation Stewardship Programme,located in CapeNature.
• Greater Cederberg Biodiversity Corridor.
• Table Mountain Fund.
Detailed information aboutthese projects can be found in Fynbos fynmense:people makingbiodiversity work
(Ashwell et al. 2006,SANBI Biodiversity Series 4,published by the SouthAfrican National Biodiversity Institute,
Pretoria) or on the C.A.P.E.website,http://www.capeaction.org.za.
Participants in the Project Developers’ Forum held on 18 and 19 March 2008 also contributed material used
in the examples in the handbook and in further refining the content.
The C.A.P.E.Co-ordination Unit oversawthe process ofproducingthe material forthe handbook. Material
was developed and compiled by Carol-Ann Foulis andJennyWhitehead.Sue Soal skilfully facilitated the
Project Developers’ Forum and helped with the conceptualisation ofthe handbook.Anne Kroon brought
her creativity and experience to bear in helping to develop theTrainer Guidelines.Judy Norton meticulously
edited the handbook. Michael Randel,as the original author ofthe Planning formonitoring and evaluation
handbook,provided in-depth and valuable comments on a draftversion ofthis handbook.
SANBI’s Publication Unitwas responsible fortechnical editing,design and layout,and cover design forthe
handbook.
Financial supportwas provided bythe Critical Ecosystems Partnership Fund and the Global Environment
Facilitythrough the C.A.P.E. Biodiversity Conservation and Sustainable Development project.
The C.A.P.E.Programme is hosted bythe SouthAfrican National Biodiversity Institute and is supported by
24 signatory partners.
VI
Acronyms
AAR
AfterAction Review
C.A.P.E. CapeAction for People and the Environment
CDRA Community Development ResourceAssociation
CEPF Critical Ecosystems Partnership Fund
CFN Cape Flats Nature
CFR Cape Floristic Region
CNC
Cape Nature Conservation
EIA Environmental ImpactAssessment
GIS Geographic Information System
LFA Logical FrameworkApproach
M&E
Monitoring and Evaluation
NGO
Non-government Organisation
OVI ObjectivelyVerifiable Indicator
PCM
Project Cycle Management
PLA Participatory Learning andAction
PME
Participatory Monitoring and Evaluation
PRA
Participatory RuralAppraisal
SA SouthAfrica
SANBI SouthAfrican National Biodiversity Institute
SCM Sustainable Conservation Management
TMF
Table Mountain Fund
TOR Terms Of Reference
WWF
WorldWide Fund for Nature
Jargon the insider's language
...
Adaptive management Management that is responsive to learning.
Action learningA learning process that,by reflecting on past experiences,draws out relevant lessons
and supports the preparation forfuture actions.
ActivitiesThe steps thatthe project musttake to achieve the Outputs.
Assumptions External factors,not under control ofthe project,that are identified as being necessaryfor
project success;part ofthe project plan (LFA).
Beneficiaries The people who will experience improved conditions (benefits) as a consequence ofthe
projecttargetingtheir needs.
Benefits The positive conditions ofchange resultingfrom a project.
Development goal Describes the benefits thatwill resultfrom the project.
Development objective See Developmentgoal.
Enabling factors SeeAssumptions.
Ex-ante evaluation An assessment ofthe project plan and approach,before it is implemented,to assess
whether it has agood chance ofachieving its objectives.
External factors SeeAssumptions.
VII
Formative evaluation A periodic review ofthe projectthat allows continuous feedbackto inform
ongoing changes and improvements in a service or a product.
Immediate objective See Project purpose.
ImpactThe positive differences resultingfrom the project;often seen in the benefits for specific groups.
Impact evaluation An external and in-depth study ofthe impact ofa project on its beneficiaries;usually
carried out 3-6 years afterthe project has closed.
IndicatorThe measure that is used to assess ifan objective has been achieved,or what progress has been
made.
Inputs The human,financial and material resources required to implementthe project.
Intervention A deliberate action on the part ofthe projectto influence change in a social system.
Logical FrameworkApproach (LFA) A method for project planningthatfocuses on objectives or
outcomes.
Means ofverification The location or source ofthe evidence used as indicators ofproject
achievements.
Mid-term evaluation An external assessment ofthe project,usually conducted midwaythrough the life
ofthe project,thatfocuses on its performance and assesses whetherthe objectives continue to be relevant.
Objectives Describe what we are trying to achieve.There are four levels ofobjectives in the LFA
approach:Goal, Purpose,Outputs andActivities.
Objectives analysis A tool to visualise an improved future,linked on a,‘means-end’ basis.
Outcomes The positive changes that resultfrom the project’s intervention;achieved by the use ofthe
outputs ofthe project.
Outputs Describe the responsibilities ofthe project,and the services and products itwill deliver.
Participatory Monitoring and Evaluation (PME) An approach to monitoring and evaluation that
emphasises the role and views ofthe targetgroups and beneficiaries ofthe project.
ProjectA set ofplanned activities designed to achieve specific objectives with given resources within a
specific time frame.
Project Cycle Management (PCM) A process ofmanaging a projectfrom the start ofthe
preparation and planning phase,through to implementation and evaluation.
Project purpose Describes whatthe targetgroup will be able to do because ofthe supportthey receive
from the project.
Results See Outputs.
Summative evaluation Evaluation thattakes place to reviewthe accomplishments ofthe project and
to inform high-level decisions offuture funding and policy.
SustainabilityThe ability ofsomethingto maintain its value overa long period,without external support.
Target group The people whom the projectaims to support,and whose consequentactions will bring
abouta benefitto themselves and/or others.
Terminal evaluation An evaluation atthe close ofthe project on whether it has accomplished its
objectives or not.
viii