Equity Focused Evaluation: Notes from Evaluation Conclave Kathmandu (26th February to 1st March, 2013)

[Ranjani K.Murthy]

Equity focused evaluation

Marco Segone, Michael Bamberger, and Urs Nagel

What does equity mean?

The concept of equity is based on notions of fairness and justice, and refers to equitable outcomes. It needs to be distinguished from equally of opportunity.

Why is equity important?

The concept of equity is important due to several factors:

• Inequity constitutes a violation of human rights
• Has a positive impact in reducing poverty
• Equity may have a positive impact on economic growth
• Leads to a socially fair and democratic society
• It hampers the equitable achievements of human development and MDGs

What is equity focused evaluation

An Equity-focused evaluation comprises of an assessment of the relevance, effectiveness, efficiency, impact and sustainability of policies, programmes and projects on equitable development results. We will look at national average if we did a traditional evaluation. In the case of equity focused evaluation, we would disaggregate the data across richest and poorest or other aspects of inequities and determine if inequities have reduced.

How to manage equity focused evaluations

• Preparing for the equity focused evaluation
• Preparing TOR
• Designing the evaluation
• Collecting analysing evidence
• Arriving at findings, conclusions and recommendations
• Utilising of recommendations

Preparing for the equity focused evaluation (efa)

• Determine the evaluability of the interventions’ equity dimensions
• Identifying evaluation stakeholder’s including worst off
• Identifying intended use by intended users
• Ensuring appropriate budget
• Identifying potential challenges in promoting and implementing equity focused evaluation
• Doing a SWOT of the equity focused evaluation

Special challenges for evaluating equity

• Lack of disaggregated data
• Many outcomes are sensitive or difficult to identify and measure (violence)
• Unanticipated outcomes
• Complex processes of behavioural change
• Equity evaluation requires new methodologies
• Findings can be threatening

Preparing the TOR

• Defining the scope and purpose of the evaluation
• Framing evaluation questions focusing on equity rather than just impact on poor
• Selecting technically strong and culturally sensitive evaluation team

Designing the evaluation

• Evolving a theory of change
• Attribution (alone) and contribution (along with others) analysis
• Social exclusion analysis
• Gender analysis
• Systems analysis
• Bottleneck analysis

Theory of change

• Clearly defines intended outcomes/impact
• Reach consensus among stakeholders
• Explains mechanisms and processes through which outcomes achieved
• Identifies key assumptions
• Contextual analysis
• Identification of unanticipated outcomes

Systems analysis

• Programs are embedded in existing social systems with historical traditions and linkages amongst stakeholders
• Different stakeholders have different perspectives on the program
• Boundaries can be open or closed
• New interventions cause contradictions and conflicts
• Outcomes determined are by how conflicts are resolved

 

Bottleneck supply and demand framework

Supply side

• Coverage
• budgets
• Humanpower resources

Contextual factors

• Economic
• Political
• Institutional
• Environmental
• Administrative and legal

Demand side:

• Knowledge, attitudes and practice of vulnerable group
• Community ownership
• Cultural acceptable services
• Culturally sensitive staff
• Distance
• Cost of travel for availing services
• Available transport and fees
• Poverty
• Time

Appropriate Evaluation design for equity focused evaluations

Experimental design

• Experimental design (choose control group)
• Quasi experimental design (before and after comparison)

Non experimental designs

• Interrupted time series
• Case study analysis - multiple case analysis (more systematic)and single/few case studies
• Longitudinal analysis
• Key informant interviews
• Participant observation
• Participatory group methods
  • PRA
  • Most significant changes

Equity outcome indicators

• Human development index
• Inequality adjusted human development index
• Gender  development index
• Gender empowerment measure
• Income distribution by quintile groups
• Public expenditure incidence analysis
• Access to public services by quintile

Select the appropriate measurements tools

• Supply side- project documents, public expenditure surveys
• Demand side- KAP assessment, case studies, observations, key informants, FGDs
• Contextual services- diagnostic studies, rapid surveys, secondary data,