Evaluations that make a difference: evaluations stories around the world

[Pablo Rodriguez Bilella]

A post published in https://evaluationstories.wordpress.com/
Para leerlo en Español: https://albordedelcaos.com/

----------------------------------------

Truth, naked and cold, had been turned away from every door in the village. 
Her nakedness frightened the people. When Parable found her, she
was huddled in a corner, shivering and hungry. Taking pity on her, Parable
gathered her up and took her home. There, she dressed Truth in Story,
warmed her, and sent her out again. Clothed in Story, Truth knocked again
at the villagers’ doors and was readily welcomed into the people’s houses.
They invited her to eat at their table and to warm herself by the fire.
 

https://evaluationstories.files.wordpress.com/2013/12/storytelling.jpg?w=243&h=170" alt="storytelling" width="243" height="170" class="CToWUd" />What is the value of evaluation?  What factors contribute to making an evaluation more or less valuable? What are the benefits and how are they expressed or measured? How can they be described so that they make sense to citizens and to policy makers? Can stories provide a good way of communicating evaluation findings?

A group of evaluators from around the world received an EvalPartners Innovation Grant to collect stories that would help answer these questions.

The result was Evaluations that make a difference – a collection of eight evaluation stories from around the world which is one of the first pieces of systematic research looking at factors that contribute to high quality evaluations that are used by stakeholders to improve programs and improve people’s lives.  This initiative collected stories about evaluations that made a difference, not only from the perspective of the evaluators but also from the commissioners and users.  The stories in this collection tell powerful stories about the findings in the evaluations and the ways the evaluations contributed to the impact of the programs.

The first step was to define what was meant by evaluations that make a difference. Many evaluations have sound methodologies and, in some cases, get used to inform decisions and improve programs, organizations and policies (evaluation influence). However, this exploration went beyond that and looked at what evaluations can do to improve people’s lives (evaluation impact) and contributed to social betterment.

https://evaluationstories.files.wordpress.com/2016/08/captura-de-pantalla-2016-08-22-a-las-7-37-33-a-m.png?w=500" alt="Captura de pantalla 2016-08-22 a las 7.37.33 a.m." class="CToWUd a6T" tabindex="0" />

In addition to the eight stories about evaluations that have made a difference in people’s lives, this project teased out the “enabling factors” that contributed to their impact, highlighted for the benefit of evaluators and evaluation users who want to do more impactful evaluations.

1. Give voice to the voiceless – all voices need to be heard, but those without power often go unheard

2. Provide credible evidence based on excellent design and methodologies

3. Use a positive approach emphasizing strengths, opportunities, aspirations and results

4. Actively engage users & intended beneficiaries through a utilization-focused process that gets buy-in as the evaluation progresses

5. Embed evaluation within the programme right from the start, if possible, in order to have base-line information and to promote evaluative thinking

6. Sincerely care about the evaluation so that commissioners, users and evaluators are working together to ensure credible evidence

7. Champion the evaluation – evaluators need to work with commissioners and users to help them understand how the evaluation can contribute to making decisions

8. Focus on evaluation impact – from the beginning evaluators need to think about the potential effect of the evaluation on the program and program participants.

These factors were identified from the selected evaluation stories, which have been presented in this blog during the last months, both in English and French (and in Spanish in the blog Al Borde del Caos). Below you will find the eight evaluation stories:

#EvalStories 1:How an evaluation led to rapid change: Salvaging Sri...

.

#EvalStories 2: Tumekataa kula mavi tena! We refuse to eat shit! #Eval

.

#Eval Stories 3: Evaluation in action: The Milne Bay emergency phon...

.

#Eval Stories 4: Positive Sisters: a transformative journey in the ...

.

#EvalStories 5: Learning and earning: training that works in Kenya

.

#EvalStories 6: The power of community-owned data -a First Nations ...

.

#EvalStories 7: If you don’t ask, you won’t see it! The evaluation ...

.

#EvalStories 8: Listening to the listeners – an #eval story in Nepal.

.

In addition to these blog posts, the stories and their process of selection and construction are being presented at events and conferences evaluation. The closest is in Maastricht (Netherlands), at the 12th Conference of the European Evaluation Society. If you are around, you are very welcome to join us there on September 29th, 10:00 – 11:30!
 

https://evaluationstories.files.wordpress.com/2016/08/maastricht.jpg?w=500" alt="maastricht" class="CToWUd a6T" tabindex="0" />

[Rukmini Panda]

Thanks for sharing.

[Rituu B Nanda]

Hi Pablo, I have a question on the evaluation finding in Nepal. I quote here When the programme is in a local language, people feel a sense of ownership – they think it’s a programme for and by them. And they are right!’

How is the programme by the local People? I would like to probe this sense of ownership. Who can answer my question 

This is the link to the story https://evaluationstories.wordpress.com/2016/06/23/evalstories-8-li...

[Rituu B Nanda]

Excellent work! Thanks for sharing Pablo