Value added of post project evaluation, rather than just final evaluation

[Jindra Cekan, PhD]

Hello - please see our research report on post project #evaluation where we show the value added to 'program cycle' final evaluation. (Un)expected #sustainability or not, emerging sustainability and more: http://valuingvoices.com/building-the-evidence-base-for-post-projec...

[Ranjani K.Murthy]

Dear Jindra

The term gender and social transformation is used loosely. To really assess progress towards the same, post project evaluations are a must. 

Thanks for sharing 

Ranjani

[Jindra Cekan, PhD]

Kari- hi there. Thank you for your search and absolutely more are cropping up, especially by Social impact of E3's Water projects since they made Sustainability part of their Strategic Plan. MSI found that only 2% of all USAID evaluations from 2011-15 were or included ex-posts. Trust me, I have waded through the DEC, and while 'ex-posts' appear numerous, many that are 'found" are 'found' because the DEC finds  the term inside the documents such as "

 The system supports ex ante reporting often required for project development
and it supports project monitoring and ex post reporting. ... 

…

. PPL should increase its support for ex-post evaluations and special studies to better understand the conditions under which CD support leads to ... 

So glad you agree that use is key - so how did USAID/ Colombia use the findings? So glad staff trained do continue. You may enjoy our presentation to the Local Systems Community at USAID two weeks ago: https://ac.usaid.gov/p6p1ip9v4ff/?proto=true

Warmly, Jindra

[Keri Culver]

I just did a search at dec.usaid.gov on the word "ex-post" filtered by document type: evaluation, and got a list of about 100 since the new Evaluation Policy took effect in 2011. Not all appear to be actual ex-post evaluations, and you've probably done this already. But I thought it might be useful so I'll send along the link of the search: https://decsearch.usaid.gov/search?client=dec_pdfs&proxystylesh...*&filter=0&site=default_collection&output=xml_no_dtd&proxyreload=1&ulang=en&ie=UTF-8&emdstyle=true&ip=172.16.1.4&access=p&sort=date:D:L:d1&entqr=3&entqrm=0&entsp=a__dec_results_biasing&wc=200&wc_mc=1&oe=UTF-8&ud=1&q=ex-post+inmeta:Document%2520Type%3DEvaluation&dnavs=inmeta:Document%2520Type%3DEvaluation

I think your report hits the nail on the head when you ask whether these reports are utilized, and if so, how: I think in the case I was involved with in Colombia, the work had the purpose of showing effectiveness of alternative development programs specifically, which had become a frowned-upon connotation. I don't know what the Mission or USAID/Washington might have done with it afterward, but there certainly is healthy programming in Colombia that, while not calling itself "alternative development", serves that function.

Another benefit I saw in Colombia is that a lot of the staff from the projects evaluated in this ex-post work were still at work in other projects. Their experiences certainly didn't go to waste because they were still in the field or directing efforts to increase economic development in rural areas. Whether or not they can use the lessons learned is another question, because of national and donor rules and parameters. But I am hopeful that the people we spoke with are able to incorporate their knowledge about what works (with or without an ex-post) with their current teams.  

[Jindra Cekan, PhD]

Keri - this is great - i had no idea USAID had done one in Colombia - so glad to add it to our repository at www.ValuingVoices.com/catalysts.  YES YES YES. When activities are valuable to communities, they will do their best to sustain outcomes and create new emerging ways to make lemonade from what projects created even after closure. We found the same in Niger - agriculture and health continued to be sustained w/new inputs from communities, government, private sector (http://valuingvoices.com/participation-by-all-the-key-to-sustainabi...) . Bea Rogers (Tufts) did a study on Food For Peace/USAID project which I thought added a terrific overview that sustainability requires: resources, capacities, linkages and motivation and we need to build these during implementation/ exit for sustainability. Thank you for sharing!

[Rituu B Nanda]

Thanks Keri for taking out time so that I could learn from you. I have a similar experience that when communities recognise the issue as critical to them and take responsibility will they continue to act beyond the project. Services are important but building this capacity in the community is important. 

[Keri Culver]

It seemed to me a natural response on their part, as active citizens in a country in flux, communities in flux. They wanted to continue the work of the project because the types of outcomes - primarily of support to agricultural value chains and local (small) infrastructure projects - were valuable to them. They maintained a small bridge that had been built, or a collection center for agricultural produce, because these had value for their lives. They continued to work together on such projects to improve roads sections or whatever it might be, because the government hadn't taken care of things and the mechanisms the project helped create - of debate, citizen oversight, accountability with the project funds, etc. - were useful, and helped make up for the crushing of social capital that has taken place after years of conflict. It's been a long time since we worked on this project so I'm philosophizing a bit, rather than drawing directly on data for this response. But I think the features of utility, accountability, social capital and a link to the economy  were strong predictors of ongoing activities.

[Rituu B Nanda]

Thanks Keri for sharing. Did you learn the reason why project participants had taken on the responsibility for improvements. Am eager to learn. 

[Keri Culver]

Hello,

I liked reading this report. It is very thoughtful and puts some challenges in front of us for improving the way we attend to issues of sustainability. My team and I did an ex-post evaluation in 2013-2014 (http://pdf.usaid.gov/pdf_docs/pa00jrmk.pdf) in Colombia and ran into some of the same difficulties. As much trouble as it was to find the program's participants, and the staff members who had gone on to other employment, we were also surprised and happy to see how participants had taken the improvements of their communities into their own hands. They used some tools from the project itself, but emerging and new outcomes were also quite dependent on individual champions who wanted to take what they'd learned, further. Something to think about during project implementation: how to provide the impulse and emerging training needs to people who are itching to do something good with them, whether the project is funding activities or not.

Best wishes -- Keri

[Rituu B Nanda]

Thanks for posting Jindra. Excellent interview here http://www.sixdegreesnews.org/archives/20252/in-17-years-about-1-6-...