Reflections from Community of Evaluators, Bhutan Conference

[Payal Mulchandani]

Creativity and good facilitation are at the heart of participatory research!

Recently, I attended a conference on Impact Evaluation hosted by the Community of Evaluators (COE)/ South Asia in Thimpu, Bhutan. The conference was all sorts of GREAT! Four days was divided into two days of pre conference workshops, while the two-day conference saw several interesting panel discussions.

This being one of my first international conferences on Evaluation, I was not sure what to expect. However, sitting and reflecting on the conference, I would say it gave me a ton of interesting literature (which is really keeping me busy), a bunch of new friends, profound conversations, tremendous realizations and learning, an understanding of evaluation from a global perspective with insights from different countries, and an opportunity to network and hear from a good mix of professionals: entrepreneurs, social sector leadership, researchers, Government and private sector personnel, etc.

To my absolute delight, Professor Robert Chambers hosted a full day workshop on ‘Frontiers in Participatory Evaluation’. Considering he has been one of the key influences on how we approach development research, it was quite a privilege to be two feet away from him, discussing my work!

The interactive participatory workshop comprised of many interesting games, exercises and knowledge focused presentations. He began the day with comparing evaluation to ‘cooking’; where he mentioned that a good evaluator (or chef) knows and applies a range of techniques and tools! His main message was that it is important to consider the Who/Whose questions, in terms of Whose voice? Whose reality, the importance to be constantly in touch with people we are working ‘with’ NOT ‘for’ and to understand that there is a constant change in conditions, awareness, aspirations, priorities of communities. He particularly was impressed with the gaining popularity of Participatory Statistics, which are defined as “a set of methods that enable local people to generate statistics for local level planning, learning and reflection, but which can also be aggregated at wider levels and feed into national level policy processes.”

Sharon Weir (Co-Founder, 4th Wheel) with Robert Chambers

Eventually, Robert Chambers concluded that “Innovation is in the gene of good evaluation” and the two most important aspects of good evaluation is good facilitation and harnessing the power of creativity. As example, I shared the ‘hypothetical budget’ tool, Payal and me conceptualized several years ago (to get ‘participatory statistics’ for a Social Return Investment Study), and have been using ever since. The tool according to him, signifies both – creativity and good facilitation. (So proud!)

Here is a short description of the tool:

Method

Community respondents are divided into groups of 4-5 people. Men, women and youth form separate groups. Each group is given a chart paper. The method can be used in two ways – One where participants are given a set of pre-determined options by the research team, for example: health, education, infrastructure, sanitation, water, etc.; and the other where the group should decide their own responses. Once this list is prepared, the group is given a hypothetical budget of INR.10,000 to spend on the list. Respondents can prioritize any way they want with no minimum or maximum spending specified. Scores (budgets in rupees) are then averaged for each indicator/option to understand community priorities.

Application

This method can be applied in baseline research, or in an evaluation study. It helps to understand community perception of priorities and needs, can be used to compare donor/program (percent) resource allocations with community needs (resource allocations), and serves as an inclusive triangulation tool among different stakeholders and heterogeneous communities. It also helps to build consensus on program priorities and outcome indicators among respondents.

Examples – This method has been used several times by the 4th Wheel Social Impact (4WSI) research team. Some recent studies include:

~Needs assessment study: A construction company commissioned 4WSI to conceptualize and design their Corporate Social Responsibility (CSR) initiatives for construction workers at their site. Workers came from different states of India, having their own small regional groups and specific residence locations at the labor colony. All residents of the labour colony were invited to the Focus Group Discussion where CSR plans of the company were shared; they were briefed about the exercise and told the rules of the game. In this particular setting, the groups were asked to list top five priorities in the labor colony first and then allocate budgets.

~Social Return on Investment (SROI) study: A CSR foundation had invested in improving the education status of 30 villages in Gujarat. We used this method to garner responses of diverse stakeholders- parents, teachers, local leaders, etc. We listed twelve education initiatives, before hand and asked respondents to do two things – One, allocate budgets to the different initiatives and secondly, put a price on the perceived benefits of the different initiatives. This helped to compare the foundation’s budget allocations to the (average) budget allocations of the different stakeholders. It also helped to gain a sense of the impact perceived by the community in monetary terms (this is slightly challenging).

Considerations

~ This method can build high expectations among a group. It is important to brief the groups on the aims and use of  results of this exercise.
~There is a possibility of some people dictating to the group. It is important to tell respondents to take everyone’s view and consensus, if needed, scores of the smaller group can also be averaged to arrive at one figure.

This article is written by Sharon Weir, Co-Founder of 4WSI. Published: http://the4thwheel.com/blog/2017/06/22/creativity-and-good-facilita...

[Rituu B Nanda]

Thanks for sharing Payal. What would you consider highest level of participation in research and evaluation?