Stakeholder Participation in Evaluation

[Ranjani K.Murthy]

My name is Ranjani.K.Murthy. I work as an Independent Researcher. I made a presentation on participation of stakeholders in evaluation recently in a Pune workshop. The presentation is attached hereStakeholderparticipationinevaluation.pptx

This presentation examines why stakeholder’s participation in evaluation is important; what is the present status of participation of stakeholders in evaluation in India; what are the challenges in getting the participation of stakeholders; and what are the ways forward for participation in evaluation. Gender intensified and gender specific constraints to women’s participation in evaluation are analysed.

Stakeholders in evaluation include marginalised communities and women amongst them, implementing organisations, evaluation team, and donor organisations. Their participation can be a principle by itself, can be a tool for enhancing the efficiency of the evaluation, and can be a tool for strengthening effectiveness of the evaluation. It is the first and last which should be important rationale for the participation of stakeholders in evaluation.

The present status of evaluations in India, with exceptions, is rooted in power hierarchy between evaluation team and marginalised communities, between evaluation team and implementing organisations, between evaluation team members based on gender, caste, race etc., and between donors and evaluation team. 

Some of the challenges to marginalised communities’ participation in evaluation include that the terms of reference is written and in a language which is not really know to them, the timing of the evaluation in the year, the time of the evaluation during the day and presence of caste and gender hierarchies.  Most of the general barriers like “English written TOR” have a gender intensified impact affecting women more than men as women’s literacy is lower and women have lesser knowledge of English

The challenges to implementing agencies’ participation in evaluation include the fear that evaluation is often linked to funding,  (at times) domination by evaluation team, the fact that they have to manage multiple donors, and field level staff’s world load and their lack of knowledge of English.  Majority of the field level staff are women, with few women being at the leadership levels. Similarly there are challenges to participation at the evaluation team and donor levels.   

Looking at ways forward the presentation would like to share one experience from India where the evaluation was commissioned by the implementing agency, and was seen as a learning process.  The terms of reference was evolved by the staff and the gender expert. Participatory and gender aware methods were identified and staff were trained in these methods. In addition interview schedules were used which were evolved collectively and in local language. Each staff did the impact assessment in another staff’s area and a senior staff consolidated the findings.  The focus of the evaluation was to assess gender and poverty impact, and methods such as gender division of labour mapping,  gender based access and control over resources mapping, body mapping, women’s access to institution mapping and happiness index were used.  The information which emerged was triangulated with information from focus group discussions with groups and data from government service providers like schools, nutrition centers, health facilities etc.  A before after comparison (and reasons for difference mapping) and member and non-member comparison (and reasons for difference mapping) was made to ascertain causality of change.  Such a process was owned more by the implementing agency and to some extent by the marginalised communities/women themselves.    

I seek your inputs to the following questions:

i) Who are the stakeholders in evaluation?

ii) What are the constraints in participation of marginalized communities, and women and sexual minorities amongst them in evaluations?

iii) What are the constraints faced by implementing agencies in participation in evaluation? In particular by women staff?

iv) Would you have any experience to share on overcoming the constraints faced by marginalized women and sexual minorities and women staff in evaluation?

[26ud4hdar71g0]

i) Who are the stakeholders in evaluation?

stakeholders, as the name implies, are those who have stake in the project designing, development, implementation, and evaluation. Above all, stakeholders are those people who are directly or indirectly affected by the process or end result of the project. Oftentimes, those stakeholders who involve in the project designing and evaluation are quite different so that  unable to get the required/dreamed data. In short, i would say stakeholders are those affected by the project directly or indirectly, intended or unintended. Yet, the level of stakeholders varies depending on the magnitude of the effect. if it is  GBV project for instance, women and girls are highly vulnerable to the violence than men and boys, 

ii) What are the constraints in participation of marginalized communities, and women and sexual minorities amongst them in evaluations?

the first constraint is in identification of the right stakeholders in the evaluation  As mentioned above, we may let involve ppl who were not a part during the need and gap identification and prioritization  If  x was a part of the need assessment and project designing phase and Y is invited to involve in the project evaluation then the evaluation has failed from its start. How can Y evaluate X's need satisfaction? This is what happens usually with marginalized community and sex minorities. X decided for woman and she will be asked at the end of the protect to evaluate if x was correct on her behalf. we are also elite and think-tank driven, we are not good at grass root knowledge and community animation, we are not innovative to present ourselves at their level, we prefer to use jargon and technical words and examples than  simple and cultural sensitive. Most cultures doesn't encourage women to speak up and entertain her views yet we expect them to give us their reflection which is unrealistic. I think these are few of the constraints 

iii) What are the constraints faced by implementing agencies in participation in evaluation? In particular by women staff?

time bound and big deadlines from donors. If the project is late, then an evaluation will not be carried out. Evaluation is always a scapegoat for late or poor performance of a project. Budget is not sufficiently allocated to evaluations and learnings so that evaluations done quickly to satisfy donors expectation than to really learn and capitalize on them. If gender response staff is not there, women most likely will be left out 

iv) Would you have any experience to share on overcoming the constraints faced by marginalized women and sexual minorities and women staff in evaluation?

we usually empower sexual minorities lead the evaluation so that there will be a big eye who will look at the gender balance componnet. Or, we will empower other sexual majority to recognize the need and priorities of sexual minorities. Both approaches work

[Ranjani K.Murthy]

Dear Belete

Thanks for your very useful inputs - in particular on women's participation in evaluation and that of sexual minorities. I agree totally with what you have said!

Yes, it is true that at times late entrants into project come for evaluations and this gives a wrong impression about the project.

You have mentioned that women rarely speak in public, do they do so in private? If the woman in the evaluation team meets them on a one to one basis

With warm regards

Ranjani 

 

Belete said:

i) Who are the stakeholders in evaluation?

stakeholders, as the name implies, are those who have stake in the project designing, development, implementation, and evaluation. Above all, stakeholders are those people who are directly or indirectly affected by the process or end result of the project. Oftentimes, those stakeholders who involve in the project designing and evaluation are quite different so that  unable to get the required/dreamed data. In short, i would say stakeholders are those affected by the project directly or indirectly, intended or unintended. Yet, the level of stakeholders varies depending on the magnitude of the effect. if it is  GBV project for instance, women and girls are highly vulnerable to the violence than men and boys, 

ii) What are the constraints in participation of marginalized communities, and women and sexual minorities amongst them in evaluations?

the first constraint is in identification of the right stakeholders in the evaluation  As mentioned above, we may let involve ppl who were not a part during the need and gap identification and prioritization  If  x was a part of the need assessment and project designing phase and Y is invited to involve in the project evaluation then the evaluation has failed from its start. How can Y evaluate X's need satisfaction? This is what happens usually with marginalized community and sex minorities. X decided for woman and she will be asked at the end of the protect to evaluate if x was correct on her behalf. we are also elite and think-tank driven, we are not good at grass root knowledge and community animation, we are not innovative to present ourselves at their level, we prefer to use jargon and technical words and examples than  simple and cultural sensitive. Most cultures doesn't encourage women to speak up and entertain her views yet we expect them to give us their reflection which is unrealistic. I think these are few of the constraints 

iii) What are the constraints faced by implementing agencies in participation in evaluation? In particular by women staff?

time bound and big deadlines from donors. If the project is late, then an evaluation will not be carried out. Evaluation is always a scapegoat for late or poor performance of a project. Budget is not sufficiently allocated to evaluations and learnings so that evaluations done quickly to satisfy donors expectation than to really learn and capitalize on them. If gender response staff is not there, women most likely will be left out 

iv) Would you have any experience to share on overcoming the constraints faced by marginalized women and sexual minorities and women staff in evaluation?

we usually empower sexual minorities lead the evaluation so that there will be a big eye who will look at the gender balance componnet. Or, we will empower other sexual majority to recognize the need and priorities of sexual minorities. Both approaches work

[26ud4hdar71g0]

Dear Ranjani, 

Me too thanks for your reply ! I think women do speak freely when they are with other women, i would say familiar and peer women. For instance a rural somali woman speak more freely and confidently with another somali rural woman than urbanized Somalia women. similarity and peer approach i think is more effective. As you put it, addressing individual interview and free-talk would be more effective. Or even FGD of same ppl works allot. A women with disability will talk freely with another women with disability and this has been proved our project peer education. The peer education on GBV is carried out by women an girls with disability targeting the whole community and women and girls with disability specifically . And during these peer education, girls and women with disability grow speaking and communicating with the peer educators more freely, even than the neighbors. Similarity matter in communication and participation. even, check yourself who are your friends? who do you find it more easy to talk to? Are not most of such ppl share many similarities with you? in term so education, choice, test, fun, etc.. and Birds with same feather flies together. Love to hearing it from you too :)