Power, Policies and Social/Gender Relations: Can Evaluations Change the Equation?

[Ranjani K.Murthy]

Cross posted from http://www.comminit.com/global/content/power-policies-and-socialgen..., January, 20, 2015

The Context

In the year 2013, the UN Evaluation Group decided to join EvalPartners in declaring 2015 as the International Year of Evaluation. The year 2015 was identified as a strategic year since the evaluation community seeks to mainstream evaluation in the development and implementation of the forthcoming Sustainable Development Goals. The United Nations Capital Development Fund (UNCDF) notes that the EvalYear will position evaluation in the policy arena by being a catalyst for important conversations and thinking, at international, regional, national, and sub-national level, on the role of evaluation in good governance for equitable and sustainable human development (UNCDF, 2013). Twenty six events on evaluations are taking place in different countries in 2015 (listed here). It appears that it is mainly the evaluation community, donors and policy makers who are taking part in these events.

Power, Policy and Social/Gender relations

Locating evaluations in the policy arena is important as policies do contribute towards equitable and sustainable human development. The Oxford Dictionary rightly defines policies as "A course or principle of action adopted or proposed by an organization or individual". That is the policy arena is simply not about policies of government and donors, but it is also about policies of men and women in families (do men allow their wives to work? Do women and men welcome the birth of girl child and prevent female genital mutilation?).

Policies, equally, are made by leaders of community institutions. Do the privileged leaders take decisions in favour of minorities, women, Dalits, Blacks, differently abled, LGBTs [lesbian, gay, bisexual, transgender], etc.? For example, do the upper castes given Dalits equal access to and control over commons, allow inter caste/religious marriages, prevent bride trading, etc.? Policies are also about rules of those controlling markets. Do employers recruit without bias, pay just and equal wages, follow the labour laws of the country/international labour laws, provide maternity and paternity leave, care facilities, flexible working space and time, etc.? Yes, evaluations of 'national policies' are crucial, provided they are accompanied by evaluations of informal policies of marriage, family, community, markets. Gender and social relations of power mediate each of these institutions .

Using evaluations to challenge power, social relations and policy

Different evaluation strategies are appropriate for assessing policy shifts in each institutional side towards equitable and sustainable human development. The usual distinctions between different approaches to evaluations like appreciative inquiry (AI), outcome mapping, development evaluation, democratic evaluation, participatory evaluations are useful, but what may be required is evaluation approaches tailor made to each level.

At the marriage/family levels, pictorial or written diaries may be useful for evaluating changes in policies. The process of preparing diaries entails sitting with groups of women, LGBTs, children, differently abled or elderly from low income groups, asking them their indicators for equity within marriages and families (including violence, forced marriage etc.). One could add to the same (show what others have identified) and ensure that different domains of human development are covered. After consolidating indicators emerging from at-least 5 groups, pictures could be drawn and printed in diaries representing these indicators. If literacy levels are high, there is no need to develop visual diaries; written diaries would do. The diaries could be used for self-assessment by the women themselves once in three months, and lessons could be drawn on why some women show improvement and others do not, or why, on some aspects, there is improvement, but not in others (Noponen, n.d). Awards could be given to those men or elderly women who show progressive attitudes, practices, etc. This method, which has been modified in this blog, was developed by Helzi Noponon and has been tried by NGOs [non-governmental organisations] in India.

At the community level as well, such a process could be adopted with indicators being developed with support by Dalits, Blacks, landless households, minorities, LGBTs and differently abled, covering spheres of equitable and sustainable human development. Progress may be rated on issues such as equitable access/control over common property resources, facilities in the village, access to roads, decision making, access to burial ground, child sex ratio etc. Just practices around eating together and marriages may be other indicators. Lessons could be learnt across villages or slums as to why some habitats are progressing towards equitable and sustainable human development while others are not. Awards could be institutionalized by NGO or governments to institutions which show maximum progress.

Assessing changes in policies of markets is complex as there are labour markets, financial markets and commodity markets. It may be useful to develop indicators with workers, borrowers, small scale producers and petty traders, and then examine what has changed changing over a period of time and why (positive and negative). For example, number of days of employment for different groups, wages and gender gap in wages, enrollment in labour welfare board, enrollment in trade union, absence of sexual- or caste-based harassment in work place, child care facility at place of work, etc., may be aspects that emerge from workers. Apart from labour assessments of progress towards equitable and sustainable human development, it may be useful to evaluate policies and performance of banks - on financial services for marginalized - and performance of manufacturing and service industries - related to employment of workers from marginalized groups, their salary and wages, their access to training and promotions, nature of employment (permanent vs. contract) and safety at work place/ while traveling, etc.

Policies of state can be assessed by the marginalized communities, as well as external evaluators. An example of community based evaluation is assessment by slum women in Chennai, South India, of the government’s food distribution system, school, health facilities, cooking gas supply and flood relief provision from a gender/equity lens. They evolved criterion of what they wanted from each service and rated each aspect on a 1-5 scale (higher the number the better).  For example, they evolved five criterion for assessing public distribution outlets of food grains, oil, pulses, sugar, etc.: timeliness, reasonableness of price, lack of corruption, availability of all items and no abuse of women, and, based on their rating of each criterion, the average score was 1.4 on a scale of 1 to 5 (see Murthy, 2015). Emerging issues were taken up with government at higher levels. Of course, (marginalized) community assessment of state policies are not enough, and state policies have to be assessed on the issue of relevance, targeting, effectiveness, efficiency, sustainability, using mixed methods. Nevertheless, both evaluation methodologies have to go hand-in-hand.

In Conclusion

Equitable and sustainable human development can materialize only if formal and informal policies of state, market, community, family and marriage are attuned with these goals and implemented. Evaluations of these written and unwritten policies must be gender/socially transformative and must involve marginalized groups at all institutional levels. They may be combined with evaluations led by formal institutions, adopting mixed methods. However, the latter alone is not enough. That is, external evaluators need to share their power with the marginalized for evaluating policy and challenging power and social relations. The process of evaluation itself needs to be socially/gender transformative.

References

Murthy, 2015, Toolkit on Gender-sensitive Participatory Evaluation Methods, Indian Institute of Social Studies Trust, New Delhi

UNCDF, 2013, Declaring 2015 as the International Year of Evaluation  

Noponon, H, The Internal Learning System-Assessing Impact While Addressing Part...

[Ranjani K.Murthy]

Dear Rituu

I would like to refer to two different experiences. The first in Bangladesh with a woman who was poor five years ago and had shifted to being non-poor (in her estimate) and with assets on her name. The discussion explored the road she and her household member traveled, what contributed to change-the IFAD project and otherwise and challenges in transforming gender and economic relations. I used 'road map of change' evolved by Linda Mayoux with modification.

The other experience is the discussions with a collective of dalit women in a village in Kancheepuram where the NGO Guide works. The discussions were held in an area were there was a huge poster of a girl baby whose birthday celebrations had just got over. I asked them what had changed in their dalit hamlet in the last ten years, why and how.  The women pointed to how they had fought for a water tank in their village and opening of an ICDS center in their village after a lot of collective struggle. Girl child was traditionally valued in dalit community and in between they had started following the upper caste. Now the girl child was being revalued. Yet another change was the greater confidence with which they walked into the Caste Hindu are- being organised privileged groups were scared of imposing restrictions   Women do not think  through around practical and strategic gender interests; both are important and the means adopted - struggles is most crucial to their sense of empowerment  

[Rituu B Nanda]

I loved this write up Ranjani. The last sentence sums it up beautifully- the evaluation itself needs to be an empowering and transformative process for both communities and evaluation team. 

Please can you share your best experience in engaging the communities in evaluation? Why was it the best? Thanks a lot for generously sharing your experiences with us and an opportunity to learn from you. Warm greetings!