E-discussion: Evaluating SDGs with an equity-focused & gender responsive lens (no one left behind)

[Rituu B Nanda]

The purpose of the consultations (18th Jan-18th Feb 2016)

Following the approval of the 17 Sustainable Development Goals (SDGs) by the UN General Assembly and by the international development community in 2015, EvalPartners (including EvalGender+) and United Nations Evaluation Group (UNEG)’s members have begun to form working groups to strengthen monitoring and evaluation systems to assess these different goals.  The purpose of the present consultations, organized by EvalGender+, UNEG and the Independent Evaluation Office (IEO) of UN Women, is to provide guidance to strengthen M&E systems to assess all SDGs with an equity-focused and gender-responsive lens, in addition to Goal 5 (Achieve gender equality and empower all women and girls) and Goal 10 (Reduce inequality within and among countries).  The above mentioned agencies and network, in collaboration with other strategic stakeholders, are planning to prepare a guidance note that will assist both actors directly involved in social equity and gender equality, as well as all actors involved with the evaluation of the SDGs, in ensuring that social equity and gender equality are adequately addressed in all of the SDG evaluations. It is intended to produce a first version of this Guidance note by June 2016.  The SDG strategy is to work through, and to help strengthen existing M&E systems at the national and local levels and consequently the focus of the present consultations is on indicators and approaches that can be implemented through existing M&E structures – many of which may have limited experience and resources to address social equity and gender equality issues.

In addition to their importance as stand-alone sustainable development goals, both of these are cross-cutting themes that must be integrated into the assessment of all of the other goals.  For example, the achievement of Goal 2 (end hunger), Goal 3 (ensure healthy lives and promote well-being for all at all ages), Goal 7 (ensure access to affordable, reliable, sustainable modern energy) and Goal 13 (take urgent action to combat climate change) – to mention only four, all have important gender dimensions that affect the achievement of these goals.  Similarly, there are social equity dimensions to all goals. To read more on SDGs visit the link "Transforming our world:  The 2030 agenda for sustainable development.https://sustainabledevelopment.un.org/post2015/transformingourworld

The consultations identify four important themes that must be addressed in the M&E systems to assess SDGs with an equity-focused and gender-equality lens, in addition to goals 5 and 10:

• Theme 1:  The relevance of “new metrics” (measurement tools and indicators) for the evaluation of SDGs from an equity-focused and gender-responsive perspective.
• Theme 2:  Evaluation and complexity – Dealing with the increasing complexity of development and interconnectedness of SDGs to ensure “no one is left behind”
• Theme 3:  Towards equity-focused and gender-responsive national evaluation systems – Multi-stakeholder partnerships to strengthen national evaluation capacities
• Theme 4:  Demand for and use of evidence from equity-focused and gender responsive evaluation to inform equitable development

Each of these themes has different implications in different countries, regions, sectors and according to the type of organization.  Consequently we invite you to share your experiences and perspectives to assist the EvalGender+, UNEG, UN Women IEO teams in ensuring that the Guidance Note will reflect the diversity of experiences and perspectives in different countries, regions and types of organization.

 

Theme 1:  The relevance of “new metrics” (measurement tools and indicators) for the evaluation of SDGs from an equity-focused and gender-responsive perspective.

In recent years a number of “new metrics” have evolved which can potentially widen the range of indicators and measures available for the monitoring and evaluation of development results from equity and gender responsive perspectives. These include:

• Data that can now be collected through mobile phones, tablets, internet, GPS mapping and other new information technologies
• Big Data collected from satellites and drones, remote sensors, analysis of twitter and social media, mobile phone records, digital electronic transfers including purchase of mobile-phone air time and ATM withdrawals and crowdsourcing
• Participatory consultations (e.g. Most Significant Change, Outcome Harvesting, PRA)
• Concept mapping
• Mixed methods evaluations and
• Feminist research methods (e.g. oral history, feminist ethnography and content analysis, power relations, social justice and empowerment approaches)

Participants are invited to share their thoughts and experiences on the following questions (as well as others they propose)as they relate to equity-focused and gender responsive evaluation.

• In your experience what are some of the limitations of current data collection methods and the kinds of indicators they produce?
• What are the most difficult issues to measure with respect to social equity? and with respect to gender equality?
• What are the new challenges for assessing sustainable social equity and gender equality?
• What have proved some of the most effective methods?
• In addition to those mentioned above, what other new metrics are you familiar with?
• Which of the new metrics show the greatest promise?

Theme 2: Evaluation and complexity – Dealing with the increasing complexity of development and interconnectedness of SDGs to ensure “no one is left behind”

As SDGs are interconnected, national policies and programme to implement them will be complex. As programs grow in size and scope, the number of partners and stakeholders and in terms of the kinds of social and behavioral changes they seek to produce, they become more complex – both in terms of how they are designed and implemented, but also in terms of how they must be evaluated.  Complexity is defined in terms of: (a) the nature of the programme, (b) the number of partners and stakeholders and the patterns of interaction among them (including the level of consensus or disagreement among them on the goals of the programs), (c) the number of external (contextual) factors that influence how the programme is implemented and its outcomes and (d) the complexity of the causal chains through which outcomes are to be achieved.  A number of additional factors are particularly important for the evaluation of social equity and gender equality, including: (i) social and cultural constraints and pressures, (ii) the power relationships and social definition of gender relations and social equity, (iii) multiple influences on processes of behavioral change, (iv) the role of social media, and (v) the long, non-linear causal chains through which changes are produced.

Participants are invited to share their thoughts and experiences on the following questions (as well as others they propose) as they relate to equity-focused and gender responsive evaluation.

• Which dimensions of complexity are most important in your work on social equity and gender equality?
• How does complexity affect our understanding of the effectiveness of different interventions on the production of changes in social equity and gender equality?
• What methods and approaches have you found most effective for understanding the outcomes of complex programs on social equity and gender equality?
• The processes of change are long, involving many actors and contextual factors.  Also the processes are not linear as advances on one front often involve set-backs on others.  What kinds of evaluation strategies have you found most effective in these complex scenarios?
• What are the special challenges for understanding the impacts of different interventions on the most vulnerable populations?  What evaluation methods are most effective for studying these very sensitive processes of change?

Theme 3: Towards equity-focused and gender-responsive national evaluation systems – Multi-stakeholder partnerships to strengthen national evaluation capacities

 

The SDGs pose challenges for national evaluation systems as the SDGs require the involvement of a broader range of stakeholders, a broadening of the range of indicators to be measured and the methodological and organizational problems required to assess sustainability which requires collecting data over a much longer period of time.  Many programmes are intended to produce benefits that continue over five or even ten years and the evaluation must (ideally) continue over all of this period.  So instead of conventional evaluations that often only cover the 3-5 years of project implementation, the SDG evaluations may be required to continue for twice as long.  The application of a social equity and gender equality lens will often present additional challenges for national evaluation systems, including the fact that the evaluation of gender outcomes and impacts is often the weakest part of many national evaluation systemsand the methodologies for evaluating social equity are also not well developed in many countries (or in the evaluation literature in general).  Given resource constraints of many evaluation agencies, it will often not be possible to consider specialized evaluations that focus exclusively on equity and gender, and it will be necessary to adapt standard M&E methodologies to address these issues.  It will be important to consider the extent to which some of the multi-shareholder partnerships can bring in agencies with expertise in these areas and with additional resources that may permit the selective application of gender and equity focused data collection and analysis methodologies

 

Participants are invited to share their thoughts and experiences on the following questions (as well as others they propose) as they relate to equity focused and gender-responsive evaluation.

• In your experience what will be the main challenges that national evaluation systems will face when evaluating social equity and gender equality?
• In the countries with which you are familiar, how well established are the methodologies for evaluating these two areas.
• Which kinds of organization have the most experience in the evaluation of these two areas?  Are these organizations already part of the national evaluation systems?  If not, what will be required to ensure their active involvement?
• What kinds of evaluation capacity development will be required to strengthen the capacity of the national evaluation systems to address these issues?
• What are the example of successful partnership in your country or in your area of work to strengthen M&E systems in general, and equity-focused and gender-responsive systems in particular?
• What are the opportunities and challenges for such partnerships?

Theme 4:  Demand for and use of evidence from equity-focused and gender responsive evaluation to inform equitable development

 

Experience from all regions and sectors shows that one of the biggest challenges facing evaluation systems is the very low rate of utilization of evaluations.  In many cases evaluation findings do not reach many of the key organizations and groups (including community and women’s organizations), in other cases they are not presented in a form which is easily accessible to some groups, particularly the most vulnerable.  Even when evaluations are reviewed, action is often not taken on many of the recommendations.  These challenges are likely to be even more serious for social equity and gender equality as these themes are less familiar to many organizations and the mechanisms to review and action are often less developed.

 

Participants are invited to share their thoughts and experiences on the following questions (as well as others they propose) as they relate to equity focused and gender-responsive evaluation.

• In your experience what are the factors affecting the demand for and use of evaluation?
• Are there additional factors affecting the demand and utilization of social equity and gender equality evaluations?
• How could the demand and utilization of these evaluations be increased?
• What types of evaluation presentations would you propose to make evaluation more accessible to stakeholders?

 

 

 

[Rituu B Nanda]

Ponge, IT can definitely help. I was wondering what about sensitive issues. Would people share openly? What can we do that they share openly. I like to use community life competence approach. How do you encourage people to share experiences on sensitive issues?

[Rukmini Panda]

I am very happy to be a part of this discussion. I want to learn a lot from this. In my opinion, when we talk about M&E at national level, we have to be very careful at which level we are going to propose. It is very important to reflect the data from the ground. 

It has been a great concern to define "care work" of women in indicators. As the care work of women is not recognized as work, the status of women is greatly impacted by it. Hence, We may discuss on it.

Thank you Rituu and others

Rukmini

[Rituu B Nanda]

Hi Rukmini,

Thanks for your response. How can we ensure to as you say " reflect the data from the ground.". Would you have any experience to share?

Thanks

Rituu

[Rukmini Panda]

Dear Rituu

Yes, we have a couple experience on this. We had engagement on implementation of PWDV Act in Odisha. We did status reports and shred the findings with the WCD department at state and at district. Asked for developing M&E tool on a quarterly basis. After a long engagement, we had to develop the M&E framework and the government machinery used it which reflected data from the district level. Similar experience we have regarding Tribal Sub Plan. What we had experience is that research, presenting it at right forum and a continuous involvement may cause a systematic process of M&E.

Thanks for your initiative Rituu. Some other time I will write my other experiences.

Regards,

Rukmini

[Rituu B Nanda]

Thanks Rukmini. How can we engage the community in M&E? what can we do to get their voice and aspirations? After all the development projects and programmes for them? 

[Mary Nderitu]

Theme 3:  Towards equity-focused and gender-responsive national evaluation systems – Multi-stakeholder partnerships to strengthen national evaluation capacities

For national evaluation capacities to work needs multiple stakeholders (non-state actors and state actors) - individual and group actors, while this is true, the starting point is how does change happen if we have to achieve effective and sustained development. We look at the framework and context to ensure that they reflect and capture multi stakeholders and that the "change logic" is documented, monitored, tested and revised.

The kind of evaluation capacity development would be needed is shaped by: socio-cultural beliefs and practice; methodological appropriateness/ strategies used; power differentials - entrenched vested interest due to status quo; capture non-discrimination and advocacy for sustainable livelihoods; what were the contexts that affected how change happened; what was the process or pathway for change - is it demonstrative effects or cumulative progress; priorities of nations/countries; and does the evaluation focus on intended use - policy influence, decision making on more inclusiveness, new or re-programming since the ultimate is social justice for all and gender inclusion

[Rituu B Nanda]

Thanks for responding Mary! I like your point about documenting perspectives of different stakeholders. What about different stakeholders reflecting jointly on the data?

[Awuor Ponge]

Dear Mary,

Thank you for your response to this. I'm fully in agreement with your position. I only wanted to add that, as we seek to strengthen the national evaluation capacities, we need to have in mind the specific indicators that will show us whether we are making progress or not towards our targets. We need to ask ourselves, what do we intend to achieve through gender-responsive and equity-focused national evaluation systems? What targets do we set for ourselves over time? What are the indicators that will show us that we are on the right track? Of particular importance, will be, what will be the beacon to show us that we have finally attained our goal of a gender-responsive and equity-focused national evaluation system?

Thank you!

[1zr1j3q0xzczh]

Theme 1.national statistical offices often provide general statistics without disaggration of data by gender,geographical lication,rural to urban,children and the girl-child,to name a few.These frequently hide from the policy researcher and the public the real situations on which to base our interventions for a gender-equal society.We need to establish new mechanisms for collecting data on all situations faced by all people at all levels of the society.School drop out rates,child marriages and data on women dropping out of productive jobs and the labor force are the areas for which we need regularly collected data for our analysis and effective interventions and eventually support change with equality in mind.

[Rituu B Nanda]

Seyhan, thanks for your response. Would you have any new ways you have used for data collection to capture gender and equity lens?

[Tomasa Ravines]

Hello: 

Congratulations for this initiative! I am feeling happy to participate. I want to share some comments.

Theme 1 ¿What are the most difficult issues to measure with respect to gender equality?

From my experience as an evaluator from a middle-income country, formulation and implementation of public policies for gender equality is a central issue. While there have been major achievements, the context remains adverse in many ways, depending on the gender problem, the conservative influence in public policies, the macho political culture in decision makers, etc., this define a difficult and uncertainty scenery,  in which you can move forward, halt or go backward. The evaluation has to answer these challenges. Therefore, the most difficult items to measure are the degree of efficiency and effectiveness of public policies geared towards gender equality. Also the degree of corruption which affects the execution of public spending. Another difficult aspect is measure sustainability because of the polarized and changing situations. It is important to consider in the evaluation strategic alliances, advocacy, participation of civil society, among other strategies. Unfortunately process indicators are important but sometimes they are not easy to be accepted especially when the pressure is to show the outcomes about cost-benefit.

¿What are the new challenges for assessing sustainable social equity and gender equality?

Considering the above, the challenges are: define smart basic process indicators managed by the country office PME system, which include the medium and long term, which can be supported not only technically but politically,  they should be easily communicated and legitimate with the different actors specially with functionaries and decision makers. It is important promote validations in the different levels and specialized areas to get consensus.  Another challenge is to invest in strengthening national evaluation capacities, one way could be promote the on line free evaluation courses from IOCE and UNWomen with the regional evaluation offices oriented to stakeholders.

What new metrics are more promising?

a. The Cloud: from my experience data collection through the cloud has been very efficient, it facilitates  handling large amounts of information, the access is easy, it is cheap and allows access to different participants, b. Facebook and Web pages: to collect information about messages, documents, etc., c. video: it is very nice to visualize the experiences in the field, the messages, testimonies and practices of beneficiaries, and d. Skype: it is very useful for interviews and meetings.

Have a great weekend. 

Tomy

[Rituu B Nanda]

Thanks for your response, Tomasa. What are the gender and equity issues in your country?