Donna Mertens and Transformative Mixed Methods: More justice-oriented methods are needed!

[Fabiola Amariles]

During four days from 20-23 September 2016, Donna Mertens, the well-known professor and author of several books on mixed methods in research and on diverse participatory and inclusive approaches to evaluation, conducted a course on Transformative Evaluation to 35 professionals of several disciplines from Chile, Bolivia, Mexico, Colombia, Argentina, and Paraguay in Santiago, Chile.  The course was sponsored by Fulbright Foundation, Universidad Santiago de Chile and American Evaluation Association (AEA) and supported by the initiative EvalYouth for Latin America, the LA&C Evaluation Network RELAC-Chile and CLEAR.

The course proved to be a great opportunity for strengthening the capacities of Latin American evaluation professionals for incorporating a transformative approach into evaluation, and at the same time it served as a forum for an active exchange of experiences among people from different countries, cultures and perspectives from our region.

We took this opportunity to interview Donna on her perception about challenges and opportunities for promoting the use of a transformative approach in evaluation in Latin America. Also to seek her opinion on the use of Equity Focused and Gender Responsive (EFGR) evaluations to contribute to social justice, from the perspective of the rich exchange of experiences that took place in the course.

We asked her to reflect on which were the three major topics of relevance that emerged from the discussions during the course. These are her impressions:

DM: First of all, I am overwhelmed with the emotional connection that we made with each other in the course. This was a wonderful experience that made me very happy!  Some topics that caught my attention were:

1)   Resilience: I think that despite the historical context of power structures that have been obstructive to development, Chilean people sincerely care about the poor and would like to see a change. The need to make visible unequal situations appears to be an important issue. I would advise on not to oversimplify what needs to be done or think that we can go now and change the world, that would be naive!  We need to put it in a challenge context in terms of history, power relations and economics.

2) Consciousness of evaluators about the need for evaluation models that involve justice-oriented frameworks:

It was not a surprise for me that people feel frustrated with forms and methods of evaluation that do not ensure involvement of the communities where they work. The transformative evaluation is an approach that offers strategies for addressing these challenges, based on the framework of justice and equity. It is necessary to align it with assumptions that take into account the context and specific needs of marginalized groups.

3) Areas that need additional attention: The discussions also led to some important issues that may require further reflection if the region wish to advance to social transformations:

• The use of evaluation networks to advocate for better evaluation;
• The recognition of important dimensions of diversity in our stakeholder groups: Use strategies and mixed transformative methods to explore the meaning of diversity and the uses of power in our stakeholder groups, including a focus on women and girls, people with disabilities, indigenous people, and those who live in poverty;
• Implication of awareness in response to diversity.

“We were lucky to have some economists in the training course that raised important issues about, for example, budgets, power and their relation with public policies. We are not giving enough importance to these issues”, Donna said referring to the formation of transformative evaluators who take care of people’s rights.

Another topic for discussion is the diversity of contexts and historical implications for social interventions in countries like Chile, Peru, México, and Colombia. Some countries do not have development cooperation funds; this fact changes the discourse and strategies for advocacy with decision makers and on how we approach different governments within countries. We need to have this into account when introducing transformative evaluation as well.

One important lesson from the course is how to approach the Transformative Evaluation, starting from the multi-phased evaluation design. Donna reminds that we should start with whatever understanding of the problem is, and prevents about the common expression of: “I know what the problem is and what the solution is”. This is a formula for failure.

Donna leaves us with a final reflection about the role of networks to contextualize strategies for the introduction of a transformative approach in evaluation. EvalPartners is already working with parliamentarians all over the world to create an evaluation culture and support human development at the country level through a rigorous production of evidence for social change. “We need to do more on bringing more partners into action, for example, keep the voices of EvalYouth and EvalGender+ active in the discussion. Together we can change the world for better! If we do not do it, it doesn’t happen”.

 

Interview by Fabiola Amariles ([e-mail]), EvalGender+

and Claudia Olavarría ([e-mail]), EvalYouth  

 

[Rituu B Nanda]

Hi Fabiola, please may I ask two questions-:

• Who is the evaluation in this case transformative for?
• What are the factors that facilitate this transformation?

Thanks. I can't attend such workshops and would love to learn from you.

[Donna Mertens]

It was an honor and a privilege to work with you all in Chile and people from other parts of Latin America. I am grateful for the opportunity and hope for more chances to work together in the future. Gracias por todos.