What's in a baseline?

[John Donnelly]

Recently I was in Afghanistan working with an NGO to establish a baseline for a project aimed at improving the participation of women in civic and political activity and also improving women's empowerment. The NGO had engaged me to help them with establishing a baseline after someone had given them my name as a practitioner who has a preference for practical, understandable methods where people can engage on their terms, in preference to using set surveys to extract data in response to prescribed questions. I hope the distinction I have briefly described here is clear because I think it is a critical distinction and one which is vital if disadvantaged groups are to have their voices heard.

When I arrived at the NGO's office I soon found that not everyone was as keen on my approach as I had come to believe. The M&E staff especially were keen to develop a survey, recruit and train people to take the surveys out and interview the statistically selected few who would be deemed as representative. After all this is how they had always done baselines.

So the first day or so was spent distinguishing between a baseline which measures outputs (such as, how many community change groups established, how many mullahs trained in gender in Islam), which is what the M&E staff wanted - "because we have a responsibility to the donor to show we have done the things we said we would do" - and a locally/community owned baseline set at the outcome/impact level (based on how will the women know if they are more empowered or are participating more?). My argument being that because the project has not yet started the baseline for outputs is zero for every output. However, if we could get an idea what the current situation is in terms of the project outcomes then we could do some serious monitoring the of project's theory of change during the implementation of the project rather than waiting until the end of the project and then conducting an evaluation. Another good reason for good outcome/impact level monitoring by project stakeholders is the fact that access to communities by outsiders, such as myself is not recommended for security reasons so the likelihood of the conventional, independent evaluation at the project's end would at best be another survey based, data extraction process which is not much use to those providing the data because they will probably never see the resulting report - and what would it mean to them anyway.

Because the objectives of increased participation in civic and political activity and increased empowerment for women were already determined we decided to decided to bypass the 'ideal situation' where women might determine their own objectives and move on to determining indicators.

To determine impact one has to get the evidence from those who are  the so called beneficiaries of whatever the project is - in this case women in specific locations. The first step in this process is to ask those women (as many as possible) the questions, "how would you (individually/collectively) know if you had greater participation in civic and political activity and how would you know if you are more empowered?" The answers to these questions are the indicators of outcome/impact. They are the indicators which will show whether the change desired by both the women and the project implementer and the donor has occurred, is occurring or not - and what the women say is the evidence.

To establish the baseline for the project is then a matter of establishing the status quo in relation to the indicators. This process means that it is not just the measure of the indicator which is determined by the women but the indicator itself. This results in a baseline owned by those to whom it refers. When this process is repeated periodically using the same measuring tools it is then easy to see if the desired change is happening or not. If it is not happening, those women who have said that it is not can be involved in the decision making regarding what to do about the situation. Because the most important aspect of monitoring is the decision made based on the information gained from monitoring, when the women are included they then have ownership over the content and the process of the monitoring. This is getting towards 'real' participation and 'real' partnership.  

Anyway the above process is what was agreed upon. Because I couldn't got out to some of the target communities we trialed the process with NGO staff from other projects. Almost everyone was convinced that this process was closer to the rhetoric of real participation and even closer to 'grassroots' development. Project staff, and the staff of project partners are currently going through the baseline process.

The tool they are using is Dr. Ravi Jayakaran's Ten Seed Technique which I think is the best knowledge creation/data collection tool. I use it all the time.

A baseline should be an integral part of a project and no something which precedes the project. The baseline is the first formal monitoring activity and the production of the baseline, owned and understood by those to whom it refers, and it relating to outcomes/impact is the first project output. The periodic monitoring of these indicators, using the same tools is in effect a active, dynamic research process. At project end there is really no need for an evaluation only a substantiation of what is already known and a celebration of the project's successes.

Baselines should the first activity in a participatory monitoring process. We need to put more emphasis on the E and less on the M in M&E.

[John Donnelly]

Thanks for all your comments. I did not mean to say "more emphasis on E and less on M". I actually meant to write it the other way around. That is, more emphasis on M and less on E. I failed to 'monitor' what I was writing and only later (evaluation) found that it was wrong - when it was (is) too late.

[Kanchan Lama]

Thanks for posting this critical experience. In situation like Afghanistan implementing participatory processes at grassroots level is really a critical issue, and I think this cannot be compared to other common situations, especially when it is done by a foreigner. The 10 seeds technique is very practical, yet it also requires to plan who applies the technique, and in which context. I also agree to your view that involvement of women is essential to build ownership..and would like to know more about your position for maximizing E over M...!

 

[Egidie Murekatete]

I would agree to focus on E and less on M, however M is also important element that keeps managers in a good postion to monitor the progress while waiting for the E outcomes. To avoid surprise and master your programme  implementation, focus on both and take into consideration the E cost and feasibility before project start-up!

[samia abdull hakim a hameed]

  Thanks MR John for yours efforts .

[Susanne Lucie BAUER]

Baselines yes, they often come not before but only when the project takes-off, and not always at the right time. We are experiencing this at present with a new GIZ project in Darfur, Sudan: talking about capacity building for reconstruction in development. Yet times of tension and confrontations between Sudanese Government forces and rebel groups continue in 2016. The development discourse trembles between humanitarian aid and development assistance, local partners are happy to have us among them, yet we have to strictly adhere to UNDSS rules & regulations, and cannot move freely. My task of conducting an Employment&Labour Market Analysis (ELMA) is not straightforward. Monitoring indicators is part of the puzzle, and project staff is relieved as long as these are low-profile, easy to fulfill and even more simple to understand.

[Rituu B Nanda]

Jyotsna, please could you share your Africa experience. i would love to learn from you. Thanks

[Kedar Nath Bhatta]

I agreed that developing participatory monitoring indicators with beneficiaries will certainly be useful  and enhance ownership on the outcome and impact result. therefore, it is better to focus on M rather than E. The continuous monitoring of the indicator on the basis of theory of change is important.

[Jyotsna Roy]

Thank you for a succinct piece. I agree with you John that "To establish the baseline for the project is then a matter of establishing the status quo in relation to the indicators. This process means that it is not just the measure of the indicator which is determined by the women but the indicator itself. This results in a baseline owned by those to whom it refers..." In my years of work I have evaluated projects with very poor indicators.

Participation  is a "value" which M & E people have to imbibe. 

I shared a similar situation with the Afar community in Africa. 

Thank you Ritu!!

[Rituu B Nanda]

Yes John we need to strengthen our monitoring ! I think working in social situations which are complex and dynamic, participatory monitoring or I would say self assessment by the communities is key. I will try out 10 seed technique. Participatory action research is a process which in my experience is very useful for continuous action and reflection.

[John Donnelly]

G'day Rituu,

It should say "more emphasis on the M and less on the E in M & E".