Recently I was in Afghanistan working with an NGO to establish a baseline for a project aimed at improving the participation of women in civic and political activity and also improving women's empowerment. The NGO had engaged me to help them with establishing a baseline after someone had given them my name as a practitioner who has a preference for practical, understandable methods where people can engage on their terms, in preference to using set surveys to extract data in response to prescribed questions. I hope the distinction I have briefly described here is clear because I think it is a critical distinction and one which is vital if disadvantaged groups are to have their voices heard.
When I arrived at the NGO's office I soon found that not everyone was as keen on my approach as I had come to believe. The M&E staff especially were keen to develop a survey, recruit and train people to take the surveys out and interview the statistically selected few who would be deemed as representative. After all this is how they had always done baselines.
So the first day or so was spent distinguishing between a baseline which measures outputs (such as, how many community change groups established, how many mullahs trained in gender in Islam), which is what the M&E staff wanted - "because we have a responsibility to the donor to show we have done the things we said we would do" - and a locally/community owned baseline set at the outcome/impact level (based on how will the women know if they are more empowered or are participating more?). My argument being that because the project has not yet started the baseline for outputs is zero for every output. However, if we could get an idea what the current situation is in terms of the project outcomes then we could do some serious monitoring the of project's theory of change during the implementation of the project rather than waiting until the end of the project and then conducting an evaluation. Another good reason for good outcome/impact level monitoring by project stakeholders is the fact that access to communities by outsiders, such as myself is not recommended for security reasons so the likelihood of the conventional, independent evaluation at the project's end would at best be another survey based, data extraction process which is not much use to those providing the data because they will probably never see the resulting report - and what would it mean to them anyway.
Because the objectives of increased participation in civic and political activity and increased empowerment for women were already determined we decided to decided to bypass the 'ideal situation' where women might determine their own objectives and move on to determining indicators.
To determine impact one has to get the evidence from those who are the so called beneficiaries of whatever the project is - in this case women in specific locations. The first step in this process is to ask those women (as many as possible) the questions, "how would you (individually/collectively) know if you had greater participation in civic and political activity and how would you know if you are more empowered?" The answers to these questions are the indicators of outcome/impact. They are the indicators which will show whether the change desired by both the women and the project implementer and the donor has occurred, is occurring or not - and what the women say is the evidence.
To establish the baseline for the project is then a matter of establishing the status quo in relation to the indicators. This process means that it is not just the measure of the indicator which is determined by the women but the indicator itself. This results in a baseline owned by those to whom it refers. When this process is repeated periodically using the same measuring tools it is then easy to see if the desired change is happening or not. If it is not happening, those women who have said that it is not can be involved in the decision making regarding what to do about the situation. Because the most important aspect of monitoring is the decision made based on the information gained from monitoring, when the women are included they then have ownership over the content and the process of the monitoring. This is getting towards 'real' participation and 'real' partnership.
Anyway the above process is what was agreed upon. Because I couldn't got out to some of the target communities we trialed the process with NGO staff from other projects. Almost everyone was convinced that this process was closer to the rhetoric of real participation and even closer to 'grassroots' development. Project staff, and the staff of project partners are currently going through the baseline process.
The tool they are using is Dr. Ravi Jayakaran's Ten Seed Technique which I think is the best knowledge creation/data collection tool. I use it all the time.
A baseline should be an integral part of a project and no something which precedes the project. The baseline is the first formal monitoring activity and the production of the baseline, owned and understood by those to whom it refers, and it relating to outcomes/impact is the first project output. The periodic monitoring of these indicators, using the same tools is in effect a active, dynamic research process. At project end there is really no need for an evaluation only a substantiation of what is already known and a celebration of the project's successes.
Baselines should the first activity in a participatory monitoring process. We need to put more emphasis on the E and less on the M in M&E.
Add a Comment