Ways to effectively work with research/evaluation agencies

[RITESH LADDHA]

Hello all, I am a monitoring and evaluation practitioner, currently associated with a reputed national NGO, and I have more than 15 years of experience in the development sector. During this journey, I have collaborated with multiple research/evaluation agencies to execute research or evaluation-related activities.

Working with an agency is not always a cakewalk as I have mixed experience with them. In our sector, we primarily recruit agencies for two main reasons – 1) to ensure the external validity of the assessment and 2) to fill the resource gap in executing such studies, as NGOs usually don’t have such a workforce in-house.

Based on my experience, to work with agencies smoothly/effectively, I am jotting down a few points that need to be considered. These may sound very common but trust me; I didn’t come across any such compilation. 

 Before finalising the agency -

• Draft your terms of reference (TOR)/Scope of work as precisely as possible. You must include ‘areas of inquiry’ for each target group. You may also indicate a preferred way of collecting data (quantitative/qualitative) for each inquiry area.
• While selecting any agency, please consider the team structure and profile. Please give special attention to the gender and inclusion aspect in the team proposed. Please ensure that the proposed/finalised team works on the study for the entire duration.
• Please listen to all shortlisted agencies (physical/virtual) about their understanding of the TOR and proposed solution. Please also give attention to the ethical practices offered by the agency.

 After selecting the agency -

• Please do the inception meeting with the agency and provide them with all the relevant information. The inception report should have all the operational details of the study.
• Please ensure all data collection tools are vetted through the M&E and respective programme teams.
• Please participate in the orientation/ training programme of enumerators on data collection tools. This is crucial to know how they are going to administer the questionnaire.

 During data collection -

• Please observe a few data collection sites to get to know the enumerators’ capability, response rate, and quality of response.

 During data analysis –

• Please discuss with the agency and finalise all the required tables that need to be generated from the data. This would work well for descriptive analysis.
• For any inferential analysis, again, based on your experience and agency experience, list all the required tables. Both these steps will help you get the relevant information from the study and reduce the time for back and forth during the report review.
• Once tables are discussed, please finalise the report's chapterisation plan/flow of report.

  On receiving the draft report –

• Please ensure the draft report is vetted through the M&E and respective programme teams.
• Please ask the agency for the raw dataset and analysis file/do file (STATA) / SPSS syntax. Please validate and run them on a sample basis. It will ensure the quality of data analysis.

 I hope these points are helpful and practical to adapt.

[Sakshi Tekam]

This is very useful and insightful for me who is just starting. Thank you for the suggestions. Keep posting

[Marion A. Cabanes]

This is very useful information! Thank you!

[Audria Choudhury]

We're actually looking at agencies to work with for a potential tool validation, so this is perfect timing! Very useful, thank you.

[Laia Luthi Solé]

Thanks Ritesh,

I agree with this point "Please ask the agency for the raw dataset and analysis file/do file (STATA) / SPSS syntax. Please validate and run them on a sample basis. It will ensure the quality of data analysis." because I have had an experience where I witnessed that the data was not being collected correctly, and having this point in the ToRs helped bring up the issue.

I would also like to share that, in my experience, participating in (some of) the data collection process itself was very interesting for me and for the agency, and gave me more information to review the analysis. Of course, this takes time, but it still worth it if you can only be there for a couple of days.

cheers,

Laia

[Alpaxee Kashyap]

Thank you so much for posting this Ritesh, really useful!