What is your biggest concern for evaluations during COVID-19?

[Elizabeth Dyke]

I hosted a Zoom call on Thursday, April 9th, 2020 to discuss evaluation in international development during COVID-19.

I asked people when registering for the call (about 75 people) to identify their biggest concern for evaluations at this time. Then I grouped these by theme. Below is this list. 

What are your biggest concerns?

What is your biggest concern for evaluations during COVID-19?

Comments from Registrants on Discussion on Evaluation in International Development

with Canadian Association of International Development Professionals April 9, 2020

Led by Elizabeth Dyke, Independent Health and Social Development Consultant

[e-mail]

Overall Impact on Human Life

• The impact of response given aspects of its transmission, severity and health care efforts
• Catastrophic level of loss of life in developing countries

Overall Impact of COVID-19 on Programs

• That COVID-19 will likely have huge negative repercussions on other health programs
• How do we make sure a multi-sectoral approach to pandemic is prioritized, with programming, response, and for evaluation?
• Regular programming has stopped
• Health Economics implications

Overall Impact of COVID-19 on Evaluations

• How COVID-19 will affect data?
• Programme implementation distortions reducing evaluability
• Practical conduct of evaluation during Covid-19
• Measuring secondary impacts of COVID-19 on program outcomes
• Impact of COVID-19 on results
• Recidivism due to COVID-19 and program results reflecting that
• Delays between intervention and data collection
• Going through the motions of evaluation when it is clearly the wrong time
• Not having enough response information
• Implementation challenges
• How halting interventions for different periods of time impact on evaluations and what should be minimum requirements
• Key stakeholder priorities have shifted

 Equity Issues

• Gender, race and class dimensions
• Going beyond elites, hearing the voices of the hard to reach
• Insensitivity to the realities of people in dire straits
• How to ensure that the uncounted are made visible during the research process despite increased mobility constraints
• Inability to reach marginalized and vulnerable people for inclusion in adapted data collection practices
• Food security

Safety Issues

• Putting communities at risk
• Health and safety of workers
• Data collection, safety, security, ethics, field work and observations which are not possible during COVID
• Emotional wellbeing of children

Design and Data Collection Generally

• Data Collection
• Access for data collection
• Field data collection
• Getting the data
• Evaluation design and data collection constraints
• Qualitative data collection on (off) the field
• My biggest concern is how to collect qualitative data during covid-19
• Reaching the right people
• Gathering data from clients
• Interaction with clients/beneficiaries
• L'organisation pratique de l'évaluation dans ce contexte

Adaptation that is Needed (specifically and generally)

• How to rapidly re-scope/re-tool already-begun evaluations to deliver information that helps tackle the pandemic
• Evolving evaluation practices need to align to the new social paradigm
• Adaptation
• Not having knowledge/capacities to respond

Impact of Travel Restrictions

• Inability to travel overseas and collect data
• Travel restrictions preventing us from doing field work
• Travel to and from
• Field verification
• How to carry them out given restrictions on travel, meetings and gatherings
• Not being able to collect data due to movement restrictions
• In-country movement restrictions
• Mobility to conduct evaluations on the ground

Shift to Local Data Collection?

• Field data collection and training data collectors
• Smooth transition to localization
• Access and Accountability
• In person data collection
• When can we consider re-starting data collection activities?

Remote Data Collection including Challenges

• Less information is available using ICT for interviews than using site visits and in person
• How to conduct robust research in areas with limited digital access
• How to transition to remote evaluation and closer collaborative linkages with Southern partners
• Collecting data remotely
• Conducting /facilitating innovative approaches through online technology
• How to support capacity building of our colleagues in the south
• How do we effectively evaluate remotely/online?
• I was asked if evaluations interviews can be done online. I personally doubt that and wonder how can we move forward?
• Remotely doing multi country evaluation of community based adaptation and livelihood projects
• Les moyens de vérification de la fiabilité des données

 That Evaluations will not be valued/will not be done/will not be rigorous

• That it will be skipped
• That evaluations will stop
• The importance of evaluation
• Ensure that rigorous evaluations take place to continue informing sound public policies, programs and projects
• Cancellation/ scaled down to the point of evals losing their value
• There are so many other needs at the moment that evaluations will not be given attention needed

Overall Impact on Evaluations and on Businesses

• Business Management
• Lower opportunities for evaluation contracts
• Impact on on-going Monitoring and Evaluation assignments