Over the years the Early Action partners - now partners on Explore Adapt Renew have grappled with the challenge of evaluating the impact of our Early Action services. In this blogpost we share the tools we are using to measure and demonstrate the impact of Early Action.
Why is measuring prevention so hard?
- Clear causality is difficult: it's hard to prove that a preventive measure directly caused the absence of a crisis because establishing causality is complex.
- Long timeframes: through Early Action we aim to avoid future negative outcomes, which can take a long time to occur. This means that it may be challenging to observe the final impact of prevention efforts, especially if working with individuals or groups for a limited time.
- Multiple influencing factors: we operate within complex systems with many variables. Isolating the specific impact of a preventive intervention becomes difficult when other factors also contribute to the outcome, making it challenging to determine the exact contribution of prevention and de-escalation measures.
- Ethical considerations: it would be unethical to exclude a comparison group from receiving a potentially beneficial intervention in order to establish its impact, particularly if there are ethical concerns about withholding support.
- Data availability and quality: obtaining reliable and comprehensive data on all the ways organisaitons support people is difficult. Biases, limitations, and incomplete reporting can affect the accuracy and reliability of measurements.
How have we tried to overcome these challenges?
While these factors make measuring Early Action projects complex, we have found some ways to gather useful data, understand change and demonstrate impact.
- All partners have worked to strengthen their organisation's MEL systems, processes and culture. Read our blog on Strengthening MEL for Long-term Change
- We created an overall partnership Theory of Change for Early Action services
- We have collectively agreed indicators for measuring outcomes at early, mid and late stage Early Action services
- We have developed tools to capture the impact from the perspective of beneficiaries and from those designing and delivering these services.
Our MEL Framework and Tools
Theory of Change - Early Action Pathway
Early Action Outcomes and Indicators
Tools we are using
- Beneficiary Interview - click here to view
- Organisation Case Study - click here to view
- Six month report - collecting quantitative indicator data
We hope it’s useful to see the outcomes, indicators and tools we have designed for our Early Action work. Please feel free to adapt any of this to your needs. If you have any questions or feedback - please get in touch with the EAR team.
Lastly, we would like to thank all the learning teams who have helped us on this Early Action evaluation journey so far.
