Report: Measuring the impact of humanitarian aid: A review of current practice
This report investigates how the impact of humanitarian aid is measured. It maps out the existing concepts, methods and practises of impact assessment used by humanitarian actors. The focus is on the mechanics of measurement and analysis, and it looks at how this is implicated in the humanitarian system, how it is conceived and how it operates. The report asked questions around how impact can be measured, why this is increasingly being demanded, and whether it is possible to do it better. It also explores the benefits, dangers and costs that paying greater attention to impact might entail.
Measuring the impact of humanitarian interventions is difficult because of a lack of basic data. It faces different problems - for example, relief interventions are often of short duration, insecurity may limit access to populations, and the space for analysis and research is constrained. Furthermore there is the risk that the measurement of humanitarian aid becomes to technical and that it doesn’t take sufficient account of the principled endeavour in which the process as well as the outcome is important.
Analysing the impact of a humanitarian intervention is not straightforward. A number of constraints and factors particular to humanitarian action make impact measurement difficult. These constraints are difficulties of the operating environment, the need to act quickly in situations of immediate crisis, an organisational culture that values action over analysis, and the fact that there is little consensus around the core objectives of humanitarian aid.
There are several tools that measure the impact of humanitarian intervention. However this report concludes that organisations have limited skills and capacity to use the existing tools, and as a result there is a need for greater investment in the skills and capacities needed to demonstrate impact more robustly. Overall, there has been significant under-investment in evaluation and impact analysis according to the report.
The report proposes that in order to improve impact analysis
- Measuring the impact of humanitarian aid should not be restricted to the project level. More system-wide evaluations are required;
- Project-based approaches should be adopted that focus on determining the impact of a particular intervention through a pathway from inputs to impact;
- Impact should not be limited to the evaluation process. Impact should be considered in ongoing monitoring processes, and through techniques such as real-time evaluation.
Bibliographic Details
Author: Charles-Antione Hofmann, Les Roberts, Jeremy Shoham and Paul Harvey
Publisher: 2004, Overseas Development Institute
Executive summary
Introduction
Scope and methodology
The impact of humanitarian aid: definitions, objectives and context
Definitions
Defining ‘impact’
The origins and uses of impact assessment
The levels and objectives of impact analysis
Conceptual frameworks
The levels of impact analysis
The functions of impact analysis
Assessing negative impacts
Perspectives from new public management
Perspectives from the private sector
Chapter summary
Measuring and analysing impact: methods, indicators and constraints
Methods for measuring and analysing impact
Participatory approaches
Indicators
Mortality
Surveillance systems
Surveys
Constraints to measuring impact
The emergency context and the nature of the system
Lack of consensus over the objectives of humanitarian aid
Lack of baseline data and seasonality issues
Control groups and regression analysis
Problems of attribution
Impact assessment beyond the project level
Measurement or analysis?
Chapter summary
Impact assessment: current practice in the health and nutrition sectors
Impact assessment and the health sector
Impact assessment and the food and nutrition sector
General ration programmes
Selective feeding programmes
Interpreting nutritional status data
Micronutrient status
Livelihoods approaches
Chapter summary
Conclusions and recommendations
Beyond the project level
Measuring, analysing or demonstrating impact?
The evidence-base of humanitarian aid
The new public management agenda
The way forward: approaches to impact assessment
References