There is a hierarchy in how development and humanitarian work is measured, and almost all public communication takes place on the lowest rung of it. Understanding the ladder makes it immediately obvious which claims are being supported by evidence and which are being supported by phrasing.
The measurement ladder
| Level | Example | What it requires | Frequency in reporting |
|---|---|---|---|
| Input | $50,000 spent; 1,100 cartons procured | Accounting records | Common |
| Activity | Twelve screening rounds conducted | Activity logs | Common |
| Output | 3,400 children screened; 610 admitted to treatment | Programme records | Common |
| Outcome | 82 per cent of admitted children reached discharge criteria | Follow-up records with defined criteria | Uncommon |
| Impact | Mortality in the catchment fell relative to what it would have been | A comparison group or credible modelling | Rare |
Why “people reached” means so little
The phrase has no standard definition, which is exactly why it is popular. Depending on the organisation it can mean any of the following, and all of them are defensible interpretations of the same words.
- People who received a tangible item or service.
- People who attended a session, whether or not anything changed.
- People who were screened, including those found not to need help.
- Household members counted from a single recipient — one food parcel becoming six people reached.
- The population of a district in which a programme operated.
- The audience of a radio message or a poster campaign.
Because the same phrase covers all six, comparing two organisations’ reach figures compares nothing. The useful question is simply: reached with what, and how do you know they received it.
The counterfactual, which is where most claims quietly fail
Suppose malnutrition admissions in a district fall over a year in which a nutrition programme operated. The programme may have caused it. So may a good harvest, a road being repaired, a different agency’s work in the same district, a change in the screening protocol, or the population moving elsewhere.
Establishing that the programme caused the change requires knowing what would have happened without it. Rigorous approaches — randomised designs, matched comparison districts, difference-in-differences analysis — exist and are expensive. Their absence in a small organisation is entirely reasonable. What is not reasonable is asserting causation while lacking the comparison that would establish it.
Without a counterfactual, an impact claim is a story about a coincidence you happened to be present for.
The central problem in attributing programme effects
What a defensible claim actually contains
- A defined indicatorNot “improved nutrition” but a specific measure with a threshold — recovery to a stated discharge criterion, for example.
- A denominator610 children recovered out of how many admitted? A numerator alone conceals default and mortality rates.
- A time period and a locationClaims without dates cannot be checked against anything, and district-level work should not be described with national figures.
- A measurement methodWho collected the data, how, and with what sample. Self-reported figures from staff whose funding depends on them are weaker evidence than third-party verification.
- An acknowledged limitationThe most credible reporting names what it cannot establish. Organisations that state their own uncertainty are usually the ones measuring something.
Four claims that should prompt a question
| Claim | Why it is a problem |
|---|---|
| “$X saves a life” | Treats an average delivered cost as a guaranteed individual outcome, ignoring that not every treated child survives and not every survivor needed treatment |
| “100 per cent of your donation goes to the field” | Either untrue, or true because someone else is paying the overheads, or true because the overheads are not being incurred — which is a weakness |
| “We have reached 1 million people” | Almost always an aggregation of loosely defined contact across years and programmes |
| An identical success percentage reported for three consecutive years | Real programmes fluctuate; unchanging figures suggest a target being restated rather than a measurement being taken |
The honest position for a small organisation
an organisation cannot run a controlled evaluation, and pretending otherwise is worse than admitting it. But the alternative is not silence. Outputs reported with dates, quantities, named suppliers, destinations and losses are verifiable, cumulative, and become the raw material for genuine outcome measurement later.
HopePlates works on child malnutrition relief and therapeutic nutrition. We describe what a donation buys in inputs and outputs — quantities procured, suppliers, destinations, dates — and publish raised and spent totals rather than raised alone. We deliberately avoid converting those into lives-saved arithmetic, because the conversion requires assumptions we would be inventing rather than measuring.
Frequently asked questions
What is the difference between an output and an outcome?
An output is what the organisation delivered — children screened, cartons distributed, sessions held. An outcome is what changed for people — recovery rates, retention in school, illness avoided. Outputs are countable from internal records; outcomes require follow-up.
Why is “people reached” criticised so often?
Because it has no standard definition. It can mean anything from someone receiving treatment to someone hearing a radio message, so two organisations’ figures are not comparable and neither indicates benefit.
Is it wrong for a charity to say a donation saves a life?
It is a simplification that converts an average delivered cost into a promised individual outcome. The underlying figure is usually a real cost calculation; the framing overstates certainty, because survival depends on infection, distance, follow-up and much else.
Should I only fund organisations with rigorous impact evaluations?
That rule would exclude nearly every small and new organisation, including many good ones. A more practical test is whether output reporting is specific, dated and verifiable, and whether the organisation is honest about what it has not measured.
What does 100 per cent of your donation goes to the field mean?
Usually that a separate funder covers administration, or that the organisation classifies costs unusually, or that it is not incurring costs it arguably should. It is a marketing statement rather than an accounting one, and it is worth asking which of the three applies.
How can I check an outcome claim myself?
Ask for the indicator definition, the denominator, the period, the location and who collected the data. Any claim missing one of those five cannot be evaluated by anyone outside the organisation, including the organisation’s own board.
Sources and further reading
- Sphere Handbook — performance indicators and minimum standards, including recovery and default rate thresholds
- Core Humanitarian Standard on Quality and Accountability
- OECD DAC criteria for evaluating development assistance
- Methodological literature on counterfactual analysis and attribution in programme evaluation
- HopePlates transparency page — dated outputs, suppliers and spent totals