Experiment
Outcome Learning Loop
Design interventions so Defra can learn whether actions produced the intended service, behavioural and environmental outcomes.
01
Problem to address
Defra can often measure transactions and uptake more easily than consequences. Without a linked evidence chain, policy teams cannot reliably tell whether an intervention worked or where to act next.
02
What the evidence tells us
Services often lack basic performance data. Strategy teams cannot always tell whether intended outcomes were achieved, and make choices within risk. The same intervention can work in some places and not others.
03
The proposition to explore
The idea
Connect policy intent, interventions, participation, real-world observations and outcomes so learning from what happened informs what Defra does next.
04
Hypothesis
If policy intent, intervention, participation, action, observation and outcome are deliberately linked, Defra can progressively improve targeting and stop treating delivery completion as evidence of impact.
05
How it could work
Define the outcome chain when an intervention is designed. Instrument relevant service and operational events, retain longitudinal evidence, bring in real-world observations, and feed learning back into the next targeting or policy cycle.
Today
- POLICY
- SCHEME
- APPLICATION
- ACTION
- ???
Future model
- INTENDEDOUTCOME
- INTERVENTION
- PARTICIPATION
- ACTION
- OBSERVED CHANGE
- OUTCOME
- LEARNING
- ADAPT POLICY↺
06
Example: bring it to life
Today:
“Pay farmers to plant sphagnum.”
Future:
Where does sphagnum establish successfully?
Combine:
- intervention
- soil
- hydrology
- elevation
- land condition
- farmer observations
- field officer evidence
- environmental monitoring
- time.
DASH could eventually reveal:
“On free-draining sites with characteristics X/Y, establishment is consistently poor.”
The next scheme changes.
And the farmer eventually sees:
Based on what we know about this land, this intervention is unlikely to succeed here.
That connects the strategy lead and upland farmer's questions:
Where should we intervene?
and:
Will this work here?
They are actually the same question at different scales.
07
AI opportunity
AI can join heterogeneous evidence, surface patterns and generate hypotheses. Causal attribution still requires appropriate evaluation methods and should not be delegated to a language model.
08
Potential value
- Stronger policy evaluation
- Better targeting of future investment
- Clearer line from service activity to outcomes
- Foundation for ANTICIPATE and simulation
09
Reduce uncertainty without overcommitting
Smallest meaningful experiment
Choose one intervention, one geography and one observable outcome. Build the smallest complete evidence chain and test whether it changes a real policy or operational decision.
10
Open questions and risks
What we still need to learn
- Environmental outcomes may take years
- Confounding factors make attribution difficult
- Outcome measures must be agreed early, not retrofitted