Familiar simulations can hide failures
A policy can look reliable in a familiar simulation, then lose performance abruptly after a small change in dynamics or observation.
Deep reinforcement learning / robustness
Can we measure whether a policy will suit an upcoming environment before a failure at deployment? That is the question behind this dissertation project.
Interactive experiment map
Vary dynamics and observations separately, collect the policy’s behaviour, then use the meta-model to estimate suitability.
Case study
A policy can look reliable in a familiar simulation, then lose performance abruptly after a small change in dynamics or observation.
Generate environments with controlled shifts and collect policy behaviour and returns. A second-order model then links the shift signals to policy suitability.
The public repository brings together the DomainShift experiment code, tests, paper material, and associated dissertation.
Experiment detail
Establish how the agent behaves in the environment used for learning.
Alter dynamics and observations separately so it is clear which part of the environment changed.
For each controlled change, measure trajectories, state visitation, and returns.
Use a meta-model to predict whether the policy is compatible with the upcoming domain.
Evidence boundary
The project sets out the case for intrinsic shift quantification and implements it as a research framework. This is dissertation research; it does not give a universal guarantee of out-of-domain policy safety.