Testing in Real: Sample-efficient failure discovery of contextual failures with Bayesian active learning

Project Website Conference Paper

Experts can participate in root-cause analysis of failures by observing failures and providing possible failure modes.

In this work, we look at how we can combine expert-driven analysis for a multi-modal failure discovery in real-world platforms. Our pipeline consists of learning expert based feedback using GPs as surrogate models, and coverage-based active learning strategy called Expected Coverage Improvement (ECI) for limited data evaluation.

By balancing coverage with failure discovery in targeted regions we are performing efficient exploration with ECI. An example of what different techniques (optimization, ECI, and random walk) look like in a 2D search space for Push-T task in sim.