Corrigibility, n.

We are builiding multi-agent environments for studying the corrigibility at scale.

Can interactive agent-agent/human-agent games be used to distinguish agents that preserve their principal’s ability to notice, initiate, and complete correction from agents that are merely obedient, sycophantic, paternalistic, or overly conservative? And do these behavioral differences generalize across different games?

There is already a useful precedent in GPTNT, which turns Keep Talking and Nobody Explodes into a multimodal AI collaboration benchmark.

We are currently building environments designed around complementary corrigibility problems.

The first type of environments will focus on asymmetric information and vigilance. In these environments, the players will possess different information, and the subordinate agent will sometimes discover evidence that should cause the pricipal agent to reconsider the current plan. The environments will test whether the agent provides the information necessary for its principal to notice a problem, intervene, and successfully change course, particularly when doing so conflicts with task completion or approval.

The second type of environments will focus on commitment, delegation, and reversibility. The agent will be able to initiate multi-step actions, commit resources, and delegate work to other agents or processes. In these environments, some decisions will become increasingly expensive or impossible to reverse. The environment will test whether the agent preserves correction channels, avoids unnecessary lock-in, and propagates corrections through delegated systems.

We are set to gradually open-source the work. If you would like to contribute to these efforts, feel free to email alfaxad[at]nadhari.ai