Failover and recovery in presence of Quota

The presence of quota in the cluster changes

Quota complicates master failover and recovery in several ways. The new master should determine if it is possible to satisfy the total quota and notify an operator in case it's not (imagine simultaneous failovers of multiple agents). The new master should hint the allocator how many agents might reconnect in the future to help it decide how to satisfy quota before the majority of agents reconnect.

The allocator interface should be updated with some sort of recovery information, which will allow it to react properly (e.g. seize offers and hold off resources for some time).