COMPLETED_TXN_COMPONENTS table is never cleaned up unless Compactor runs

The Initiator thread in the metastore repeatedly loops over entries in the COMPLETED_TXN_COMPONENTS table to determine which partitions / tables might need to be compacted. However, entries are never removed from this table except by a completed Compactor run.

In a cluster where most tables / partitions are write-once read-many, this results in stale entries in this table never being cleaned up. In a small test cluster, we have observed approximately 45k entries in this table (virtually equal to the number of partitions in the cluster) while < 100 of these tables have delta files at all. Since most of the tables will never get enough writes to trigger a compaction (and in fact have only ever been written to once), the initiator thread keeps trying to evaluate them on every loop.

On this test cluster, it takes approximately 10 minutes to loop through all the entries and results in severe performance degradation on metastore operations. With the default run timing of 5 minutes, the initiator basically never stops running.

On a production cluster with 2M partitions, this would be a non-starter.

The initiator thread should proactively remove entries from COMPLETED_TXN_COMPONENTS when it determines that a compaction is not needed, so that they are not evaluated again on the next loop.