Uploaded image for project: 'Kudu'
  1. Kudu
  2. KUDU-2824

Make some tables in high priority in MM compaction

    XMLWordPrintableJSON

Details

    Description

      In a Kudu cluster with thousands of tables, it's hard for a specified tablet's maintenance OPs to be launched when their scores are not the highest, even if the table the tablet belongs to is high priority for Kudu users.

      For example, table A has 10 tablets and has total size of 1G, table B has 1000 tablets and has total size of 100G. Both of them have similar update writes, i.e. DRSs have similar overlaps, similar redo/undo logs, so they have similar compaction scores. However, table A has much more reads than table B, but table A and B are equal in MM, their DRS compactions are lauched equally, we have to suffer a long time util most of tablets have been compacted in the cluster to achieve a fast scan.

      So, maybe we can introduce some algorithm to detect high priority tables and speed up compaction of these tables?

      Attachments

        Activity

          People

            acelyc111 Yingchun Lai
            acelyc111 Yingchun Lai
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: