Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-22122

TxnHandler.getValidWriteIdsForTable optimization for compacted tables

    XMLWordPrintableJSON

Details

    Description

      When we do not find open writes for the given validTxnList then we either:

      1. do not have any writes on the table - we can return writeIdHwm = 0, and no invalid/aborted writes;
      2. we have only compacted writes on the table - we can return writeIdHwm = nextWriteId -1, and no invalid/aborted;
      3. we have compacted writes and some invalid writes on the table - we can return the lowest invalid write as a writeIdHwm and set it as invalid.

      What the current code does instead is sending writeIdHwm = nextWriteId -1, and sending every write as invalid. This results the same response in case 1-2, but probably a longer list in case 3.

      So we have place for some optimizations

      Attachments

        1. HIVE-22122.2.patch
          7 kB
          Denys Kuzmenko
        2. HIVE-22122.1.patch
          7 kB
          Denys Kuzmenko

        Issue Links

          Activity

            People

              dkuzmenko Denys Kuzmenko
              pvary Peter Vary
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 20m
                  20m