Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-19985

ACID: Skip decoding the ROW__ID sections for read-only queries

Log workAgile BoardRank to TopRank to BottomBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    Description

      For a base_n file there are no aborted transactions within the file and if there are no pending delete deltas, the entire ACID ROW__ID can be skipped for all read-only queries (i.e SELECT), though it still needs to be projected out for MERGE, UPDATE and DELETE queries.

      This patch tries to entirely ignore the ACID ROW__ID fields for all tables where there are no possible deletes or aborted transactions for an ACID split.

      Attachments

        1. HIVE-19985.07.patch
          32 kB
          Eugene Koifman
        2. HIVE-19985.06.patch
          33 kB
          Eugene Koifman
        3. HIVE-19985.05.patch
          40 kB
          Eugene Koifman
        4. HIVE-19985.04.patch
          28 kB
          Eugene Koifman
        5. HIVE-19985.01.patch
          10 kB
          Eugene Koifman

        Issue Links

        blocks

        Improvement - An improvement or enhancement to an existing feature or task. HIVE-20654 remove masking of "Masked writeid"

        • Major - Major loss of function.
        • Open
        is blocked by

        Bug - A problem which impairs or prevents the functions of the product. HIVE-20418 LLAP IO may not handle ORC files that have row index disabled correctly for queries with no columns selected

        • Major - Major loss of function.
        • Closed
        is related to

        Sub-task - The sub-task of the issue HIVE-16812 VectorizedOrcAcidRowBatchReader doesn't filter delete events

        • Critical - Crashes, loss of data, severe memory leak.
        • Closed

        Bug - A problem which impairs or prevents the functions of the product. HIVE-19890 ACID: Inherit bucket-id from original ROW_ID for delete deltas

        • Major - Major loss of function.
        • Closed
        is required by

        Improvement - An improvement or enhancement to an existing feature or task. HIVE-20655 Optimize arrayCopy in LlapRecordReader

        • Major - Major loss of function.
        • Open
        requires

        Improvement - An improvement or enhancement to an existing feature or task. HIVE-20640 Upgrade Hive to use ORC 1.5.3

        • Major - Major loss of function.
        • Closed

        Improvement - An improvement or enhancement to an existing feature or task. ORC-389 Add ability to not decode Acid metadata columns

        • Major - Major loss of function.
        • Closed

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            ekoifman Eugene Koifman Assign to me
            gopalv Gopal Vijayaraghavan
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment