Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-14988

Support INSERT OVERWRITE into a partition on transactional tables

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 1.0.0
    • 3.0.0
    • Transactions
    • None

    Description

      Insert overwrite operation on transactional table will currently raise an error.
      This can/should be supported

      This is implemented by using the concept of base_N/ directory in Acid.
      INSERT OVERWRITE starts a new transaction N and creates a new base_N directory where it writes the data.
      This has the effect that on read all files with transaction IDs < N are ignored.
      The advantage is that IOW doesn't need to be a mutexed operation. All readers that stared before this IOW can proceed at the
      snapshot (Acid tables support at Snapshot Isolation) they locked in.

      If mutexed semantics are desired we could add an option where IOW will request an X lock but the base_N/ based implementation can support both.

      Attachments

        1. HIVE-14988.01.patch
          6 kB
          Eugene Koifman
        2. HIVE-14988.02.patch
          7 kB
          Eugene Koifman
        3. HIVE-14988.03.patch
          16 kB
          Wei Zheng
        4. HIVE-14988.04.patch
          18 kB
          Wei Zheng
        5. HIVE-14988.05.patch
          26 kB
          Wei Zheng
        6. HIVE-14988.06.patch
          45 kB
          Wei Zheng

        Issue Links

          Activity

            People

              wzheng Wei Zheng
              ekoifman Eugene Koifman
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: