Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-14535 add insert-only ACID tables to Hive
  3. HIVE-17970

MM LOAD DATA with OVERWRITE doesn't use base_n directory concept

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 3.0.0
    • 3.0.0
    • Transactions

    Description

      Judging by

      Hive.loadTable(Path loadPath, String tableName, LoadFileType loadFileType, boolean isSrcLocal,
            boolean isSkewedStoreAsSubdir, boolean isAcid, boolean hasFollowingStatsTask,
            Long txnId, int stmtId, boolean isMmTable)
      

      LOAD DATA with OVERWRITE will delete all existing data then write new data into the table. This logic makes sense for non-acid tables but for Acid/MM it should work like INSERT OVERWRITE statement and write new data to base_n/. This way the lock manager can be used to either get an X lock for IOW and thus block all readers or let it run with SemiShared and let readers continue and make the system more concurrent.

      Attachments

        1. HIVE-17970.patch
          23 kB
          Sergey Shelukhin
        2. HIVE-17970.01.patch
          24 kB
          Sergey Shelukhin
        3. HIVE-17970.02.patch
          24 kB
          Sergey Shelukhin
        4. HIVE-17970.03.patch
          25 kB
          Sergey Shelukhin
        5. HIVE-17970.04.patch
          25 kB
          Sergey Shelukhin
        6. HIVE-17970.05.patch
          25 kB
          Sergey Shelukhin

        Issue Links

          Activity

            People

              sershe Sergey Shelukhin
              ekoifman Eugene Koifman
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: