Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-25398

Converted external tables should be able to configure purge behaviour

    XMLWordPrintableJSON

Details

    Description

      Creating non-ACID MANAGED tables is not allowed on Hive, which is instead converting these tables to External: https://issues.apache.org/jira/browse/HIVE-22158
      During table translation both TRANSLATED_TO_EXTERNAL and 'external.table.purge' are set to True. However, there could be the case that the second parameter is already set in the table properties by the User. This is ticket is adding an extra check to maintain that property if set.

      PS: A cleaner solution would be to create these Tables as External directly but there could be the case the User is taking advantage of the translation and is expecting the data NOT to be purged!

      Example:

      -- Non-ACID table will be translated to EXTERNAL
      create table c(c int) LOCATION 'etp_1' TBLPROPERTIES('transactional'='false','external.table.purge'='false');
      insert into c values(1);
      
      -- Maintain the purge=false property set above
      desc formatted c;
      select count(*) from c;
      drop table c;
      
      -- Create table in same location, data should still be there
      create table c(c int) LOCATION 'etp_1' TBLPROPERTIES('transactional'='false','external.table.purge'='false');
      desc formatted c;
      select count(*) from c;
      

      Attachments

        Issue Links

          Activity

            People

              pgaref Panagiotis Garefalakis
              pgaref Panagiotis Garefalakis
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 40m
                  40m