Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-20882 Support Hive replication to a target cluster with hive.strict.managed.tables enabled.
  3. HIVE-20967

Handle alter events when replicate to cluster with hive.strict.managed.tables enabled.

Log workAgile BoardRank to TopRank to BottomBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersConvert to IssueMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 4.0.0
    • None
    • repl

    Description

      Some of the events from Hive2 may cause conflicts in Hive3 (hive.strict.managed.tables=true) when applied. So, need to handle them properly.
      1. Alter table to convert non-acid to acid.

      • Do not allow this conversion on source of replication if strict.managed is false.

      2. Alter table or partition that changes the location.

      • For managed tables at source, the table location shouldn't be changed for the given non-partitioned table and partition location shouldn't be changed for partitioned table as alter event doesn't capture the new files list. So, it may cause data inconsistsency. So, if database is enabled for replication at source, then alter location on managed tables should be blocked.
      • For external partitioned tables, if location is changed at source, the the location should be changed for the table and any partitions which reside within the table location, but not for the partitions which are not within the table location. (may be we just need the test).

      Attachments

        1. HIVE-20967.01.patch
          15 kB
          Ashutosh Bapat
        2. HIVE-20967.03.patch
          17 kB
          Ashutosh Bapat
        3. HIVE-20967.03.patch
          17 kB
          Ashutosh Bapat
        4. HIVE-20967.04.patch
          18 kB
          Ashutosh Bapat
        5. HIVE-20967.05.patch
          18 kB
          Ashutosh Bapat

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            ashutosh.bapat Ashutosh Bapat Assign to me
            maheshk114 mahesh kumar behera
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

              Estimated:
              Original Estimate - Not Specified
              Not Specified
              Remaining:
              Remaining Estimate - 0h
              0h
              Logged:
              Time Spent - 2h
              2h

              Slack

                Issue deployment