Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-14400

Handle concurrent insert with dynamic partition

Log workAgile BoardRank to TopRank to BottomBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 2.2.0
    • 2.1.1, 2.2.0
    • Hive
    • None

    Description

      With multiple users concurrently issuing insert statements on the same partition has a side effect that some queries may not see a partition at the time when they're issued, but will realize the partition is actually there when it is trying to add such partition to the metastore and thus get AlreadyExistsException, because some earlier query just created it (race condition).

      For example, imagine such a table is created:

      create table T (name char(50)) partitioned by (ds string);
      

      and the following two queries are launched at the same time, from different sessions:

      insert into table T partition (ds) values ('Bob', 'today'); -- creates the partition 'today'
      insert into table T partition (ds) values ('Joe', 'today'); -- will fail with AlreadyExistsException
      

      Attachments

        1. HIVE-14400.1.patch
          2 kB
          Wei Zheng
        2. HIVE-14400.2.patch
          3 kB
          Wei Zheng

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            wzheng Wei Zheng Assign to me
            wzheng Wei Zheng
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment