Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-10191

ORC: Cleanup writer per-row synchronization

Log workAgile BoardRank to TopRank to BottomBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Remove per-row synchronization from ORC WriterImpl

    Description

      ORC writers were originally meant to be thread-safe, but in the present day implementation each ORC writer is entirely share-nothing which converts most of the synchronized blocks in ORC as entirely uncontested locks.

      These uncontested locks prevent the JVM from inlining/optimizing these methods, while adding no extra thread-safety to the ORC writers.

      Attachments

        1. HIVE-10191.1.patch
          8 kB
          Gopal Vijayaraghavan
        2. HIVE-10191.2.patch
          9 kB
          Gopal Vijayaraghavan
        3. HIVE-10191.3.patch
          8 kB
          Gopal Vijayaraghavan
        4. HIVE-10191.4.patch
          8 kB
          Gopal Vijayaraghavan
        5. HIVE-10191.5.patch
          8 kB
          Gopal Vijayaraghavan

        Issue Links

        incorporates

        Sub-task - The sub-task of the issue HIVE-10174 LLAP: ORC MemoryManager is singleton synchronized

        • Major - Major loss of function.
        • Resolved
        relates to

        Bug - A problem which impairs or prevents the functions of the product. HIVE-7741 Don't synchronize WriterImpl.addRow() when dynamic.partition is enabled

        • Major - Major loss of function.
        • Resolved

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            gopalv Gopal Vijayaraghavan Assign to me
            gopalv Gopal Vijayaraghavan
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment