Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-19369

Locks: Add new lock implementations for always zero-wait readers

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • None
    • Transactions
    • None

    Description

      Hive Locking with Micro-managed and full-ACID tables needs a better locking implementation which allows for no-wait readers always.

      EXCL_DROP
      EXCL_WRITE
      SHARED_WRITE
      SHARED_READ

      Short write-up

      EXCL_DROP is a "drop partition" or "drop table" and waits for all others to exit
      EXCL_WRITE excludes all writes and will wait for all existing SHARED_WRITE to exit.
      SHARED_WRITE allows all SHARED_WRITES to go through, but will wait for an EXCL_WRITE & EXCL_DROP (waiting so that you can do drop + insert in different threads).

      SHARED_READ does not wait for any lock - it fails fast for a pending EXCL_DROP, because even if there is an EXCL_WRITE or SHARED_WRITE pending, there's no semantic reason to wait for them to succeed before going ahead with a SHARED_WRITE.

      a select * => SHARED_READ
      an insert into => SHARED_WRITE
      an insert overwrite or MERGE => EXCL_WRITE
      a drop table => EXCL_DROP

      TODO:

      The fate of the compactor needs to be added to this before it is a complete description.

      Attachments

        1. HIVE-19369.1.patch
          142 kB
          Denys Kuzmenko
        2. HIVE-19369.2.patch
          142 kB
          Denys Kuzmenko
        3. HIVE-19369.3.patch
          163 kB
          Denys Kuzmenko
        4. HIVE-19369.4.patch
          163 kB
          Denys Kuzmenko
        5. HIVE-19369.5.patch
          163 kB
          Denys Kuzmenko
        6. HIVE-19369.6.patch
          160 kB
          Denys Kuzmenko
        7. HIVE-19369.7.patch
          174 kB
          Denys Kuzmenko
        8. HIVE-19369.8.patch
          177 kB
          Denys Kuzmenko
        9. HIVE-19369.9.patch
          177 kB
          Denys Kuzmenko
        10. HIVE-19369.9.patch
          177 kB
          Denys Kuzmenko

        Issue Links

          Activity

            People

              dkuzmenko Denys Kuzmenko
              gopalv Gopal Vijayaraghavan
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: