Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-9736

StorageBasedAuthProvider should batch namenode-calls where possible.

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Patch Available
    • Major
    • Resolution: Unresolved
    • 1.2.1
    • None
    • Metastore, Security

    Description

      Consider a table partitioned by 2 keys (dt, region). Say a dt partition could have 10000 associated regions. Consider that the user does:

      ALTER TABLE my_table DROP PARTITION (dt='20150101');
      

      As things stand now, StorageBasedAuthProvider will make individual DistributedFileSystem.listStatus() calls for each partition-directory, and authorize each one separately. It'd be faster to batch the calls, and examine multiple FileStatus objects at once.

      Attachments

        1. HIVE-9736.8.patch
          28 kB
          Sushanth Sowmyan
        2. HIVE-9736.7.patch
          28 kB
          Mithun Radhakrishnan
        3. HIVE-9736.6.patch
          28 kB
          Mithun Radhakrishnan
        4. HIVE-9736.5.patch
          35 kB
          Mithun Radhakrishnan
        5. HIVE-9736.4.patch
          28 kB
          Mithun Radhakrishnan
        6. HIVE-9736.3.patch
          28 kB
          Mithun Radhakrishnan
        7. HIVE-9736.2.patch
          28 kB
          Mithun Radhakrishnan
        8. HIVE-9736.1.patch
          27 kB
          Mithun Radhakrishnan

        Issue Links

          Activity

            People

              mithun Mithun Radhakrishnan
              mithun Mithun Radhakrishnan
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated: