Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-1377

getPartitionDescFromPath() in CombineHiveInputFormat should handle matching by path

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.6.0
    • 0.6.0
    • None
    • None

    Description

      The use case is:

      dir = hdfs://host:9000/user/warehouse/tableName/abc
      pathToPartitionInfo = {/user/warehouse/tableName : myPart}
      

      Then calling

       
      getPartitionDescFromPath(dir, pathToPartitionInfo)
      

      will throw an IOException because /user/warehouse/tableName is not a prefix of hdfs://host:9000/user/warehouse/tableName/abc. Currently, this is not an issue but will come up if CombineFileInputFormat is modified so what the scheme and authority are not stripped out when generating splits (see MAPREDUCE-1806).

      The proposed solution is add a case where matching is done by just the path component of the URI's.

      Attachments

        1. HIVE-1377.1.patch
          2 kB
          Paul Yang

        Activity

          People

            pauly Paul Yang
            pauly Paul Yang
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: