Details
-
Bug
-
Status: Closed
-
Critical
-
Resolution: Fixed
-
1.2.1
Description
Class HCatOutputFormat in package org.apache.hive.hcatalog.mapreduce uses function setOutput to generate idHash using below statement:
In file org/apache/hive/hcatalog/mapreduce/HCatOutputFormat.java
line 116: idHash = String.valueOf(Math.random());
The output of idHash can be similar to values like this : 7.145347157239135E-4
And, in class FileOutputCommitterContainer in package org.apache.hive.hcatalog.mapreduce;
Uses below statement to compute final partition path:
In org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java
line 366: String finalLocn = jobLocation.replaceAll(Path.SEPARATOR + SCRATCH_DIR_NAME + "
d
.?
d+","");
line 367: partPath = new Path(finalLocn);
Regex used here is incorrect, since it will only remove integers after the SCRATCH_DIR_NAME, and hence will append 'E-4' (for the above example) in the final partition location.
Attachments
Attachments
Issue Links
- fixes
-
HIVE-11475 Bad rename of directory during commit, when using HCat dynamic-partitioning.
- Open
- links to