Details
-
Sub-task
-
Status: Closed
-
Major
-
Resolution: Fixed
-
None
-
None
Description
https://github.com/apache/hive/blob/master/llap-common/src/java/org/apache/hadoop/hive/llap/tez/Converters.java#L182 I see this copy take a few milliseconds sometimes. Delay here adds up for all tasks of a single vertex in LlapTaskCommunicator as it processes tasks one by one. User payload never changes in this codepath. Copy is made because of limitations of Protobuf library. Protobuf adds a UnsafeByteOperations class that avoid copying of ByteBuffers in 3.1 version. This can be resolved when Protobuf is upgraded.
Attachments
Issue Links
- links to