Details
-
Improvement
-
Status: To Do
-
Major
-
Resolution: Unresolved
-
None
-
None
Description
As as owner of a Turing, Volta, Pascal, or Jetson TX1 device, I would like to run inference using TensorRT with FP16 DType tensors to make use of dedicated hardware optimizations present.
Reference: https://devblogs.nvidia.com/tensor-core-ai-performance-milestones/
https://devblogs.nvidia.com/programming-tensor-cores-cuda-9/