Details
-
Improvement
-
Status: Resolved
-
Minor
-
Resolution: Not A Problem
-
2.3.0, 2.4.4
-
None
Description
I use native BLAS to improvement ML/MLLIB performance on Yarn.
The file spark-env.sh which is modified by SPARK-21305 said that I should set OPENBLAS_NUM_THREADS=1 to disable multi-threading of OpenBLAS, but it does not take effect.
I modify spark.conf to set spark.executorEnv.OPENBLAS_NUM_THREADS=1,and the performance improve.
I think MKL_NUM_THREADS is the same.
Attachments
Attachments
Issue Links
- links to