Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
0.14.0, 0.15.0, 0.14.1, 0.14.2, 0.16.0, 0.17.0
-
None
-
None
-
Important
Description
libthrift:0.13.0 is affected with CVE-2020-13949
Currently I am using Spark 3.3.0 and Hive 2.3.9 with Hadoop 3.3.4.
When we do an upgrade to use libthrift:0.14.0 and above jar, below exception is thrown while starting the Spark Thriftserver.
org.apache.hive.service.ServiceException: Failed to Start HiveServer2 at org.apache.hive.service.CompositeService.start(CompositeService.java:79) at org.apache.hive.service.server.HiveServer2.start(HiveServer2.java:104) at org.apache.spark.sql.hive.thriftserver.HiveThriftServer2.start(HiveThriftServer2.scala:154) at org.apache.spark.sql.hive.thriftserver.HiveThriftServer2$.startWithContext(HiveThriftServer2.scala:64) at org.apache.spark.sql.hive.thriftserver.HiveThriftServer2$.main(HiveThriftServer2.scala:104) at org.apache.spark.sql.hive.thriftserver.HiveThriftServer2.main(HiveThriftServer2.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52) at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:958) at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180) at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203) at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90) at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1046) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1055) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) Caused by: java.lang.NoSuchMethodError: org.apache.thrift.server.TThreadPoolServer$Args.requestTimeout(I)Lorg/apache/thrift/server/TThreadPoolServer$Args; at org.apache.hive.service.cli.thrift.ThriftBinaryCLIService.initializeServer(ThriftBinaryCLIService.java:101) at org.apache.hive.service.cli.thrift.ThriftCLIService.start(ThriftCLIService.java:176) at org.apache.hive.service.CompositeService.start(CompositeService.java:69) at org.apache.hive.service.server.HiveServer2.start(HiveServer2.java:104) at org.apache.spark.sql.hive.thriftserver.HiveThriftServer2.start(HiveThriftServer2.scala:154) at org.apache.spark.sql.hive.thriftserver.HiveThriftServer2$.startWithContext(HiveThriftServer2.scala:64) at org.apache.spark.sql.hive.thriftserver.HiveThriftServer2$.main(HiveThriftServer2.scala:104) at org.apache.spark.sql.hive.thriftserver.HiveThriftServer2.main(HiveThriftServer2.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52) at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:958) at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180) at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203) at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90) at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1046) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1055) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
After detailed investigation, found out that since 0.14.0 version, in class org.apache.thrift.server.TThreadPoolServer, requestTimeout property has been removed.
Even in latest hive release (3.1.3), it is still referencing to the requestTimeout attribute has been removed.
Wanted to know as part of what change and why this was removed? Is there any alternative to use?
