Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
Description
The README for the Spark connector details the classpath requirements for the HBase server side, but does not talk about how to set up the Spark classpath for HBase.
The Cloudera docs https://docs.cloudera.com/cdp-private-cloud-base/7.1.9/accessing-hbase/topics/hbase-configure-spark-connector.html suggest using "hbase mapredcp" It is, however inconsistent, as "hbase mapredcp" includes the unshaded hadoop libraries, while the example command line omits the hadoop libraries (and seem to depend on the on the existing Hadoop JARs on the Spark classpath).
Figure this out, and update the documentation.
UPDATE:
SPARK-33618 has reverted to using unshaded Hadoop.
We only have to document the `hbase mapredcp` option.
Attachments
Issue Links
- depends upon
-
HBASE-28213 Evaluate using hbase-shaded-client-byo-hadoop for Spark connector
- Resolved