Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-28214

Document Spark classpath requirements for the Spark connector

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • spark
    • None

    Description

      The README for the Spark connector details the classpath requirements for the HBase server side, but does not talk about how to set up the Spark classpath for HBase.

      The Cloudera docs https://docs.cloudera.com/cdp-private-cloud-base/7.1.9/accessing-hbase/topics/hbase-configure-spark-connector.html suggest using "hbase mapredcp" It is, however inconsistent, as "hbase mapredcp" includes the unshaded hadoop libraries, while the example command line omits the hadoop libraries (and seem to depend on the on the existing Hadoop JARs on the Spark classpath).

      Figure this out, and update the documentation.

      UPDATE:
      SPARK-33618 has reverted to using unshaded Hadoop.
      We only have to document the `hbase mapredcp` option.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              stoty Istvan Toth
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated: