Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-22052

pom cleaning; filter out jersey-core in hadoop2 to match hadoop3 and remove redunant version specifications

    XMLWordPrintableJSON

    Details

    • Type: Task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.0.0, 2.2.0, 2.0.5, 2.3.0, 2.1.4
    • Component/s: None
    • Labels:
      None
    • Release Note:
      Hide
      <!-- markdown -->
      Fixed awkward dependency issue that prevented site building.

      #### note specific to HBase 2.1.4
      HBase 2.1.4 shipped with an early version of this fix that incorrectly altered the libraries included in our binary assembly for using Apache Hadoop 2.7 (the current build default Hadoop version for 2.1.z). For folks running out of the box against a Hadoop 2.7 cluster (or folks who skip the installation step of [replacing the bundled Hadoop libraries](http://hbase.apache.org/book.html#hadoop)) this will result in a failure at Region Server startup due to a missing class definition. e.g.:
      ```
      2019-03-27 09:02:05,779 ERROR [main] regionserver.HRegionServer: Failed construction RegionServer
      java.lang.NoClassDefFoundError: org/apache/htrace/SamplerBuilder
      at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:644)
      at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:628)
      at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:149)
      at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2667)
      at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:93)
      at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2701)
      at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2683)
      at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:372)
      at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:171)
      at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:356)
      at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)
      at org.apache.hadoop.hbase.util.CommonFSUtils.getRootDir(CommonFSUtils.java:362)
      at org.apache.hadoop.hbase.util.CommonFSUtils.isValidWALRootDir(CommonFSUtils.java:411)
      at org.apache.hadoop.hbase.util.CommonFSUtils.getWALRootDir(CommonFSUtils.java:387)
      at org.apache.hadoop.hbase.regionserver.HRegionServer.initializeFileSystem(HRegionServer.java:704)
      at org.apache.hadoop.hbase.regionserver.HRegionServer.<init>(HRegionServer.java:613)
      at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
      at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
      at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
      at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
      at org.apache.hadoop.hbase.regionserver.HRegionServer.constructRegionServer(HRegionServer.java:3029)
      at org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.start(HRegionServerCommandLine.java:63)
      at org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.run(HRegionServerCommandLine.java:87)
      at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
      at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:149)
      at org.apache.hadoop.hbase.regionserver.HRegionServer.main(HRegionServer.java:3047)
      Caused by: java.lang.ClassNotFoundException: org.apache.htrace.SamplerBuilder
      at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
      at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
      ... 26 more

      ```

      Workaround via any _one_ of the following:
      * If you are running against a Hadoop cluster that is 2.8+, ensure you replace the Hadoop libaries in the default binary assembly with those for your version.
      * If you are running against a Hadoop cluster that is 2.8+, build the binary assembly from the source release while specifying your Hadoop version.
      * If you are running against a Hadoop cluster that is a supported 2.7 release, ensure the `hadoop` executable is in the `PATH` seen at Region Server startup and that you are not using the `HBASE_DISABLE_HADOOP_CLASSPATH_LOOKUP` bypass.
      * For any supported Hadoop version, manually make the Apache HTrace artifact `htrace-core-3.1.0-incubating.jar` available to all Region Servers via the HBASE_CLASSPATH environment variable.
      * For any supported Hadoop version, manually make the Apache HTrace artifact `htrace-core-3.1.0-incubating.jar` available to all Region Servers by copying it into the directory `${HBASE_HOME}/lib/client-facing-thirdparty/`.
      Show
      <!-- markdown --> Fixed awkward dependency issue that prevented site building. #### note specific to HBase 2.1.4 HBase 2.1.4 shipped with an early version of this fix that incorrectly altered the libraries included in our binary assembly for using Apache Hadoop 2.7 (the current build default Hadoop version for 2.1.z). For folks running out of the box against a Hadoop 2.7 cluster (or folks who skip the installation step of [replacing the bundled Hadoop libraries]( http://hbase.apache.org/book.html#hadoop) ) this will result in a failure at Region Server startup due to a missing class definition. e.g.: ``` 2019-03-27 09:02:05,779 ERROR [main] regionserver.HRegionServer: Failed construction RegionServer java.lang.NoClassDefFoundError: org/apache/htrace/SamplerBuilder at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:644) at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:628) at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:149) at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2667) at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:93) at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2701) at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2683) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:372) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:171) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:356) at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295) at org.apache.hadoop.hbase.util.CommonFSUtils.getRootDir(CommonFSUtils.java:362) at org.apache.hadoop.hbase.util.CommonFSUtils.isValidWALRootDir(CommonFSUtils.java:411) at org.apache.hadoop.hbase.util.CommonFSUtils.getWALRootDir(CommonFSUtils.java:387) at org.apache.hadoop.hbase.regionserver.HRegionServer.initializeFileSystem(HRegionServer.java:704) at org.apache.hadoop.hbase.regionserver.HRegionServer.<init>(HRegionServer.java:613) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.regionserver.HRegionServer.constructRegionServer(HRegionServer.java:3029) at org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.start(HRegionServerCommandLine.java:63) at org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.run(HRegionServerCommandLine.java:87) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:149) at org.apache.hadoop.hbase.regionserver.HRegionServer.main(HRegionServer.java:3047) Caused by: java.lang.ClassNotFoundException: org.apache.htrace.SamplerBuilder at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 26 more ``` Workaround via any _one_ of the following: * If you are running against a Hadoop cluster that is 2.8+, ensure you replace the Hadoop libaries in the default binary assembly with those for your version. * If you are running against a Hadoop cluster that is 2.8+, build the binary assembly from the source release while specifying your Hadoop version. * If you are running against a Hadoop cluster that is a supported 2.7 release, ensure the `hadoop` executable is in the `PATH` seen at Region Server startup and that you are not using the `HBASE_DISABLE_HADOOP_CLASSPATH_LOOKUP` bypass. * For any supported Hadoop version, manually make the Apache HTrace artifact `htrace-core-3.1.0-incubating.jar` available to all Region Servers via the HBASE_CLASSPATH environment variable. * For any supported Hadoop version, manually make the Apache HTrace artifact `htrace-core-3.1.0-incubating.jar` available to all Region Servers by copying it into the directory `${HBASE_HOME}/lib/client-facing-thirdparty/`.

      Description

      Working on HBASE-22029, where we fail compile of hbase-it module four hours into an RC build, it looks like transitive include of jersey-core is the culprit. hadoop3 profile does a bunch of jersey-core exclusions. This issue is about having hadoop2 profile and hadoop3 profiles match around jersey-core treatment. Some miscellaneous cleanups are also done.

        Attachments

        1. HBASE-22052.2.1.003.addendum1.txt
          0.0 kB
          Michael Stack
        2. HBASE-22052.branch-2.0.001.patch
          16 kB
          Michael Stack
        3. HBASE-22052.branch-2.0.002.patch
          15 kB
          Michael Stack
        4. HBASE-22052.branch-2.0.002.patch
          15 kB
          Michael Stack
        5. HBASE-22052.branch-2.0.003.patch
          17 kB
          Michael Stack
        6. HBASE-22052.branch-2.0.004.patch
          39 kB
          Michael Stack
        7. HBASE-22052.branch-2.1.001.patch
          19 kB
          Michael Stack
        8. HBASE-22052.branch-2.1.002.patch
          19 kB
          Michael Stack
        9. HBASE-22052.branch-2.1.003.patch
          20 kB
          Michael Stack
        10. HBASE-22052.branch-2.2.001.patch
          18 kB
          Michael Stack
        11. HBASE-22052.branch-2.2.002.patch
          19 kB
          Michael Stack
        12. HBASE-22052-branch-2.1.003.addendum2.txt
          2 kB
          Michael Stack
        13. nothing_change.patch
          2 kB
          Michael Stack
        14. nothing_change.patch
          2 kB
          Michael Stack

          Issue Links

            Activity

              People

              • Assignee:
                stack Michael Stack
                Reporter:
                stack Michael Stack
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: