Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-11985

Improve Solaris support in Hadoop

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 2.7.0
    • None
    • build, conf

    • Solaris x86, Solaris sparc

    • Solaris Native C x86 sparc

    Description

      At present the Hadoop native components aren't fully supported on Solaris primarily due to differences between Linux and Solaris. This top-level task will be used to group together both existing and new issues related to this work. A second goal is to improve Hadoop performance on Solaris wherever possible.

      Steve Loughran suggested a top-level JIRA was the best way to manage the work.

      Attachments

        Issue Links

          is blocked by

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-11790 leveldb usage should be disabled by default or smarter about platforms

          • Major - Major loss of function.
          • Open
          is related to

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-12603 TestSymlinkLocalFSFileContext#testSetTimesSymlinkToDir occasionally fail

          • Major - Major loss of function.
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. MAPREDUCE-6390 Improve Solaris support in Hadoop Map/Reduce

          • Major - Major loss of function.
          • Open
          relates to

          Sub-task - The sub-task of the issue HADOOP-12036 Consolidate all of the cmake extensions in one directory

          • Major - Major loss of function.
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. HDFS-8478 Improve Solaris support in HDFS

          • Major - Major loss of function.
          • Open

          New Feature - A new feature of the product, which has yet to be developed. YARN-3719 Improve Solaris support in YARN

          • Major - Major loss of function.
          • Open
          1.
          		 NativeIO.java flags and identifiers must be set correctly for each platform, not hardcoded to their Linux values Sub-task Resolved Martin Walsh  
          2.
          		 Set up a nightly Jenkins build on Solaris. Sub-task Open Unassigned  
          3.
          		 Fix FIONREAD #include on Solaris Sub-task Resolved Alan Burlison  
          4.
          		 Native compilation fails on Solaris due to use of getgrouplist function. Sub-task Resolved Malcolm Kavalsky

          0%
          Original Estimate - 168h
          Remaining Estimate - 168h
          5.
          		 Native compilation fails on Solaris due to use of syscall function. Sub-task Open Malcolm Kavalsky

          0%
          Original Estimate - 168h
          Remaining Estimate - 168h
          6.
          		 Native code needs to be built to match the 32/64 bitness of the JVM Sub-task Resolved Alan Burlison  
          7.
          		 Solaris does not support RLIMIT_MEMLOCK as in Linux Sub-task Resolved Alan Burlison

          0%
          Original Estimate - 24h
          Remaining Estimate - 24h
          8.
          		 JNI build should use default cmake FindJNI.cmake Sub-task Resolved Alan Burlison  
          9.
          		 CMake CMAKE_C_FLAGS are non-portable Sub-task Resolved Alan Burlison  
          10.
          		 Investigate providing SPARC hardware-accelerated CRC32 code Sub-task Open Unassigned  
          11.
          		 Investigate JNI for improving byte array comparison performance Sub-task Open Alan Burlison  
          12.
          		 Consolidate all of the cmake extensions in one directory Sub-task Resolved Alan Burlison  
          13.
          		 Migrate Hadoop Pipes native build to new CMake framework Sub-task Resolved Alan Burlison  
          14.
          		 Make hadoop-common-project Native code -Wall-clean Sub-task Resolved Alan Burlison  
          15.
          		 hadoop-common's JNIFlags.cmake is redundant and can be removed Sub-task Resolved Alan Burlison  
          16.
          		 Remove unused Linux-specific constants in NativeIO Sub-task Resolved Martin Walsh  
          17.
          		 Surefire needs to make sure the JVMs it fires up fit within the memory available Sub-task Patch Available Alan Burlison  
          18.
          		 Make DomainSocket.close() portable Sub-task Patch Available Alan Burlison  
          19.
          		 DomainSocket: Solaris does not support timeouts on AF_UNIX sockets Sub-task Patch Available Alan Burlison  
          20.
          		 Improve versioning and compatibility support in native library for downstream hadoop-common users. Sub-task Open Alan Burlison  
          21.
          		 Hadoop needs a SysInfo class for Solaris Sub-task Open Alan Burlison  
          22.
          		 ShellBasedIdMapping needs suport for Solaris Sub-task Resolved Alan Burlison  
          23.
          		 Sundry symlink problems on Solaris Sub-task Open Alan Burlison  
          24.
          		 Misaligned memory accesses in CRC32 native code causes coredumps Sub-task Open Alan Burlison  
          25.
          		 Misuse of sun.misc.Unsafe by org.apache.hadoop.io.FastByteComparisons causes misaligned memory access coredumps Sub-task Open Alan Burlison  
          26.
          		 Misuse of sun.misc.Unsafe by org.apache.hadoop.io.FastByteComparisons$LexicographicalComparerHolder$UnsafeComparer.compareTo causes misaligned memory access coredumps Sub-task Open Alan Burlison  
          27.
          		 Minor cleanups needed for CMake 3.X Sub-task Resolved Alan Burlison  
          28.
          		 Checks for SPARC architecture need to include 64-bit SPARC Sub-task Resolved Alan Burlison  
          29.
          		 lz4.c does not detect 64-bit mode properly Sub-task Resolved Alan Burlison  

          Activity

            People

              alanburlison Alan Burlison
              alanburlison Alan Burlison
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:

                Time Tracking

                  Estimated:
                  Original Estimate - 360h
                  360h
                  Remaining:
                  Remaining Estimate - 360h
                  360h
                  Logged:
                  Time Spent - Not Specified
                  Not Specified