Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-20774

FSHDFSUtils#isSameHdfs doesn't handle S3 filesystems correctly.

    XMLWordPrintableJSON

Details

    Description

      FSHDFSUtils#isSameHdfs retrieves the Canonical Service Name from Hadoop to determine if source and destination are on the same filesystem. NativeS3FileSystem, S3FileSystem and presumably S3NativeFileSystem (com.amazon) always return null in getCanonicalServiceName() which incorrectly causes isSameHdfs to return false even when they could be the same. 

      Error encountered while trying to perform bulk load from S3 to HBase on S3 backed by the same bucket. This is causing bulk loads from S3 to copy all the data to the workers and back up to S3.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              aheyne Austin Heyne
              Votes:
              0 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

                Created:
                Updated: