Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-16911

Distcp with snapshot diff to support Ozone filesystem.

    XMLWordPrintableJSON

Details

    • Reviewed

    Description

      Currently in DistcpSync i.e the step which applies the diff b/w 2 provided snapshots as arguments to the distcp job with -diff option, only DistributedFilesystem and WebHDFS filesytems are supported.

       

      // currently we require both the source and the target file system are
      // DistributedFileSystem or (S)WebHdfsFileSystem.
      if (!(srcFs instanceof DistributedFileSystem
              || srcFs instanceof WebHdfsFileSystem)) {
        throw new IllegalArgumentException("Unsupported source file system: "
            + srcFs.getScheme() + "://. " +
            "Supported file systems: hdfs://, webhdfs:// and swebhdfs://.");
      }
      if (!(tgtFs instanceof DistributedFileSystem
          || tgtFs instanceof WebHdfsFileSystem)) {
        throw new IllegalArgumentException("Unsupported target file system: "
            + tgtFs.getScheme() + "://. " +
            "Supported file systems: hdfs://, webhdfs:// and swebhdfs://.");
      }

      As Ozone now supports snapshot feature after HDDS-6517, add support to use distcp with ozone snapshots.

       

      Attachments

        Issue Links

          Activity

            People

              sadanand_shenoy Sadanand Shenoy
              sadanand_shenoy Sadanand Shenoy
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: