Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-8569 Support Hadoop 3 on Beam
  3. BEAM-7937

Support Hadoop 3.x on Hadoop File System

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • P3
    • Resolution: Fixed
    • None
    • 2.27.0
    • io-java-hadoop-file-system
    • None

    Description

      I try to get a Beam pipeline to have input from Azure Data Lake Storage Gen 2. ADLS Gen 2 Support Hadoop 3.2+. I tried using Apache Beam 2.8.1 later on 2.14.0. I am getting below error:
      Error: Caused by: org.apache.hadoop.fs.UnsupportedFileSystemException: No FileSystem for scheme "wasbs"

      Attachments

        Activity

          People

            piotr-szuberski Piotr Szuberski
            reenusaluja Reenu Saluja
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: