Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-12549

Configure Hadoop version in Spark portable runner.

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Open
    • P3
    • Resolution: Unresolved
    • None
    • None
    • runner-spark

    Description

      It may be as simple as changing the Spark runner configuration to "hadoopVersionXYZ" (e.g. hadoopVersion321) but I haven't gotten a chance to verify. https://github.com/apache/beam/blob/cae01f09c72ee8dce1df5fca683c89534720ea9a/runners/spark/job-server/spark_job_server.gradle#L55

      Attachments

        Issue Links

          blocks

          Wish - General wishlist item. BEAM-8970 Spark portable runner supports Yarn

          • P3 - Non-urgent bugs, features, and improvements
          • Resolved

          Activity

            People

              Unassigned Unassigned
              ibzib Kyle Weaver
              Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated: