Uploaded image for project: 'REEF (Retired)'
  1. REEF (Retired)
  2. REEF-751

Add a standard way to provide cluster-local configuration

    XMLWordPrintableJSON

Details

    Description

      When running a REEF application on a specific cluster, we sometimes have to make cluster-specific configuration choices beyond what's available via e.g. the YARN Configuration. One example are clusters which only allow us to open TCP ports in a specific range.

      Right now, we require the application to add such configuration to the Driver, e.g. via the DriverConfigurationProviders mechanism. This is awkward, because it limits the portability of applications: In order for an app to run such a cluster, one would have to change the app's code. This is undesirable.

      Instead, we should have a standard mechanism by which one can provide additional runtime-level configuration to be picked up by the REEF client. Let's use this JIRA to discuss potential solutions to this issue.

      Attachments

        Activity

          People

            Unassigned Unassigned
            markus.weimer Markus Weimer
            Votes:
            1 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: