Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-29233

Add regex expression checks for executorEnv in K8S mode

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 3.0.0
    • 3.0.0
    • Kubernetes, Spark Core
    • None

    Description

      In k8s mode, there are some naming regular expression requirements and checks for the key of pod environment variables, such as:

      • In version k8s v-1.10.5, the naming rules for pod env satisfy the requirements of regular expressions: [-. _a-zA-Z] [-. _a-zA-Z0-9].*
      • In version k8s v-1.6, the naming rules for pod env satisfy the requirement of regular expressions: [A-Za-z_] [A-Za-z0-9_]*

      Therefore, in spark on k8s mode, spark should add relevant regular checking rules when creating executor Env, and stricter validation rules should be used to filter out illegal key values that do not meet the requirements of the naming rules of Env for creating pod by k8s. Otherwise, it will lead to the problem that the pod can not be created normally and the tasks will be suspended.

       

      To solve the problem above, a regular validation to executorEnv is added and committed. 

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            merrily01 merrily01
            merrily01 merrily01
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment