Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.4.0
    • 3.4.0
    • Project Infra
    • None

    Description

      Python addtion:

       

      $ diff  scala2.12-java11-r-ubuntu/Dockerfile scala2.12-java11-python3-r-ubuntu/Dockerfile
      
      31a32,33
      >     apt install -y python3 python3-pip && \
      >     pip3 install --upgrade pip setuptools && \
      33a36
      >     mkdir /opt/spark/python && \
      68a72,73
      >     mv python/pyspark /opt/spark/python/pyspark/; \
      >     mv python/lib /opt/spark/python/lib/; \ 

       

      R specific:

       

      $ diff  scala2.12-java11-python3-ubuntu/Dockerfile scala2.12-java11-python3-r-ubuntu/Dockerfile
      
      33a34
      >     apt install -y r-base r-base-dev && \
      72a74
      >     mv R /opt/spark/; \
      78a81
      > ENV R_HOME /usr/lib/R 

       

      We'd better

      • add a base template and apply R/python3 on demand.
      • Support specify spark version, java version, scala version

      Attachments

        Issue Links

          Activity

            People

              yikunkero Yikun Jiang
              yikunkero Yikun Jiang
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: