Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-1251

Python 3 Support

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: In Progress
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: sdk-py-core
    • Labels:
      None

      Description

      I have been trying to use google datalab with python3. As I see there are several packages that does not support python3 yet which google datalab depends on. This is one of them.

      https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6

        Attachments

          Issue Links

          1.
          Support Python native types in Beam typehints Sub-task Open Udi Meiri

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 0.5h
          2.
          Redesign test_split_at_fraction_exhaustive tests for Python 3 Sub-task In Progress Unassigned

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 4.5h
          3.
          Several VcfIO tests fail in Python 3 with TypeError: cannot use a string pattern on a bytes-like object Sub-task Open Simon  
          4.
          Support DoFns with Keyword-only arguments in Python 3. Sub-task Open yoshiki obata

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 13h 40m
          5.
          Using --save_main_session fails on Python 3 when main module has invocations of superclass method using 'super' . Sub-task Open Valentyn Tymofieiev

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 4h 10m
          6.
          Dill fails to pickle avro.RecordSchema classes on Python 3. Sub-task Open Valentyn Tymofieiev

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 7.5h
          7.
          Add Streaming wordcount test to Dataflow ValidatesContainer test suite Sub-task Open Unassigned  
          8.
          Rename ToStringCoder into ToBytesCoder Sub-task Open Francesco Perera  
          9.
          Add ValidatesRunner test suite for Flink on Python 3. Sub-task Open Valentyn Tymofieiev

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 2h 40m
          10.
          Enable Python3 tests for Spark Sub-task Open Kyle Weaver  
          11.
          Clean up Python 2 codepaths once Beam no longer supports Python 2. Sub-task Open Unassigned  
          12.
          Improve Avro IO integration test coverage on Python 3. Sub-task Open Valentyn Tymofieiev

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 5h 40m
          13.
          deadlock using save_main_session and logging Sub-task Open Valentyn Tymofieiev  
          14.
          Accept Py3 wheels in SDK harness container. Sub-task Open Unassigned  
          15.
          Migrate to "typing" module typing types in Beam typehints (on Py2 and Py3). Sub-task Open Udi Meiri  
          16.
          Use a Python3-compatible profiler in apache_beam.utils.profiler Sub-task Open Unassigned  
          17.
          Add Python 2 deprecation warnings starting from 2.17.0 release. Sub-task Open Unassigned  
          18.
          Generate Python SDK docs using Python 3 Sub-task Open Unassigned  
          19.
          Run pylint in Python 3 Sub-task Open Unassigned  
          20.
          Add a Python 3 test scenario for MongoDB IO Sub-task Open Yichi Zhang  
          21.
          Provide a way to better control minor+patch versions of Python 3.x interpreters used to run Beam tests locally and on Jenkins. Sub-task Triage Needed Unassigned  

            Activity

              People

              • Assignee:
                tvalentyn Valentyn Tymofieiev
                Reporter:
                eyad.alsibai@gmail.com Eyad Sibai
              • Votes:
                39 Vote for this issue
                Watchers:
                63 Start watching this issue

                Dates

                • Created:
                  Updated:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 355h 50m
                  355h 50m