Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-38082

Update minimum numpy version to 1.15

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.3.0
    • 3.3.0
    • ML, MLlib, PySpark
    • None

    Description

      Currently, we use set numpy version in extras_require to be >=1.7.

      However, 1.7 has been released almost 9 years ago and since then some methods that we use have been deprecated in favor of new additions and anew API (numpy.typing, that is of some interest to us, has been added.

      We should update minimum version requirement to one of the following

      • >=1.9.0 ‒ this is minimum reasonable bound, that will allow us to replace deprecated tostring calls with tobytes.
      • >=1.15.0 (released 2018-07-23) ‒ this is reasonable bound to match our minimum supported pandas version.
      • >=1.20.0 (released 2021-01-30) ‒ to fully utilize numpy typing.

      The last one might be somewhat controversial, but 1.15 shouldn't require much discussion.

      Attachments

        Activity

          People

            zero323 Maciej Szymkiewicz
            zero323 Maciej Szymkiewicz
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: