Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-21722

Enable timezone-aware timestamp type when creating Pandas DataFrame.

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Incomplete
    • 2.3.0
    • None
    • PySpark, SQL

    Description

      Make Pandas DataFrame with timezone-aware timestamp type when converting DataFrame to Pandas DataFrame by pyspark.sql.DataFrame.toPandas.
      The session local timezone is used for the timezone.

      Attachments

        Issue Links

          relates to

          Sub-task - The sub-task of the issue SPARK-21375 Add date and timestamp support to ArrowConverters for toPandas() collection

          • Major - Major loss of function.
          • Resolved
          links to

          Pull request #18933 [Github] Pull Request #18933 (ueshin)

          Activity

            People

              Unassigned Unassigned
              ueshin Takuya Ueshin
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: