Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-6652

[Python] to_pandas conversion removes timezone from type

    XMLWordPrintableJSON

Details

    Description

      Calling to_pandas on a pyarrow.Array with a timezone aware timestamp type, removes the timezone in the resulting pandas.Series.

      >>> import pyarrow as pa
      >>> a = pa.array([1], type=pa.timestamp('us', tz='America/Los_Angeles'))
      >>> a.to_pandas()
      0   1970-01-01 00:00:00.000001
      dtype: datetime64[ns]
      

      Previous behavior from 0.14.1 of converting a pyarrow.Column to_pandas retained the timezone.

      In [4]: import pyarrow as pa 
         ...: a = pa.array([1], type=pa.timestamp('us', tz='America/Los_Angeles'))  
         ...: c = pa.Column.from_array('ts', a) 
      
      In [5]: c.to_pandas()                                                                                                        
      Out[5]: 
      0   1969-12-31 16:00:00.000001-08:00
      Name: ts, dtype: datetime64[ns, America/Los_Angeles]
      

      Attachments

        Issue Links

          Activity

            People

              jorisvandenbossche Joris Van den Bossche
              bryanc Bryan Cutler
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 2h 40m
                  2h 40m