Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-6385

ISO 8601 timestamp parsing does not support arbitrary precision second fractions

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Duplicate
    • 1.2.1
    • None
    • SQL
    • None

    Description

      The ISO 8601 timestamp parsing implemented as a resolution to SPARK-4149 does not support arbitrary precision fractions of seconds, only millisecond precision. Parsing 2015-02-02T00:00:07.900GMT-00:00 will succeed, while 2015-02-02T00:00:07.9000GMT-00:00 will fail.

      The issue is caused by the fixed precision of the parsed format in DataTypeConversions.scala#L66. I'm willing to implement a fix, but pointers on the direction would be appreciated.

      Attachments

        Issue Links

          duplicates

          Bug - A problem which impairs or prevents the functions of the product. SPARK-9794 ISO DateTime parser is too strict

          • Major - Major loss of function.
          • Resolved
          relates to

          Bug - A problem which impairs or prevents the functions of the product. SPARK-29904 Parse timestamps in microsecond precision by JSON/CSV datasources

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              Unassigned Unassigned
              bruun Nick Bruun
              Votes:
              1 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: