Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-28779

CSV writer doesn't handle older Mac line endings

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Not A Problem
    • 2.3.0, 2.4.0
    • None
    • Spark Core
    • None

    Description

      The spark csv writer does not consider "\r"  as a newline in string type columns. As a result, the resulting csv are not quoted, and they get corrupted.

      All \n, \r\n and \r should be considered as newline to allow robust csv serialization.

      Attachments

        Activity

          People

            Unassigned Unassigned
            parisni nicolas paris
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: