Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-47904

Preserve case in Avro schema when using enableStableIdentifiersForUnionType

    XMLWordPrintableJSON

Details

    Description

      When enableStableIdentifiersForUnionType is enabled, all of the types are lowercased which creates a problem when field types are case-sensitive: 

      Schema.createEnum("myENUM", "", null, List[String]("E1", "e2").asJava),
Schema.createRecord("myRecord2", "", null, false, List[Schema.Field](new Schema.Field("F", Schema.create(Type.FLOAT))).asJava)

       would become

      struct<member_myenum: string, member_myrecord2: struct<f: float>>

      but instead should be 

      struct<member_myENUM: string, member_myRecord2: struct<F: float>>

       

      Attachments

        Activity

          People

            ivan.sadikov Ivan Sadikov
            ivan.sadikov Ivan Sadikov
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: