Uploaded image for project: 'Apache Avro'
  1. Apache Avro
  2. AVRO-2644

Non-Deterministic avsc Directory Compilation

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • None
    • 1.9.2
    • None
    • None

    Description

      We're trying to use the `compile {src dir} {output dir}` command in
      `avro-tools` and finding that there are some non-deterministic
      behaviors between systems, depending on how the OS sorts files.

      Example:
      schemas/Component.avsc
        - defines Component record type in the namespace `com.test`

      schemas/Parent.avsc
        - defines a Parent record,  in the same `com.test` namespace, with a
      field of type `com.test.Component`

      With the command, `java -jar avro-tools-1.9.1.jar compile schemas/
      out-dir/`, some systems compile the directory in the order Component,
      Parent while others compile in the order Parent, Component. The latter
      fails as Component has not been defined when it is referenced by
      Parent.

      We have also tried using the IDL and importing the dependency types,
      and then converting them to avsc, and finally compiling the entire
      directory, but that fails as the generated avsc files embed/ duplicate
      the "Component" types each time it is used.

      OS:
      Linux 857aaf92e059 4.15.0-70-generic #79-Ubuntu SMP Tue Nov 12
      10:36:11 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

      Avro:
      version 1.9.1 

       

       

      Would a PR be accepted that enforces LANG=C semantics or would that have to be shipped as a breaking change?

       

      Coming from this thread in the mailing list: http://mail-archives.apache.org/mod_mbox/avro-user/201911.mbox/%3CCALGL%2BUDH03pCyKAQ5a%2B_fvwnUVougwwEXe8%2BHFAuR8Q%3D2cqYmw%40mail.gmail.com%3E

      Attachments

        Issue Links

          Activity

            People

              austince Austin Cawley-Edwards
              austince Austin Cawley-Edwards
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: