Uploaded image for project: 'ORC'
  1. ORC
  2. ORC-1376

[C++] Support schema evolution

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Reopened
    • Major
    • Resolution: Unresolved
    • None
    • None
    • C++
    • None

    Description

      The Java ORC reader supports schema evolution by comparing readSchema with fileSchema [1] and creates a set of ConvertTreeReaders [2] to do the conversion. The conversion happens when reading columnar data into vector and evaluating expressions for predicate push down. The C++ reader does not support it yet.

      The type conversion on the Java side originates from Apache Hive. We should define what conversions are supposed to implement on the C++ side.

      [1] https://github.com/apache/orc/blob/main/java/core/src/java/org/apache/orc/impl/SchemaEvolution.java
      [2] https://github.com/apache/orc/blob/main/java/core/src/java/org/apache/orc/impl/ConvertTreeReaderFactory.java#L60:14

      Attachments

        Issue Links

          contains

          New Feature - A new feature of the product, which has yet to be developed. ORC-1389 [C++] Support schema evolution from string group to primitve/string group

          • Major - Major loss of function.
          • Open

          New Feature - A new feature of the product, which has yet to be developed. ORC-1390 [C++] Support schema evolution from string group to decimal/timestamp

          • Major - Major loss of function.
          • Open

          New Feature - A new feature of the product, which has yet to be developed. ORC-1391 [C++] Support schema evolution from timestamp to primtive/decimal/string group

          • Major - Major loss of function.
          • Open

          New Feature - A new feature of the product, which has yet to be developed. ORC-1392 [C++] Support schema evolution with date and binary

          • Major - Major loss of function.
          • Open

          New Feature - A new feature of the product, which has yet to be developed. ORC-1385 [C++] Support schema evolution from numeric to numeric

          • Major - Major loss of function.
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. ORC-1386 [C++] Support schema evolution from primtive to string group/decimal/timestamp

          • Major - Major loss of function.
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. ORC-1388 [C++] Support schema evolution from decimal to timestamp/string group

          • Major - Major loss of function.
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. ORC-1387 [C++] Support schema evolution from decimal to numeric/decimal

          • Major - Major loss of function.
          • Closed
          links to

          Web Link GitHub Pull Request #1427

          Web Link GitHub Pull Request #1441

          Activity

            People

              ffacs Yuanping Wu
              wgtmac Gang Wu
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated: