Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-7086

Enhance row-set scan framework to use external schema

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.15.0
    • 1.16.0
    • None

    Description

      Modify the row-set scan framework to work with an external (partial) schema; inserting "type conversion shims" to convert as needed. The reader provides an "input schema" the data types the reader is prepared to handle. An optional "output schema" describes the types of the value vectors to create. The type conversion "shims" give the reader the "setFoo" method it wants to use, while converting the data to the type needed for the vector. For example, the CSV reader might read only text fields, while the shim converts a column to an INT.

      This is just the framework layer, DRILL-7011 will combine this mechanism with the plan-side features to enable use of the feature in the new row-set based CSV reader.

      Attachments

        Issue Links

          Activity

            People

              Paul.Rogers Paul Rogers
              Paul.Rogers Paul Rogers
              Arina Ielchiieva Arina Ielchiieva
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: