Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-2183

DataImportHandler treatment of case for dynamic column mapping vs explicit mapping

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Won't Fix
    • 1.4.1
    • None
    • None
    • OOB install, jetty, Win XP

    Description

      There is a difference to how DIH treats the case of columns when using the DataImportHandler and using explicit mapping vs dynamic mapping. The exact test cases I used are described below:

      -----------------------------

      From http://wiki.apache.org/solr/DataImportHandler#A_shorter_data-config : "It is possible to totally avoid the field entries in entities if the names of the fields are same (case does not matter) as those in Solr schema"

      I confirmed that matching the schema.xml field case to the database table is needed for dynamic fields, and the wiki statement above is incorrect, or at the very least confusing, possibly a bug.

      My database is Oracle 10g and the column names have been created in all uppercase in the database.

      In Oracle:
      Table name: wide_table
      Column names: COLUMN_1 ... COLUMN_100 (yes, uppercase)

      Please see following scenarios and results I found:

      data-config.xml
      <entity name="item" query="select column_1,column_100 from wide_table">
      <field column="column_100" name="id"/>
      </entity>

      schema.xml
      <dynamicField name="column_*" type="string" indexed="true" stored="true" multiValued="true" />

      Result:
      Nothing Imported

      =========

      data-config.xml
      <entity name="item" query="select COLUMN_1,COLUMN_100 from wide_table">
      <field column="column_100" name="id"/>
      </entity>

      schema.xml
      <dynamicField name="column_*" type="string" indexed="true" stored="true" multiValued="true" />

      Result:
      Note query column names changed to uppercase.
      Nothing Imported

      =========

      data-config.xml
      <entity name="item" query="select column_1,column_100 from wide_table">
      <field column="COLUMN_100" name="id"/>
      </entity>

      schema.xml
      <dynamicField name="column_*" type="string" indexed="true" stored="true" multiValued="true" />

      Result:
      Note ONLY the field entry was changed to caps

      All records imported, with only COLUMN_100 id field.

      ============

      data-config.xml
      <entity name="item" query="select column_1,column_100 from wide_table">
      <field column="COLUMN_100" name="id"/>
      </entity>

      schema.xml
      <dynamicField name="COLUMN_*" type="string" indexed="true" stored="true" multiValued="true" />

      Result:
      Note BOTH the field entry was changed to caps in data-config.xml, and the dynamicField wildcard in schema.xml

      All records imported, with all fields specified. This is the behavior desired.

      =============

      Attachments

        Issue Links

          Activity

            People

              arafalov Alexandre Rafalovitch
              harrysmithwla K A
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: