Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-47361 Improve JDBC data sources
  3. SPARK-42627

Spark: Getting SQLException: Unsupported type -102 reading from Oracle

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 3.3.2
    • 4.0.0
    • SQL

    Description

       

      Exception in thread "main" org.apache.spark.SparkSQLException: Unrecognized SQL type -102
          at org.apache.spark.sql.errors.QueryExecutionErrors$.unrecognizedSqlTypeError(QueryExecutionErrors.scala:832)
          at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$.getCatalystType(JdbcUtils.scala:225)
          at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$.$anonfun$getSchema$1(JdbcUtils.scala:308)
          at scala.Option.getOrElse(Option.scala:189)
          at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$.getSchema(JdbcUtils.scala:308)
          at org.apache.spark.sql.execution.datasources.jdbc.JDBCRDD$.getQueryOutputSchema(JDBCRDD.scala:70)
          at org.apache.spark.sql.execution.datasources.jdbc.JDBCRDD$.resolveTable(JDBCRDD.scala:58)
          at org.apache.spark.sql.execution.datasources.jdbc.JDBCRelation$.getSchema(JDBCRelation.scala:242)
          at org.apache.spark.sql.execution.datasources.jdbc.JdbcRelationProvider.createRelation(JdbcRelationProvider.scala:37)
          at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:350)
          at org.apache.spark.sql.DataFrameReader.loadV1Source(DataFrameReader.scala:228)
          at org.apache.spark.sql.DataFrameReader.$anonfun$load$2(DataFrameReader.scala:210)
          at scala.Option.getOrElse(Option.scala:189)
          at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:210)
          at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:171)
       
      

      oracle driver

      <dependency>
          <groupId>com.oracle.database.jdbc</groupId>
          <artifactId>ojdbc8</artifactId>
          <version>21.9.0.0</version>
      </dependency>

       

      oracle sql:

       

      CREATE TABLE "ORDERS" 
         (    "ORDER_ID" NUMBER(9,0) NOT NULL ENABLE, 
          "ORDER_DATE" TIMESTAMP (3) WITH LOCAL TIME ZONE NOT NULL ENABLE, 
          "CUSTOMER_NAME" VARCHAR2(255) NOT NULL ENABLE, 
          "PRICE" NUMBER(10,5) NOT NULL ENABLE, 
          "PRODUCT_ID" NUMBER(9,0) NOT NULL ENABLE, 
          "ORDER_STATUS" NUMBER(1,0) NOT NULL ENABLE, 
           PRIMARY KEY ("ORDER_ID")
        USING INDEX PCTFREE 10 INITRANS 2 MAXTRANS 255 COMPUTE STATISTICS 
        STORAGE(INITIAL 65536 NEXT 1048576 MINEXTENTS 1 MAXEXTENTS 2147483645
        PCTINCREASE 0 FREELISTS 1 FREELIST GROUPS 1 BUFFER_POOL DEFAULT FLASH_CACHE DEFAULT CELL_FLASH_CACHE DEFAULT)
        TABLESPACE "LOGMINER_TBS"  ENABLE, 
           SUPPLEMENTAL LOG DATA (ALL) COLUMNS
         ) SEGMENT CREATION IMMEDIATE 
        PCTFREE 10 PCTUSED 40 INITRANS 1 MAXTRANS 255 NOCOMPRESS LOGGING
        STORAGE(INITIAL 65536 NEXT 1048576 MINEXTENTS 1 MAXEXTENTS 2147483645
        PCTINCREASE 0 FREELISTS 1 FREELIST GROUPS 1 BUFFER_POOL DEFAULT FLASH_CACHE DEFAULT CELL_FLASH_CACHE DEFAULT)
        TABLESPACE "LOGMINER_TBS"
       
      

      yao 

      Attachments

        Issue Links

          Activity

            People

              yao Kent Yao
              melin melin
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: