Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Incomplete
-
2.0.0, 2.2.0
-
None
-
Important
Description
The reason for this is that Phoenix JDBC driver does not support "INSERT", but "UPSERT".
Exception for the following program:
17/08/15 12:18:53 ERROR Executor: Exception in task 0.0 in stage 1.0 (TID 1)
org.apache.phoenix.exception.PhoenixParserException: ERROR 601 (42P00): Syntax error. Encountered "INSERT" at line 1, column 1.
at org.apache.phoenix.exception.PhoenixParserException.newException(PhoenixParserException.java:33)
public class HbaseJDBCSpark { private static final SparkSession sparkSession = SparkSession.builder() .config("spark.sql.warehouse.dir", "file:///temp") .config("spark.driver.memory", "5g") .master("local[*]").appName("Spark2JdbcDs").getOrCreate(); static final String JDBC_URL = "jdbc:phoenix:somehost:2181:/hbase-unsecure"; public static void main(String[] args) { final Properties connectionProperties = new Properties(); Dataset<Row> jdbcDF = sparkSession.read() .jdbc(JDBC_URL, "javatest", connectionProperties); jdbcDF.show(); String url = JDBC_URL; Properties p = new Properties(); p.put("driver", "org.apache.phoenix.jdbc.PhoenixDriver"); //p.put("batchsize", "100000"); jdbcDF.write().mode(SaveMode.Append).jdbc(url, "javatest", p); sparkSession.close(); } // Create variables }