Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-23193

Insert into Spark Table statement cannot specify column names

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Minor
    • Resolution: Duplicate
    • 2.2.1
    • None
    • SQL
    • None

    Description

      The following code shows the insert statement cannot specify the column names. The error is

      insert into aaa (age, name)  values (12, 'nn')
      -----------------^^^

          at org.apache.spark.sql.catalyst.parser.ParseException.withCommand(ParseDriver.scala:217)
          at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parse(ParseDriver.scala:114)
          at org.apache.spark.sql.execution.SparkSqlParser.parse(SparkSqlParser.scala:48)
          at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parsePlan(ParseDriver.scala:68)
          at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:632)
          at com.---.iqi.spk.TestInsert.main(TestInsert.java:75)

       

      public class TestInsert {

          public static final SparkSession spark
                  = SparkSession.builder()
                          .config("spark.sql.warehouse.dir", "file:///temp")
                          .config("spark.driver.memory", "5g")
                          .enableHiveSupport()
                          .master("local[*]").appName("Spark2JdbcDs").getOrCreate();

           

          public static class Person implements Serializable {

              private String name;
              private int age;

              public String getName() {
                  return name;
      {{        }}}

              public void setName(String name) {
                  this.name = name;
      {{        }}}

              public int getAge() {
                  return age;
      {{        }}}

              public void setAge(int age) {
                  this.age = age;
      {{        }}}
      {{    }}}

          public static void main(String[] args) throws Exception {

              // Create an instance of a Bean class
              Person person = new Person();
              person.setName("Andy");
              person.setAge(32);

      // Encoders are created for Java beans
              Encoder<Person> personEncoder = Encoders.bean(Person.class);
              Dataset<Person> javaBeanDS = spark.createDataset(
                      Collections.singletonList(person),
                      personEncoder
              );
              javaBeanDS.show();
              javaBeanDS.createTempView("Abc");
              spark.sql("drop table aaa");
              spark.sql("create table aaa as select * from abc");
              spark.sql("insert into aaa  values (12, 'nn')");
              spark.sql("select * from aaa").show();
              spark.sql("insert into aaa (age, name)  values (12, 'nn')");
              spark.sql("select * from aaa").show();
              
              spark.close();
       
      {{    }}}
      }

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              zwu.net@gmail.com Paul Wu
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: