Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-33358

Flink SQL Client fails to start in Flink on YARN

    XMLWordPrintableJSON

Details

    Description

      Flink SQL Client fails to start in Flink on YARN with below error

      flink-yarn-session -tm 2048 -s 2 -d
      
      /usr/lib/flink/bin/sql-client.sh 
      
      Exception in thread "main" org.apache.flink.table.client.SqlClientException: Could not read from command line.
      	at org.apache.flink.table.client.cli.CliClient.getAndExecuteStatements(CliClient.java:221)
      	at org.apache.flink.table.client.cli.CliClient.executeInteractive(CliClient.java:179)
      	at org.apache.flink.table.client.cli.CliClient.executeInInteractiveMode(CliClient.java:121)
      	at org.apache.flink.table.client.cli.CliClient.executeInInteractiveMode(CliClient.java:114)
      	at org.apache.flink.table.client.SqlClient.openCli(SqlClient.java:169)
      	at org.apache.flink.table.client.SqlClient.start(SqlClient.java:118)
      	at org.apache.flink.table.client.SqlClient.startClient(SqlClient.java:228)
      	at org.apache.flink.table.client.SqlClient.main(SqlClient.java:179)
      Caused by: java.lang.NoClassDefFoundError: Could not initialize class org.apache.flink.table.client.config.SqlClientOptions
      	at org.apache.flink.table.client.cli.parser.SqlClientSyntaxHighlighter.highlight(SqlClientSyntaxHighlighter.java:59)
      	at org.jline.reader.impl.LineReaderImpl.getHighlightedBuffer(LineReaderImpl.java:3633)
      	at org.jline.reader.impl.LineReaderImpl.getDisplayedBufferWithPrompts(LineReaderImpl.java:3615)
      	at org.jline.reader.impl.LineReaderImpl.redisplay(LineReaderImpl.java:3554)
      	at org.jline.reader.impl.LineReaderImpl.doCleanup(LineReaderImpl.java:2340)
      	at org.jline.reader.impl.LineReaderImpl.cleanup(LineReaderImpl.java:2332)
      	at org.jline.reader.impl.LineReaderImpl.readLine(LineReaderImpl.java:626)
      	at org.apache.flink.table.client.cli.CliClient.getAndExecuteStatements(CliClient.java:194)
      	... 7 more
      

      The issue is due to the old jline jar from Hadoop (3.3.3) classpath (/usr/lib/hadoop-yarn/lib/jline-3.9.0.jar) taking first precedence. Flink-1.18 requires jline-3.21.0.jar.

      Placing flink-sql-client.jar (bundled with jline-3.21) before the Hadoop classpath fixes the issue.

      diff --git a/flink-table/flink-sql-client/bin/sql-client.sh b/flink-table/flink-sql-client/bin/sql-client.sh
      index 24746c5dc8..4ab8635de2 100755
      --- a/flink-table/flink-sql-client/bin/sql-client.sh
      +++ b/flink-table/flink-sql-client/bin/sql-client.sh
      @@ -89,7 +89,7 @@ if [[ "$CC_CLASSPATH" =~ .*flink-sql-client.*.jar ]]; then
       elif [ -n "$FLINK_SQL_CLIENT_JAR" ]; then
       
           # start client with jar
      -    exec "$JAVA_RUN" $FLINK_ENV_JAVA_OPTS $JVM_ARGS "${log_setting[@]}" -classpath "`manglePathList "$CC_CLASSPATH:$INTERNAL_HADOOP_CLASSPATHS:$FLINK_SQL_CLIENT_JAR"`" org.apache.flink.table.client.SqlClient "$@" --jar "`manglePath $FLINK_SQL_CLIENT_JAR`"
      +    exec "$JAVA_RUN" $FLINK_ENV_JAVA_OPTS $JVM_ARGS "${log_setting[@]}" -classpath "`manglePathList "$CC_CLASSPATH:$FLINK_SQL_CLIENT_JAR:$INTERNAL_HADOOP_CLASSPATHS`" org.apache.flink.table.client.SqlClient "$@" --jar "`manglePath $FLINK_SQL_CLIENT_JAR`"
       
       # write error message to stderr
       else
      

      Attachments

        Issue Links

          Activity

            People

              prabhujoseph Prabhu Joseph
              prabhujoseph Prabhu Joseph
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: