Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-22857

beeline background broke the terminal

Log workAgile BoardRank to TopRank to BottomBulk Copy AttachmentsBulk Move AttachmentsAdd voteVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Blocker
    • Resolution: Unresolved
    • 1.2.1
    • None
    • Beeline, HiveServer2
    • None

    Description

      Hello,

      I am currently working on a tool which allows to:

      • Take a csv file which contains a table list
      • Make a loop which will run through the list of tables and send it to beeline to retrieve the ddl of the tables which will be sent to a file

      in my example there are 8 tables and I would like to send 5 beeline requests in the background. As soon as one of the tasks ends, the 6th request is sent to beeline etc etc. As soon as a new request ends, it sends a new request in the background.

      There must always be 5 requests at the same time.
      When I send the requests, I program well but the terminal bug at the end of the program. I write for example ls, nothing is displayed, i press enter, the command executes and print output.

      Do you have an idea?

      The code

      #!/bin/bash

      func_try(){
      beelineUsername="xxx"
      beelinePassword="xxx"
      jdcb_url='xxx'
      extra_param='hive.support.concurrency=true;hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;hive.compactor.initiator.on=true;hive.compactor.worker.threads=1;'
      request="SHOW CREATE TABLE base.${$1}"
      ref_pid=$BASHPID
      change=$2

      ddl=$( beeline -n "$beelineUsername" -p "$beelinePassword" --fastConnect=true -u "$jdcb_url$extra_param" --silent=true --outputformat="csv2" -f "./resultats/$change.hql" 2>&1)
      ddlTable=$(echo "$ddl" > "./pid/${change}_${ref_pid}_ref.txt")
      rm "${folder}/${changer}.hql"
      }

      ARG_PAR=5
      folder="./resultats"

      rm -f ."/resultats/*.hql"

      count=1

      while read -r Table;
      do

      1. Get Table to send in beeline
        if [ ! -z "$Table" ]
        then
        nb_file=$(ls ${folder} | wc -l)
      2. This loop is bloking when $ARG_PAR=5 and wait one of beeline request end
      3. for lauch the next request
        while [ $nb_file -eq ${ARG_PAR} ]
        do
        nb_file=$(ls ${folder} | wc -l)
        sleep 5
        done
      1. Create token. This token contain the request to send in beeline
        echo "SHOW CREATE TABLE braff00.$Table" > "${folder}/${count}.hql"
        func_try "${Table}" "${count}" &
        fi
        let "count+=1"
        done < "./table.txt"

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned Assign to me
            kaiiim karim

            Dates

              Created:
              Updated:

              Slack

                Issue deployment