Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-2385

Store statements not getting processed

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • 0.9.1
    • 0.9.2, 0.10.0, 0.11
    • None
    • None
    • Reviewed

    Description

      The actual script in which we got this issue is pretty big and complex. The script has total 4 STORE statements and one of the STORE statement is not getting executed.
      The script executes 3 sets of jobs (excluding one STORE which is not getting executed) consisting of 10, 11 and 19 jobs.

      The below script could be used to illustrate the issue but with Multiquery turned off;

      A = LOAD 'input1' as (f1:chararray,f2:chararray,f3:chararray);
      Z = group A all;
      Z1 = foreach Z generate COUNT(A) as count;
      B = foreach A generate f1,f2,f3,(100-Z1.count) as diff;
      C = order B by diff;
      STORE C INTO 'output/C_out';
      
      D = DISTINCT C ;
      store D into 'output/F_out';
      

      For this script, if run with Multiquery turned off, the Store command for D is not getting executed.
      I can see that the statements are getting parsed and LOStore created for D , but still, it is not getting executed.
      The above script works fine with Pig 0.8.(This issue still exists in Trunk as well)

      Attachments

        1. PIG-2385-0.patch
          1 kB
          Daniel Dai
        2. PIG-2385-1.patch
          2 kB
          Daniel Dai

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            daijy Daniel Dai
            vivekp Vivek Padmanabhan
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment