Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-5370

Union onschema + columnprune dropping used fields

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 0.18.0
    • None
    • None
    • Reviewed

    Description

      After PIG-5312, below query started failing.

      A = load 'input.txt' as (a1:int, a2:chararray, a3:int);
      B = FOREACH (GROUP A by (a1,a2)) {
          A_FOREACH = FOREACH A GENERATE a2,a3;
          GENERATE A, FLATTEN(A_FOREACH) as (a2,a3);
      }
      C = load 'input2.txt' as (A:bag{tuple:(a1: int,a2: chararray,a3:int)},a2: chararray,a3:int);
      
      D = UNION ONSCHEMA B, C;
      
      dump D;
      
      input1.txt
      1       a       3
      2       b       4
      2       c       5
      1       a       6
      2       b       7
      1       c       8
      
      input2.txt
      {(10,a0,30),(20,b0,40)} zzz     222
      
      Expected output
      ({(10,a0,30),(20,b0,40)},zzz,222)
      ({(1,a,6),(1,a,3)},a,6)
      ({(1,a,6),(1,a,3)},a,3)
      ({(1,c,8)},c,8)
      ({(2,b,7),(2,b,4)},b,7)
      ({(2,b,7),(2,b,4)},b,4)
      ({(2,c,5)},c,5)
      
      Actual (incorrect) output
      ({(10,a0,30),(20,b0,40)})    ****ONLY 1 Field ****
      ({(1,a,6),(1,a,3)},a,6)
      ({(1,a,6),(1,a,3)},a,3)
      ({(1,c,8)},c,8)
      ({(2,b,7),(2,b,4)},b,7)
      ({(2,b,7),(2,b,4)},b,4)
      ({(2,c,5)},c,5)
      

      Attachments

        1. pig-5370-v2.patch
          7 kB
          Koji Noguchi
        2. pig-5370-v1.patch
          5 kB
          Koji Noguchi

        Issue Links

          Activity

            People

              knoguchi Koji Noguchi
              knoguchi Koji Noguchi
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: