Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-5453

FLATTEN shifting fields incorrectly

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Reopened
    • Major
    • Resolution: Unresolved
    • None
    • 0.19.0
    • impl
    • None
    • Reviewed

    Description

      Follow up from PIG-5201, PIG-5452.  

      When flatten-ed tuple has less or more fields than specified, entire fields shift incorrectly. 

      Input

      A       (a,b,c)
      B       (a,b,c)
      C       (a,b,c)
      Y       (a,b)
      Z       (a,b,c,d,e,f)
      E

      Script

      A = load 'input.txt' as (a1:chararray, a2:tuple());
      B = FOREACH A GENERATE a1, FLATTEN(a2) as (b1:chararray,b2:chararray,b3:chararray), a1 as a4;
      dump B; 

      Incorrect results

      (A,a,b,c,A)
      (B,a,b,c,B)
      (C,a,b,c,C)
      (Y,a,b,Y,)
      (Z,a,b,c,d)
      (E,,,,E)

      E is correct.  It's fixed as part of PIG-5201, PIG-5452.
      Y has shifted a4(Y) to the left incorrectly. 
      Should have been (Y,a,b,,Y)
      Z has dropped a4(Z) and overwrote the result with content of FLATTEN(a2).
      Should have been (Z,a,b,c,Z).

       

      Attachments

        1. pig-5453-v01.patch
          4 kB
          Koji Noguchi
        2. pig-5453-v02.patch
          4 kB
          Koji Noguchi

        Activity

          People

            knoguchi Koji Noguchi
            knoguchi Koji Noguchi
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: