Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-3350

JsonLoader() is not working properly for maps

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Minor
    • Resolution: Unresolved
    • 0.11.1
    • 0.11.1
    • data
    • None

    Description

      The code that is loading map values is ignoring the load specification and always creating String objects.

      You can reproduce the bug with the following steps:

      input file:
      a#1

      script:
      A = load 'input' as (a:map[int]);
      B = store A into 'output' using JsonStorage();
      C = load 'output' using JsonLoader() as (a:map[int]);
      D = foreach C generate a+1;

      you get the following error:
      java.lang.ClassCastException: java.lang.String cannot be cast to java.lang.Number
      at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.Add.genericGetNext(Add.java:89)
      at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.Add.getNext(Add.java:114)
      at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.getNext(PhysicalOperator.java:346)
      at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:372)
      at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:297)
      at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:283)
      at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:278)
      at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64)
      at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
      at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
      at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:214)

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              patc888 pat chan
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated: