Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-7322

revisiting TestMRIntermediateDataEncryption

    XMLWordPrintableJSON

Details

    Description

      I was reviewing TestMRIntermediateDataEncryption. The unit test has actually little to do with encryption.
      I have the following conclusion:

      • Enabling/Disabling MRJobConfig.MR_ENCRYPTED_INTERMEDIATE_DATA does not change the behavior of the unit test.
      • There are no spill files generated by either mappers/reducers
      • Wrapping I/O streams with Crypto never happens during the execution of the unit test.

      Unless I misunderstand the purpose of that unit test, I suggest that it gets re-implemented so that it validates encryption in spilled intermediate data.

      Attachments

        1. MAPREDUCE-7322.001.patch
          89 kB
          Ahmed Hussein
        2. MAPREDUCE-7322.002.patch
          84 kB
          Ahmed Hussein
        3. MAPREDUCE-7322.003.patch
          88 kB
          Ahmed Hussein
        4. MAPREDUCE-7322.004.patch
          95 kB
          Ahmed Hussein
        5. MAPREDUCE-7322.005.patch
          95 kB
          Ahmed Hussein
        6. MAPREDUCE-7322.006.patch
          105 kB
          Ahmed Hussein
        7. MAPREDUCE-7322.007.patch
          92 kB
          Ahmed Hussein
        8. MAPREDUCE-7322.008.patch
          94 kB
          Ahmed Hussein
        9. MAPREDUCE-7322.009.patch
          94 kB
          Ahmed Hussein
        10. MAPREDUCE-7322.branch-2.10.009.patch
          95 kB
          Ahmed Hussein
        11. MAPREDUCE-7322.branch-3.2.009.patch
          94 kB
          Ahmed Hussein

        Issue Links

          Activity

            People

              ahussein Ahmed Hussein
              ahussein Ahmed Hussein
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: