Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-7088

DistributedFSCheckMapper.doIO hangs with user misconfigured inputs

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Minor
    • Resolution: Unresolved
    • 2.5.0
    • None
    • client, test
    • None

    Description

      When a user configures the bufferSize to be 0, the for loop in DistributedFSCheck$DistributedFSCheckMapper.doIO function hangs endlessly. Here is the code snippet.

          int bufferSize = DEFAULT_BUFFER_SIZE;
      
          for(int i = 0; i < args.length; i++) { // parse command line
           ...
           else if (args[i].equals("-bufferSize")) {
              bufferSize = Integer.parseInt(args[++i]);
            }
           ...
          }
          public Object doIO(Reporter reporter, String name,  long offset) throws IOException {
            // open file
            FSDataInputStream in = null;
            Path p = new Path(name);
            try {
              in = fs.open(p);
            } catch(IOException e) {
              return name + "@(missing)";
            }
            in.seek(offset);
            long actualSize = 0;
            try {
              long blockSize = fs.getDefaultBlockSize(p);
              reporter.setStatus("reading " + name + "@" + offset + "/" + blockSize);
              for( int curSize = bufferSize; 
                   curSize == bufferSize && actualSize < blockSize;
                   actualSize += curSize) {
                curSize = in.read(buffer, 0, bufferSize);
              }
            } catch(IOException e) {
              ...
            } finally {
              in.close();
            }
            return new Long(actualSize);
          }
      

      Attachments

        Activity

          People

            Unassigned Unassigned
            dustinday John Doe
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: