Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-7089

ReadMapper.doIO hangs with user misconfigured inputs

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Minor
    • Resolution: Unresolved
    • 2.5.0
    • None
    • client, test
    • None

    Description

      When a user configures the bufferSize to be 0, the while loop in TestDFSIO$ReadMapper.doIO function hangs endlessly.

      This is because the loop stride, curSize is always 0, making the loop index actualSize always less than the upper bound totalSize.

      Here is the code snippet.

      int bufferSize = DEFAULT_BUFFER_SIZE;
      
      for(int i = 0; i < args.length; i++) { // parse command line
        ...
        else if (args[i].equals("-bufferSize")) {
        bufferSize = Integer.parseInt(args[++i]);
        }
        ...
      }
      
      public Long doIO(Reporter reporter, String name, long totalSize) throws IOException {
        InputStream in = (InputStream)this.stream;
        long actualSize = 0;
        while (actualSize < totalSize) {
          int curSize = in.read(buffer, 0, bufferSize);
          if(curSize < 0) break;
          actualSize += curSize;
          reporter.setStatus("reading " + name + "@" + actualSize + "/" + totalSize + " ::host = " + hostName);
        }
        return Long.valueOf(actualSize);
      }
      

      Attachments

        Activity

          People

            Unassigned Unassigned
            dustinday John Doe
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated: