Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-8707 Implement an async pure c++ HDFS client
  3. HDFS-9108

InputStreamImpl::ReadBlockContinuation stores wrong pointers of buffers

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • None
    • HDFS-8707
    • hdfs-client
    • None
    • Ubuntu x86_64, gcc 4.8.2

    • Reviewed

    Description

      Somewhere between InputStream->PositionRead and the asio code the pointer to the destination buffer gets lost. PositionRead will correctly return the number of bytes read but the buffer won't be filled.

      This only seems to effect the remote_block_reader, RPC calls are working.

      Valgrind error:
      Syscall param recvmsg(msg.msg_iov) points to uninitialised byte(s)

      msg.msg_iov[0] should equal the buffer pointer passed to PositionRead

      Hit when using a promise to make the async call block until completion.

      auto stat = std::make_shared<std::promise<Status>>();
      std::future<Status> future(stat->get_future());

      size_t readCount = 0;
      auto h = [stat, &readCount,buf](const Status &s, size_t bytes) {
      stat->set_value(s);
      readCount = bytes;
      };

      char buf[50];
      inputStream->PositionRead(buf, 50, 0, h);

      //wait for async to finish
      future.get();

      Attachments

        1. 9108-async-repro.patch
          3 kB
          James Clampffer
        2. 9108-async-repro.patch1
          4 kB
          James Clampffer
        3. HDFS-9108.000.patch
          2 kB
          Haohui Mai

        Activity

          People

            wheat9 Haohui Mai
            James Clampffer James Clampffer
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: