Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-8707 Implement an async pure c++ HDFS client
  3. HDFS-12103

libhdfs++: Provide workaround to support cancel on filesystem connect until HDFS-11437 is resolved

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • None
    • hdfs-client
    • None

    Description

      HDFS-11437 is going to take a non-trivial amount of work to do right. In the meantime it'd be nice to have a way to cancel pending connections (even when the FS claimed they are finished).

      Proposed workaround is to relax the rules about when FileSystem::CancelPending connect can be called since it isn't able to properly determine when it's connected anyway. In order to determine when the FS has connected you can do some simple RPC call since that will wait on failover. If CancelPending can be called during that first RPC call then it will effectively be canceling FileSystem::Connect

      Current cancel rules - asterisk on steps where CancelPending is allowed

      FileSystem::Connect called
      FileSystem communicates with first NN *
      FileSystem::Connect returns - even if it hasn't communicated with the active NN

      Proposed relaxation
      FileSystem::Connect called
      FileSystem communicates with first NN*
      FileSystem::Connect returns *
      FileSystem::GetFileInfo called * -any namenode RPC call will do, ignore perm errors
      RPC engine blocks until it hits the active or runs out of retries *
      FileSystem::GetFileInfo returns

      It'd be up to the user to add in the dummy NN RPC call. Once HDFS-11437 is fixed this workaround can be removed.

      Attachments

        1. HDFS-12103.HDFS-8707.000.patch
          2 kB
          James Clampffer

        Activity

          People

            James C James Clampffer
            James C James Clampffer
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: