Uploaded image for project: 'jclouds'
  1. jclouds
  2. JCLOUDS-532

Potential for read on closed stream when handling HTTP 5xx errors

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.8.0
    • 1.9.0, 2.0.0
    • jclouds-core
    • None

    Description

      I've seen this traceback when testing JClouds against a broken Cinder endpoint (though I suspect something similar is possible in other APIs):

      Caused by: org.jclouds.http.HttpResponseException: java.io.IOException: Attempted read on closed stream. connecting to DELETE http://192.168.56.20:8776/v1/c33361f3a390470c918276fdd50795ba /volumes/d77bf972-3917-43e8-85dd-894611fcdc4d HTTP/1.1
          	at org.jclouds.http.internal.BaseHttpCommandExecutorService.invoke(BaseHttpCommandExecutorService.java:162)
          	at org.jclouds.rest.internal.InvokeHttpMethod.invoke(InvokeHttpMethod.java:93)
          	at org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:76)
          	at org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:47)
          	at org.jclouds.reflect.FunctionalReflection$FunctionalInvocationHandler.handleInvocation(FunctionalReflection.java:117)
          	at com.google.common.reflect.AbstractInvocationHandler.invoke(AbstractInvocationHandler.java:87)
          	at com.sun.proxy.$Proxy98.delete(Unknown Source)
          	... 49 more
          Caused by: java.lang.RuntimeException: java.io.IOException: Attempted read on closed stream.
          	at com.google.common.base.Throwables.propagate(Throwables.java:160)
          	at org.jclouds.http.HttpUtils.toByteArrayOrNull(HttpUtils.java:131)
          	at org.jclouds.http.HttpUtils.closeClientButKeepContentStream(HttpUtils.java:163)
          	at org.jclouds.openstack.cinder.v1.handlers.CinderErrorHandler.handleError(CinderErrorHandler.java:42)
          	at org.jclouds.http.handlers.DelegatingErrorHandler.handleError(DelegatingErrorHandler.java:67)
          	at org.jclouds.http.internal.BaseHttpCommandExecutorService.shouldContinue(BaseHttpCommandExecutorService.java:180)
          	at org.jclouds.http.internal.BaseHttpCommandExecutorService.invoke(BaseHttpCommandExecutorService.java:150)
          	... 56 more
          Caused by: java.io.IOException: Attempted read on closed stream.
          	at org.apache.http.conn.EofSensorInputStream.isReadAllowed(EofSensorInputStream.java:109)
          	at org.apache.http.conn.EofSensorInputStream.read(EofSensorInputStream.java:135)
          	at java.io.FilterInputStream.read(FilterInputStream.java:133)
          	at java.io.FilterInputStream.read(FilterInputStream.java:107)
          	at com.google.common.io.ByteStreams.copy(ByteStreams.java:175)
          	at com.google.common.io.ByteStreams.toByteArray(ByteStreams.java:220)
          	at org.jclouds.http.HttpUtils.toByteArrayOrNull(HttpUtils.java:129)
          	... 61 more
      

      This seems possible if the payload in an HttpResponse is an InputStreamPayload and the HTTP status code received is a 5xx.
      The BackoffLimitedRetryHandler can hit this line to release the payload from a request, which will close the stream: https://github.com/jclouds/jclouds/blob/master/core/src/main/java/org/jclouds/http/handlers/BackoffLimitedRetryHandler.java#L99
      Then the CinderErrorHandler attempts to read the stream here after the HttpResponse is passed off to it:
      https://github.com/jclouds/jclouds/blob/master/apis/openstack-cinder/src/main/java/org/jclouds/openstack/cinder/v1/handlers/CinderErrorHandler.java#L42

      I would like to be able to write a test to reproduce this, but it doesn't seem that I can set the payload correctly in the VolumeApiExpectTest to trigger this. (I get a null payload instead of an InputStreamPayload when trying to build an HttpResponse with the latter.)

      Attachments

        Issue Links

          Activity

            People

              nacx Ignasi Barrera
              jasdeep Jasdeep Hundal
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: