Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-13204 Über-jira: S3a phase III: scale and tuning
  3. HADOOP-14774

S3A case "testRandomReadOverBuffer" failed due to improper range parameter

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 2.8.0
    • 2.9.0, 3.0.0-beta1
    • fs/s3
    • None
    • Hadoop 2.8.0
      s3-compatible storage

    Description

      Running org.apache.hadoop.fs.s3a.scale.ITestS3AInputStreamPerformance
      Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 2.821 sec <<< FAILURE! - in org.apache.hadoop.fs.s3a.scale.ITestS3AInputStreamPerformance
      testRandomReadOverBuffer(org.apache.hadoop.fs.s3a.scale.ITestS3AInputStreamPerformance)  Time elapsed: 2.605 sec  <<< FAILURE!
      java.lang.AssertionError: expected:<8192> but was:<8193>
      	at org.junit.Assert.fail(Assert.java:88)
      	at org.junit.Assert.failNotEquals(Assert.java:743)
      	at org.junit.Assert.assertEquals(Assert.java:118)
      	at org.junit.Assert.assertEquals(Assert.java:555)
      	at org.junit.Assert.assertEquals(Assert.java:542)
      	at org.apache.hadoop.fs.s3a.scale.ITestS3AInputStreamPerformance.testRandomReadOverBuffer(ITestS3AInputStreamPerformance.java:533)
      

      From log, the length of content is exceed than our expect:

      2017-08-15 16:53:16,464 [JUnit-testRandomReadOverBuffer] DEBUG http.headers (DefaultClientConnection.java:sendRequestHeader(280)) - >> GET /test-aws-s3a/test/testReadOverBuffer.bin HTTP/1.1
      2017-08-15 16:53:16,464 [JUnit-testRandomReadOverBuffer] DEBUG http.headers (DefaultClientConnection.java:sendRequestHeader(283)) - >> Host: 10.0.2.254
      2017-08-15 16:53:16,465 [JUnit-testRandomReadOverBuffer] DEBUG http.headers (DefaultClientConnection.java:sendRequestHeader(283)) - >> x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
      2017-08-15 16:53:16,465 [JUnit-testRandomReadOverBuffer] DEBUG http.headers (DefaultClientConnection.java:sendRequestHeader(283)) - >> Authorization: AWS4-HMAC-SHA256 Credential=JFDAM9KF9IY8S5P0JIV6/20170815/us-east-1/s3/aws4_request, SignedHeaders=content-type;host;range;user-agent;x-amz-content-sha256;x-amz-date, Signature=42bce4a43d2b1bf6e6d599613c60812e6716514da4ef5b3839ef0566c31279ee
      2017-08-15 16:53:16,466 [JUnit-testRandomReadOverBuffer] DEBUG http.headers (DefaultClientConnection.java:sendRequestHeader(283)) - >> X-Amz-Date: 20170815T085316Z
      2017-08-15 16:53:16,466 [JUnit-testRandomReadOverBuffer] DEBUG http.headers (DefaultClientConnection.java:sendRequestHeader(283)) - >> User-Agent: Hadoop 2.8.0, aws-sdk-java/1.10.6 Linux/3.10.0-514.21.2.el7.x86_64 Java_HotSpot(TM)_64-Bit_Server_VM/25.131-b11/1.8.0_131
      2017-08-15 16:53:16,467 [JUnit-testRandomReadOverBuffer] DEBUG http.headers (DefaultClientConnection.java:sendRequestHeader(283)) - >> Range: bytes=0-8192
      2017-08-15 16:53:16,467 [JUnit-testRandomReadOverBuffer] DEBUG http.headers (DefaultClientConnection.java:sendRequestHeader(283)) - >> Content-Type: application/x-www-form-urlencoded; charset=utf-8
      2017-08-15 16:53:16,467 [JUnit-testRandomReadOverBuffer] DEBUG http.headers (DefaultClientConnection.java:sendRequestHeader(283)) - >> Connection: Keep-Alive
      2017-08-15 16:53:16,473 [JUnit-testRandomReadOverBuffer] DEBUG http.wire (Wire.java:wire(72)) -  << "HTTP/1.1 206 Partial Content[\r][\n]"
      2017-08-15 16:53:16,475 [JUnit-testRandomReadOverBuffer] DEBUG http.wire (Wire.java:wire(72)) -  << "Content-Range: bytes 0-8192/32768[\r][\n]"
      2017-08-15 16:53:16,476 [JUnit-testRandomReadOverBuffer] DEBUG http.wire (Wire.java:wire(72)) -  << "Content-Length: 8193[\r][\n]"
      2017-08-15 16:53:16,477 [JUnit-testRandomReadOverBuffer] DEBUG http.wire (Wire.java:wire(72)) -  << "Accept-Ranges: bytes[\r][\n]"
      2017-08-15 16:53:16,478 [JUnit-testRandomReadOverBuffer] DEBUG http.wire (Wire.java:wire(72)) -  << "Last-Modified: Tue, 15 Aug 2017 08:51:39 GMT[\r][\n]"
      2017-08-15 16:53:16,479 [JUnit-testRandomReadOverBuffer] DEBUG http.wire (Wire.java:wire(72)) -  << "ETag: "e7191764798ba504d6671d4c434d2f4d"[\r][\n]"
      2017-08-15 16:53:16,480 [JUnit-testRandomReadOverBuffer] DEBUG http.wire (Wire.java:wire(72)) -  << "x-amz-request-id: tx00000000000000000001e-005992b67e-27a45-default[\r][\n]"
      2017-08-15 16:53:16,481 [JUnit-testRandomReadOverBuffer] DEBUG http.wire (Wire.java:wire(72)) -  << "Content-Type: application/octet-stream[\r][\n]"
      2017-08-15 16:53:16,482 [JUnit-testRandomReadOverBuffer] DEBUG http.wire (Wire.java:wire(72)) -  << "Date: Tue, 15 Aug 2017 08:53:18 GMT[\r][\n]"
      2017-08-15 16:53:16,483 [JUnit-testRandomReadOverBuffer] DEBUG http.wire (Wire.java:wire(72)) -  << "[\r][\n]"
      

      Attachments

        1. HADOOP-14774.003.patch
          0.9 kB
          Yonger
        2. HADOOP-14774.002.patch
          0.8 kB
          Yonger
        3. HADOOP-14774.001.patch
          0.8 kB
          Yonger
        4. failsafe-report.html
          283 kB
          Yonger

        Activity

          People

            iyonger Yonger
            iyonger Yonger
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: