Details
-
Bug
-
Status: Resolved
-
P3
-
Resolution: Fixed
-
2.7.0, 2.8.0, 2.9.0
-
None
Description
While reading a stream with limited number of records or time it throws an exception in the end. Though, it seems that there is no data loss, so it's a minor issue.
INFO: Creating new reader using [Checkpoint AT_TIMESTAMP for stream apache-beam, shard shardId-000000000002: null, Checkpoint AT_TIMESTAMP for stream apache-beam, shard shardId-000000000001: null] Dec 18, 2018 6:41:44 PM org.apache.beam.sdk.io.kinesis.KinesisReader start INFO: Starting reader using [Checkpoint AT_TIMESTAMP for stream apache-beam, shard shardId-000000000002: null, Checkpoint AT_TIMESTAMP for stream apache-beam, shard shardId-000000000001: null] Dec 18, 2018 6:41:46 PM org.apache.beam.sdk.io.kinesis.ShardReadersPool stop INFO: Closing shard iterators pool Dec 18, 2018 6:41:46 PM org.apache.beam.sdk.io.kinesis.ShardReadersPool readLoop SEVERE: Unexpected exception occurred java.lang.RuntimeException: Not retryable client failure at org.apache.beam.sdk.io.kinesis.SimplifiedKinesisClient.wrapExceptions(SimplifiedKinesisClient.java:225) at org.apache.beam.sdk.io.kinesis.SimplifiedKinesisClient.getRecords(SimplifiedKinesisClient.java:134) at org.apache.beam.sdk.io.kinesis.SimplifiedKinesisClient.getRecords(SimplifiedKinesisClient.java:119) at org.apache.beam.sdk.io.kinesis.ShardRecordsIterator.fetchRecords(ShardRecordsIterator.java:86) at org.apache.beam.sdk.io.kinesis.ShardRecordsIterator.readNextBatch(ShardRecordsIterator.java:76) at org.apache.beam.sdk.io.kinesis.ShardReadersPool.readLoop(ShardReadersPool.java:123) at org.apache.beam.sdk.io.kinesis.ShardReadersPool.lambda$startReadingShards$0(ShardReadersPool.java:114) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: com.amazonaws.AbortedException: at com.amazonaws.internal.SdkFilterInputStream.abortIfNeeded(SdkFilterInputStream.java:53) at com.amazonaws.internal.SdkFilterInputStream.read(SdkFilterInputStream.java:81) at com.amazonaws.event.ProgressInputStream.read(ProgressInputStream.java:180) at com.fasterxml.jackson.dataformat.cbor.CBORParserBootstrapper.ensureLoaded(CBORParserBootstrapper.java:196) at com.fasterxml.jackson.dataformat.cbor.CBORParserBootstrapper.constructParser(CBORParserBootstrapper.java:90) at com.fasterxml.jackson.dataformat.cbor.CBORFactory._createParser(CBORFactory.java:350) at com.fasterxml.jackson.dataformat.cbor.CBORFactory.createParser(CBORFactory.java:287) at com.fasterxml.jackson.dataformat.cbor.CBORFactory.createParser(CBORFactory.java:26) at com.amazonaws.http.JsonResponseHandler.handle(JsonResponseHandler.java:109) at com.amazonaws.http.JsonResponseHandler.handle(JsonResponseHandler.java:43) at com.amazonaws.http.response.AwsResponseHandlerAdapter.handle(AwsResponseHandlerAdapter.java:70) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleResponse(AmazonHttpClient.java:1554) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1272) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1056) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:743) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:717) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667) at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:649) at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:513) at com.amazonaws.services.kinesis.AmazonKinesisClient.doInvoke(AmazonKinesisClient.java:2276) at com.amazonaws.services.kinesis.AmazonKinesisClient.invoke(AmazonKinesisClient.java:2252) at com.amazonaws.services.kinesis.AmazonKinesisClient.executeGetRecords(AmazonKinesisClient.java:1062) at com.amazonaws.services.kinesis.AmazonKinesisClient.getRecords(AmazonKinesisClient.java:1038) at org.apache.beam.sdk.io.kinesis.SimplifiedKinesisClient.lambda$getRecords$2(SimplifiedKinesisClient.java:137) at org.apache.beam.sdk.io.kinesis.SimplifiedKinesisClient.wrapExceptions(SimplifiedKinesisClient.java:210) ... 11 more