Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-34430

Akka frame size exceeded with many ByteStreamStateHandle being used

    XMLWordPrintableJSON

Details

    • Bug
    • Status: In Progress
    • Major
    • Resolution: Unresolved
    • 1.16.3, 1.17.2, 1.19.0, 1.18.1
    • None
    • Runtime / Coordination
    • None

    Description

      The following error can happen

      Discarding oversized payload sent to Actor[akka.tcp://flink@xxxx/user/rpc/taskmanager_0#-yyyy]: max allowed size 10485760 bytes, actual size of encoded class org.apache.flink.runtime.rpc.messages.RemoteRpcInvocation was 11212046 bytes.
	
error.stack_trace
akka.remote.OversizedPayloadException: Discarding oversized payload sent to Actor[akka.tcp://flink@xxxx/user/rpc/taskmanager_0#-yyyy]: max allowed size 10485760 bytes, actual size of encoded class org.apache.flink.runtime.rpc.messages.RemoteRpcInvocation was 11212046 bytes.

      when https://issues.apache.org/jira/browse/FLINK-26050 is causing large amount of small sst files to be created and never deleted. If those files are small enough to be handled by ByteStreamStateHandle akka frame size can be exceeded.

      Attachments

        Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. FLINK-26050 Too many small sst files in rocksdb state backend when using time window created in ascending order

          • Major - Major loss of function.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. FLINK-4399 Add support for oversized messages

          • Not a Priority -
          • Open

          Activity

            People

              chesnay Chesnay Schepler
              pnowojski Piotr Nowojski
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated: