Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-34581

streaming code throws java.lang.reflect.InaccessibleObjectException

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 1.18.1
    • None
    • None

    Description

      I have a pretty simple test pipeline (read a bunch of tables from Apache Kafka, join and project them, then write to Apache Iceberg) that I run locally with Junit5 and the MiniClusterExtension. This works ok with Java 11 and Flink 1.17.2 and Flink 1.18.1.

      With Java 17, I see

      [WARN ] IcebergStreamWriter (1/1)#1 (78d9251dbbab3aae84bf303dfc080d23_626c1e687bcaad0c13c507629a5894a8_0_1) switched from RUNNING to FAILED with failure cause:
      java.io.IOException: Could not perform checkpoint 2 for operator IcebergStreamWriter (1/1)#1.
      at org.apache.flink.streaming.runtime.tasks.StreamTask.triggerCheckpointOnBarrier(StreamTask.java:1275)
      at org.apache.flink.streaming.runtime.io.checkpointing.CheckpointBarrierHandler.notifyCheckpoint(CheckpointBarrierHandler.java:147)
      at org.apache.flink.streaming.runtime.io.checkpointing.SingleCheckpointBarrierHandler.triggerCheckpoint(SingleCheckpointBarrierHandler.java:287)
      at org.apache.flink.streaming.runtime.io.checkpointing.SingleCheckpointBarrierHandler.access$100(SingleCheckpointBarrierHandler.java:64)
      at org.apache.flink.streaming.runtime.io.checkpointing.SingleCheckpointBarrierHandler$ControllerImpl.triggerGlobalCheckpoint(SingleCheckpointBarrierHandler.java:488)
      at org.apache.flink.streaming.runtime.io.checkpointing.AbstractAlignedBarrierHandlerState.triggerGlobalCheckpoint(AbstractAlignedBarrierHandlerState.java:74)
      at org.apache.flink.streaming.runtime.io.checkpointing.AbstractAlignedBarrierHandlerState.barrierReceived(AbstractAlignedBarrierHandlerState.java:66)
      at org.apache.flink.streaming.runtime.io.checkpointing.SingleCheckpointBarrierHandler.lambda$processBarrier$2(SingleCheckpointBarrierHandler.java:234)
      at org.apache.flink.streaming.runtime.io.checkpointing.SingleCheckpointBarrierHandler.markCheckpointAlignedAndTransformState(SingleCheckpointBarrierHandler.java:262)
      at org.apache.flink.streaming.runtime.io.checkpointing.SingleCheckpointBarrierHandler.processBarrier(SingleCheckpointBarrierHandler.java:231)
      at org.apache.flink.streaming.runtime.io.checkpointing.CheckpointedInputGate.handleEvent(CheckpointedInputGate.java:181)
      at org.apache.flink.streaming.runtime.io.checkpointing.CheckpointedInputGate.pollNext(CheckpointedInputGate.java:159)
      at org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.emitNext(AbstractStreamTaskNetworkInput.java:118)
      at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65)
      at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:562)
      at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:231)
      at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:858)
      at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:807)
      at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:953)
      at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:932)
      at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:746)
      at org.apache.flink.runtime.taskmanager.Task.run(Task.java:562)
      at java.base/java.lang.Thread.run(Thread.java:840)
      Caused by: java.lang.RuntimeException: java.lang.reflect.InaccessibleObjectException: Unable to make field private final java.lang.Object[] java.util.Arrays$ArrayList.a accessible: module java.base does not "opens java.util" to unnamed module @6af93788
      at com.twitter.chill.java.ArraysAsListSerializer.<init>(ArraysAsListSerializer.java:69)
      at org.apache.flink.api.java.typeutils.runtime.kryo.FlinkChillPackageRegistrar.registerSerializers(FlinkChillPackageRegistrar.java:67)
      at org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.getKryoInstance(KryoSerializer.java:513)
      at org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.checkKryoInitialized(KryoSerializer.java:522)
      at org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.serialize(KryoSerializer.java:348)
      at org.apache.flink.streaming.runtime.streamrecord.StreamElementSerializer.serialize(StreamElementSerializer.java:165)
      at org.apache.flink.streaming.runtime.streamrecord.StreamElementSerializer.serialize(StreamElementSerializer.java:43)
      at org.apache.flink.runtime.plugable.SerializationDelegate.write(SerializationDelegate.java:54)
      at org.apache.flink.runtime.io.network.api.writer.RecordWriter.serializeRecord(RecordWriter.java:141)
      at org.apache.flink.runtime.io.network.api.writer.RecordWriter.emit(RecordWriter.java:107)
      at org.apache.flink.runtime.io.network.api.writer.ChannelSelectorRecordWriter.emit(ChannelSelectorRecordWriter.java:55)
      at org.apache.flink.streaming.runtime.io.RecordWriterOutput.pushToRecordWriter(RecordWriterOutput.java:134)
      at org.apache.flink.streaming.runtime.io.RecordWriterOutput.collectAndCheckIfChained(RecordWriterOutput.java:114)
      at org.apache.flink.streaming.runtime.io.RecordWriterOutput.collect(RecordWriterOutput.java:95)
      at org.apache.flink.streaming.runtime.io.RecordWriterOutput.collect(RecordWriterOutput.java:48)
      at org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:59)
      at org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:31)
      at org.apache.iceberg.flink.sink.IcebergStreamWriter.flush(IcebergStreamWriter.java:113)
      at org.apache.iceberg.flink.sink.IcebergStreamWriter.prepareSnapshotPreBarrier(IcebergStreamWriter.java:66)
      at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.prepareSnapshotPreBarrier(RegularOperatorChain.java:89)
      at org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.checkpointState(SubtaskCheckpointCoordinatorImpl.java:322)
      at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$performCheckpoint$15(StreamTask.java:1318)
      at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:50)
      at org.apache.flink.streaming.runtime.tasks.StreamTask.performCheckpoint(StreamTask.java:1306)
      at org.apache.flink.streaming.runtime.tasks.StreamTask.triggerCheckpointOnBarrier(StreamTask.java:1263)
      ... 22 more
      Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make field private final java.lang.Object[] java.util.Arrays$ArrayList.a accessible: module java.base does not "opens java.util" to unnamed module @6af93788
      at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:354)
      at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:297)
      at java.base/java.lang.reflect.Field.checkCanSetAccessible(Field.java:178)
      at java.base/java.lang.reflect.Field.setAccessible(Field.java:172)
      at com.twitter.chill.java.ArraysAsListSerializer.<init>(ArraysAsListSerializer.java:67)
      ... 46 more

      This is blocking us right now from upgrading to Java 17.

      Attachments

        Activity

          People

            Unassigned Unassigned
            henning Henning Schmiedehausen
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated: