Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-13526

Unable to use the KafkaIO.Read.withValueDeserializer(DeserializerProvider) method

Details

    • Bug
    • Status: Resolved
    • P2
    • Resolution: Fixed
    • 2.34.0
    • 2.36.0
    • io-java-kafka
    • None

    Description

      I'm trying to configure my pipeline to read AVRO-encoded messages from Kafka with deserialization delegated to the ConfluentSchemaRegistryDeserialzerProvider, and I'm getting the following error:

      Exception in thread "main" java.lang.IllegalAccessError: failed to access class org.apache.beam.sdk.io.kafka.DeserializerProvider from class MyClass (both are in unnamed module of loader 'app')

      The way I configure the KafkaIO.Read is like:

      KafkaIO.read<String, GenericRecord>()
          .withBootstrapServers(...)
          .withKeyDeserializer(StringDeserializer::class.java)
          .withValueDeserializer(ConfluentSchemaRegistryDeserializerProvider.of<GenericRecord>(...))
          .withTopic(...)
          ...

      I checked the DeserializerProvider interface and it's package-private - it's not even listed in the official java docs for Beam (https://beam.apache.org/releases/javadoc/2.34.0/org/apache/beam/sdk/io/kafka/package-summary.html); the ConfluentSchemaRegistryDeserialzerProvider is public.

      Am I the only one trying to use this API, or am I doing sth terribly wrong?

       

      Attachments

        Issue Links

          Activity

            People

              jigga Arkadiusz Gasinski
              jigga Arkadiusz Gasinski
              Votes:
              6 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 40m
                  40m