Uploaded image for project: 'Kudu'
  1. Kudu
  2. KUDU-2241

NoSuchMethodError happened when run flume agent using kudu flume sink

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.5.0
    • Fix Version/s: None
    • Component/s: flume-sink
    • Labels:
      None

      Description

      I've installed kudu and flume components from cloudera manager, and when I start flume agent the following error happened.

      Error

      17/12/11 10:46:29 ERROR node.PollingPropertiesFileConfigurationProvider: Unhandled error
      java.lang.NoSuchMethodError: org.apache.flume.Context.getSubProperties(Ljava/lang/String;)Lorg/apache/kudu/shaded/com/google/common/collect/ImmutableMap;
      at org.apache.kudu.flume.sink.KuduSink.configure(KuduSink.java:206)
      at org.apache.flume.conf.Configurables.configure(Configurables.java:41)
      at org.apache.flume.node.AbstractConfigurationProvider.loadSinks(AbstractConfigurationProvider.java:411)
      at org.apache.flume.node.AbstractConfigurationProvider.getConfiguration(AbstractConfigurationProvider.java:102)
      at org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:141)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
      at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
      at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      at java.lang.Thread.run(Thread.java:745)

      Version infomation:

      • flume: 1.6.0-cdh5.13.0
      • kudu: 1.5.0-cdh5.13.0

      I checked the classpath of flume agent and found that maybe there is version conflict of guava jar between flume and kudu flume sink(kudu-flume-sink-1.5.0-cdh5.13.0.jar) which includes shaded guava jar.

      Maybe this problem related to the changes made here.

      To make flume agent run properly, I excluded related class(ImmutableMap etc) from shade settings in pom.xml and found that worked.

      kudu 1.5.0 top-level pom.xml

      <relocation>
      <pattern>com.google.common</pattern>
      <shadedPattern>org.apache.kudu.shaded.com.google.common</shadedPattern>
      <excludes>
      <exclude>com.google.common.collect.ImmutableMap*</exclude>
      <exclude>com.google.common.collect.ImmutableEnumMap*</exclude>
      </excludes>
      </relocation>

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              yechangyao Changyao Ye
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: