Description
METRON-793 migrated the storm topologies to the storm-kafka-client spout, which supports kerberos. To complete the kerberos work on the existing topologies, we need to be able to enable the spouts and kafka writers to use security protocols other than PLAINTEXT. Also, enabling auto-renew plugins for storm will enable the topologies to run for extended durations in a kerberized cluster.
This work was inspired by a portion of the investigatory work done at https://github.com/dlyle65535/incubator-metron/tree/kerb-testing?files=1 by:
- @dlyle65535
- @merrimanr
- @mmiklavc
This carves out a specific piece of that functionality with the following differences:
- The mpack work is not included, but the properties are set up to enable it as a follow-on
- It presumes
METRON-793, so it uses storm-kafka-client rather than storm-kafka - It adds a flag when starting the parsers to pass the security protocol and sets up the writers and the spout automatically rather than relying on the set of extra kafka configs (though both approaches would work here).
NOTE: This does not encompass MPack changes to enable kerberos (METRON-799) or fix the sensors to work with a kerberized kafka (METRON-798). That would be follow-on work.
Attachments
Issue Links
- is blocked by
-
METRON-793 Migrate to storm-kafka-client kafka spout from storm-kafka
- Done
- is depended upon by
-
METRON-804 Create a document to describe kerberizing vagrant
- Done
- links to