Triage Metrics Produced by the Profiler

Problem

The motivating example is that I would like to create an alert if the number of inbound flows to any host over a 15 minute interval is abnormal.

The value being interrogated here, the number of inbound flows, is not a static value contained within any single telemetry message. This value is calculated across multiple messages by the Profiler. The current Threat Triage process cannot be used to interrogate values calculated by the Profiler.

Proposed Solution

I am proposing that we treat the Profiler as a source of telemetry. The measurements captured by the Profiler would be enqueued into a Kafka topic. We would then treat those Profiler messages like any other telemetry. We would parse, enrich, triage, and index those messages.

This would have the following advantages.

1. We would be able to reuse the same threat triage mechanism for values calculated by the Profiler.

2. We would be able to generate profiles from the profiled data - aka meta-profiles anyone?