Details
-
New Feature
-
Status: Open
-
P3
-
Resolution: Unresolved
-
None
-
None
-
None
Description
dataflow_metrics.py expects a job object to be passed in, it cannot fully query the metrics on its own.
Updating this to use the dataflow_client to also query the job itself (non trivial since the job passed in is a wrapper for the Job proto the dataflow_client obtains, this wrapper is created by dataflow_runner.) Though we may not need any of the extra wrapper features, since we only need access to the graph. But it is unclear if this will work, as the graph DatalowRunner generates is not the same graph as the one queried out.
The job graph is use to translate step names