Details
-
Bug
-
Status: Resolved
-
P2
-
Resolution: Cannot Reproduce
-
None
-
None
Description
Chicago taxi example is failing when running on 1 node cluster with 256MB data.
Disk usage:
goenka@goenka:/home/build$ ls -lha /tmp/flink-io-b9f13afc-0c5a-40ab-9a29-037d35068c1c/ total 8.5G drwxr-xr-x 2 goenka primarygroup 4.0K Nov 12 20:27 . drwxrwxrwt 37 root root 11M Nov 12 20:27 .. -rw-r--r-- 1 goenka primarygroup 2.2G Nov 12 20:27 550b3393ce2a4c35ba37135c20ebccb3.channel -rw-r--r-- 1 goenka primarygroup 2.2G Nov 12 20:27 573f0fcb90732308857025108ffc74f6.channel -rw-r--r-- 1 goenka primarygroup 2.2G Nov 12 20:27 fb08a9ea467a1045f6087088b395ea8d.channel -rw-r--r-- 1 goenka primarygroup 2.2G Nov 12 20:27 fcca6cbf619f712d852d2371d9cb7046.channel
Error in logs:
2018-11-12 20:45:09,344 ERROR org.apache.flink.runtime.operators.BatchTask - Error in task code: GroupReduce (GroupReduce at GenerateStatistics/RunStatsGenerators/CommonStatsGenerator/CombinePerKey/GroupByKey) (1/1) java.lang.Exception: The data preparation for task 'GroupReduce (GroupReduce at GenerateStatistics/RunStatsGenerators/CommonStatsGenerator/CombinePerKey/GroupByKey)' , caused an error: Error obtaining the sorted input: Thread 'SortMerger Reading Thread' terminated due to an exception: The record exceeds the maximum s ize of a sort buffer (current maximum: 30310400 bytes). at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:479) at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:712) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.RuntimeException: Error obtaining the sorted input: Thread 'SortMerger Reading Thread' terminated due to an exception: The record exceeds the maximum size of a sort buffer (current maximum: 30310400 bytes). at org.apache.flink.runtime.operators.sort.UnilateralSortMerger.getIterator(UnilateralSortMerger.java:650) at org.apache.flink.runtime.operators.BatchTask.getInput(BatchTask.java:1108) at org.apache.flink.runtime.operators.GroupReduceDriver.prepare(GroupReduceDriver.java:99) at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:473) ... 3 more Caused by: java.io.IOException: Thread 'SortMerger Reading Thread' terminated due to an exception: The record exceeds the maximum size of a sort buffer (current maximum: 30310400 bytes). at org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ThreadBase.run(UnilateralSortMerger.java:831) Caused by: java.io.IOException: The record exceeds the maximum size of a sort buffer (current maximum: 30310400 bytes). at org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ReadingThread.go(UnilateralSortMerger.java:986) at org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ThreadBase.run(UnilateralSortMerger.java:827) 2018-11-12 20:45:09,348 INFO org.apache.flink.runtime.taskmanager.Task - GroupReduce (GroupReduce at GenerateStatistics/RunStatsGenerators/CommonStatsGenerator/CombinePerKey/GroupByKey) (1/1) (dcd3891fa9540bb4bf594a1a3d631e2e) switched from RUNNING to FAILED. java.lang.Exception: The data preparation for task 'GroupReduce (GroupReduce at GenerateStatistics/RunStatsGenerators/CommonStatsGenerator/CombinePerKey/GroupByKey)' , caused an error: Error obtaining the sorted input: Thread 'SortMerger Reading Thread' terminated due to an exception: The record exceeds the maximum s ize of a sort buffer (current maximum: 30310400 bytes). at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:479) at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:712) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.RuntimeException: Error obtaining the sorted input: Thread 'SortMerger Reading Thread' terminated due to an exception: The record exceeds the maximum size of a sort buffer (current maximum: 30310400 bytes). at org.apache.flink.runtime.operators.sort.UnilateralSortMerger.getIterator(UnilateralSortMerger.java:650) at org.apache.flink.runtime.operators.BatchTask.getInput(BatchTask.java:1108) at org.apache.flink.runtime.operators.GroupReduceDriver.prepare(GroupReduceDriver.java:99) at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:473) ... 3 more Caused by: java.io.IOException: Thread 'SortMerger Reading Thread' terminated due to an exception: The record exceeds the maximum size of a sort buffer (current maximum: 30310400 bytes). at org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ThreadBase.run(UnilateralSortMerger.java:831) Caused by: java.io.IOException: The record exceeds the maximum size of a sort buffer (current maximum: 30310400 bytes). at org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ReadingThread.go(UnilateralSortMerger.java:986) at org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ThreadBase.run(UnilateralSortMerger.java:827) 2018-11-12 20:45:09,352 INFO org.apache.flink.runtime.taskmanager.Task - Freeing task resources for GroupReduce (GroupReduce at GenerateStatistics/RunStatsGenerators/CommonStatsGenerator/CombinePerKey/GroupByKey) (1/1) (dcd3891fa9540bb4bf594a1a3d631e2e). 2018-11-12 20:45:09,357 WARN bundle_processor.create_operation - No unique name set for transform fn/write/ref_PCollection_PCollection_77:0 2018-11-12 20:45:09,361 ERROR org.apache.flink.runtime.operators.BatchTask - Error in task code: GroupReduce (GroupReduce at GenerateStatistics/RunStatsGenerators/NumericStatsGenerator/CombinePerKey/GroupByKey) (1/1) java.lang.Exception: The data preparation for task 'GroupReduce (GroupReduce at GenerateStatistics/RunStatsGenerators/NumericStatsGenerator/CombinePerKey/GroupByKey)' , caused an error: Error obtaining the sorted input: Thread 'SortMerger Reading Thread' terminated due to an exception: The record exceeds the maximum size of a sort buffer (current maximum: 30310400 bytes). at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:479) at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:712) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.RuntimeException: Error obtaining the sorted input: Thread 'SortMerger Reading Thread' terminated due to an exception: The record exceeds the maximum size of a sort buffer (current maximum: 30310400 bytes). at org.apache.flink.runtime.operators.sort.UnilateralSortMerger.getIterator(UnilateralSortMerger.java:650) at org.apache.flink.runtime.operators.BatchTask.getInput(BatchTask.java:1108) at org.apache.flink.runtime.operators.GroupReduceDriver.prepare(GroupReduceDriver.java:99) at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:473) ... 3 more Caused by: java.io.IOException: Thread 'SortMerger Reading Thread' terminated due to an exception: The record exceeds the maximum size of a sort buffer (current maximum: 30310400 bytes). at org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ThreadBase.run(UnilateralSortMerger.java:831) Caused by: java.io.IOException: The record exceeds the maximum size of a sort buffer (current maximum: 30310400 bytes). at org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ReadingThread.go(UnilateralSortMerger.java:986) at org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ThreadBase.run(UnilateralSortMerger.java:827) 2018-11-12 20:45:09,362 INFO org.apache.flink.runtime.taskmanager.Task - GroupReduce (GroupReduce at GenerateStatistics/RunStatsGenerators/NumericStatsGenerator/CombinePerKey/GroupByKey) (1/1) (51ce95407b2dd1e5b37501663e255dbd) switched from RUNNING to FAILED. java.lang.Exception: The data preparation for task 'GroupReduce (GroupReduce at GenerateStatistics/RunStatsGenerators/NumericStatsGenerator/CombinePerKey/GroupByKey)' , caused an error: Error obtaining the sorted input: Thread 'SortMerger Reading Thread' terminated due to an exception: The record exceeds the maximum size of a sort buffer (current maximum: 30310400 bytes). at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:479) at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:712) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.RuntimeException: Error obtaining the sorted input: Thread 'SortMerger Reading Thread' terminated due to an exception: The record exceeds the maximum size of a sort buffer (current maximum: 30310400 bytes). at org.apache.flink.runtime.operators.sort.UnilateralSortMerger.getIterator(UnilateralSortMerger.java:650) at org.apache.flink.runtime.operators.BatchTask.getInput(BatchTask.java:1108) at org.apache.flink.runtime.operators.GroupReduceDriver.prepare(GroupReduceDriver.java:99) at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:473) ... 3 more Caused by: java.io.IOException: Thread 'SortMerger Reading Thread' terminated due to an exception: The record exceeds the maximum size of a sort buffer (current maximum: 30310400 bytes). at org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ThreadBase.run(UnilateralSortMerger.java:831) Caused by: java.io.IOException: The record exceeds the maximum size of a sort buffer (current maximum: 30310400 bytes). at org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ReadingThread.go(UnilateralSortMerger.java:986) at org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ThreadBase.run(UnilateralSortMerger.java:827) 2018-11-12 20:45:09,363 WARN bundle_processor.create_operation - No unique name set for transform fn/read/ref_PCollection_PCollection_74:0
Job Stats:
{
"jid": "cbbca35225bdde85aeeb7146ef19ce69",
"name": "BeamApp-goenka-1113041702-c9ef27fe",
"isStoppable": false,
"state": "FAILED",
"start-time": 1542082624238,
"end-time": 1542083249151,
"duration": 624913,
"now": 1542083464093,
"timestamps": {
"RUNNING": 1542082624264,
"CANCELLING": 0,
"FAILED": 1542083249151,
"FINISHED": 0,
"SUSPENDING": 0,
"CREATED": 1542082624238,
"SUSPENDED": 0,
"RESTARTING": 0,
"CANCELED": 0,
"RECONCILING": 0,
"FAILING": 1542083247152
},
"vertices": [
{
"id": "58c1d4699c3aef634832ff9b7ea3b704",
"name": "DataSource (at DecodeData/InferFeatureTypes/DoOnce/Impulse (org.apache.beam.runners.flink.translation.wrappers.ImpulseInputFormat))",
"parallelism": 1,
"status": "FINISHED",
"start-time": 1542082624293,
"end-time": 1542082624891,
"duration": 598,
"tasks": {
"SCHEDULED": 0,
"RUNNING": 0,
"CANCELING": 0,
"CANCELED": 0,
"FAILED": 0,
"CREATED": 0,
"DEPLOYING": 0,
"RECONCILING": 0,
"FINISHED": 1
},
"metrics": {
"read-bytes": 0,
"read-bytes-complete": true,
"write-bytes": 0,
"write-bytes-complete": true,
"read-records": 0,
"read-records-complete": true,
"write-records": 1,
"write-records-complete": true
}
},
{
"id": "d4ffa588405cb2c615c35edbb049faee",
"name": "DataSource (at GenerateStatistics/RunStatsGenerators/MergeDatasetFeatureStatisticsProtos/DoOnce/Impulse (org.apache.beam.runners.flink.translation.wrappers.Impuls)",
"parallelism": 1,
"status": "FINISHED",
"start-time": 1542082624292,
"end-time": 1542082624890,
"duration": 598,
"tasks": {
"SCHEDULED": 0,
"RUNNING": 0,
"CANCELING": 0,
"CANCELED": 0,
"FAILED": 0,
"CREATED": 0,
"DEPLOYING": 0,
"RECONCILING": 0,
"FINISHED": 1
},
"metrics": {
"read-bytes": 0,
"read-bytes-complete": true,
"write-bytes": 0,
"write-bytes-complete": true,
"read-records": 0,
"read-records-complete": true,
"write-records": 1,
"write-records-complete": true
}
},
{
"id": "2654a83491f298762b2651817b7a3fca",
"name": "DataSource (at ReadData/Read/Impulse (org.apache.beam.runners.flink.translation.wrappers.ImpulseInputFormat))",
"parallelism": 1,
"status": "FINISHED",
"start-time": 1542082624292,
"end-time": 1542082624889,
"duration": 597,
"tasks": {
"SCHEDULED": 0,
"RUNNING": 0,
"CANCELING": 0,
"CANCELED": 0,
"FAILED": 0,
"CREATED": 0,
"DEPLOYING": 0,
"RECONCILING": 0,
"FINISHED": 1
},
"metrics": {
"read-bytes": 0,
"read-bytes-complete": true,
"write-bytes": 0,
"write-bytes-complete": true,
"read-records": 0,
"read-records-complete": true,
"write-records": 1,
"write-records-complete": true
}
},
{
"id": "e1d5f5310bd2666573d2b7c2faf170cb",
"name": "DataSource (at GenerateStatistics/RunStatsGenerators/NumericStatsGenerator/DoOnce/Impulse (org.apache.beam.runners.flink.translation.wrappers.ImpulseInputFormat))",
"parallelism": 1,
"status": "FINISHED",
"start-time": 1542082624292,
"end-time": 1542082624888,
"duration": 596,
"tasks": {
"SCHEDULED": 0,
"RUNNING": 0,
"CANCELING": 0,
"CANCELED": 0,
"FAILED": 0,
"CREATED": 0,
"DEPLOYING": 0,
"RECONCILING": 0,
"FINISHED": 1
},
"metrics": {
"read-bytes": 0,
"read-bytes-complete": true,
"write-bytes": 0,
"write-bytes-complete": true,
"read-records": 0,
"read-records-complete": true,
"write-records": 1,
"write-records-complete": true
}
},
{
"id": "f8111ccb6bd1714295230f2f26e70526",
"name": "DataSource (at GenerateStatistics/RunStatsGenerators/CommonStatsGenerator/DoOnce/Impulse (org.apache.beam.runners.flink.translation.wrappers.ImpulseInputFormat))",
"parallelism": 1,
"status": "FINISHED",
"start-time": 1542082624292,
"end-time": 1542082624884,
"duration": 592,
"tasks": {
"SCHEDULED": 0,
"RUNNING": 0,
"CANCELING": 0,
"CANCELED": 0,
"FAILED": 0,
"CREATED": 0,
"DEPLOYING": 0,
"RECONCILING": 0,
"FINISHED": 1
},
"metrics": {
"read-bytes": 0,
"read-bytes-complete": true,
"write-bytes": 0,
"write-bytes-complete": true,
"read-records": 0,
"read-records-complete": true,
"write-records": 1,
"write-records-complete": true
}
},
{
"id": "e2e9488c10ee82bb3299dff6498221d6",
"name": "DataSource (at GenerateStatistics/RunStatsGenerators/StringStatsGenerator/DoOnce/Impulse (org.apache.beam.runners.flink.translation.wrappers.ImpulseInputFormat))",
"parallelism": 1,
"status": "FINISHED",
"start-time": 1542082624292,
"end-time": 1542082624883,
"duration": 591,
"tasks": {
"SCHEDULED": 0,
"RUNNING": 0,
"CANCELING": 0,
"CANCELED": 0,
"FAILED": 0,
"CREATED": 0,
"DEPLOYING": 0,
"RECONCILING": 0,
"FINISHED": 1
},
"metrics": {
"read-bytes": 0,
"read-bytes-complete": true,
"write-bytes": 0,
"write-bytes-complete": true,
"read-records": 0,
"read-records-complete": true,
"write-records": 1,
"write-records-complete": true
}
},
{
"id": "b81b935c3dbeb33defbe2a8cc43faf2a",
"name": "DataSource (at WriteStatsOutput/Write/WriteImpl/DoOnce/Impulse (org.apache.beam.runners.flink.translation.wrappers.ImpulseInputFormat))",
"parallelism": 1,
"status": "FINISHED",
"start-time": 1542082624292,
"end-time": 1542082624875,
"duration": 583,
"tasks": {
"SCHEDULED": 0,
"RUNNING": 0,
"CANCELING": 0,
"CANCELED": 0,
"FAILED": 0,
"CREATED": 0,
"DEPLOYING": 0,
"RECONCILING": 0,
"FINISHED": 1
},
"metrics": {
"read-bytes": 0,
"read-bytes-complete": true,
"write-bytes": 0,
"write-bytes-complete": true,
"read-records": 0,
"read-records-complete": true,
"write-records": 1,
"write-records-complete": true
}
},
{
"id": "298e41af9e9ebc1b922447d1d6473eaf",
"name": "CHAIN MapPartition (MapPartition at 43DecodeData/InferFeatureTypes/DoOnce/Impulse.None/beam:env:docker:v1:0) -> FlatMap (FlatMap at 43DecodeData/InferFeatureTypes/DoOnce/Impulse.None/beam:env:docker:v1:0/out.0)",
"parallelism": 1,
"status": "FINISHED",
"start-time": 1542082624868,
"end-time": 1542082631331,
"duration": 6463,
"tasks": {
"SCHEDULED": 0,
"RUNNING": 0,
"CANCELING": 0,
"CANCELED": 0,
"FAILED": 0,
"CREATED": 0,
"DEPLOYING": 0,
"RECONCILING": 0,
"FINISHED": 1
},
"metrics": {
"read-bytes": 22,
"read-bytes-complete": true,
"write-bytes": 0,
"write-bytes-complete": true,
"read-records": 1,
"read-records-complete": true,
"write-records": 1,
"write-records-complete": true
}
},
{
"id": "d188f20c85ac7be031abc25e1fb5b148",
"name": "CHAIN MapPartition (MapPartition at 88GenerateStatistics/RunStatsGenerators/MergeDatasetFeatureStatisticsProtos/DoOnce/Impulse.None/beam:env:docker:v1:0) -> FlatMap (FlatMap at 88GenerateStatistics/RunStatsGenerators/MergeDatasetFeatureStatisticsProtos/DoOnce/Impulse.None/beam:env:docker:v1:0/out.0)",
"parallelism": 1,
"status": "FINISHED",
"start-time": 1542082624868,
"end-time": 1542082631366,
"duration": 6498,
"tasks": {
"SCHEDULED": 0,
"RUNNING": 0,
"CANCELING": 0,
"CANCELED": 0,
"FAILED": 0,
"CREATED": 0,
"DEPLOYING": 0,
"RECONCILING": 0,
"FINISHED": 1
},
"metrics": {
"read-bytes": 22,
"read-bytes-complete": true,
"write-bytes": 0,
"write-bytes-complete": true,
"read-records": 1,
"read-records-complete": true,
"write-records": 1,
"write-records-complete": true
}
},
{
"id": "cec5f17742db186f0123405e18da30b3",
"name": "CHAIN MapPartition (MapPartition at 21ReadData/Read/Impulse.None/beam:env:docker:v1:0) -> FlatMap (FlatMap at 21ReadData/Read/Impulse.None/beam:env:docker:v1:0/out.0) -> Map (Key Extractor) -> GroupCombine (GroupCombine at GroupCombine: ReadData/Read/Reshuffle/ReshufflePerKey/GroupByKey) -> Map (Key Extractor)",
"parallelism": 1,
"status": "FINISHED",
"start-time": 1542082624870,
"end-time": 1542082632467,
"duration": 7597,
"tasks": {
"SCHEDULED": 0,
"RUNNING": 0,
"CANCELING": 0,
"CANCELED": 0,
"FAILED": 0,
"CREATED": 0,
"DEPLOYING": 0,
"RECONCILING": 0,
"FINISHED": 1
},
"metrics": {
"read-bytes": 22,
"read-bytes-complete": true,
"write-bytes": 0,
"write-bytes-complete": true,
"read-records": 1,
"read-records-complete": true,
"write-records": 24,
"write-records-complete": true
}
},
{
"id": "f2b94c0ad43c028476beca14f5b03871",
"name": "GroupReduce (GroupReduce at ReadData/Read/Reshuffle/ReshufflePerKey/GroupByKey)",
"parallelism": 1,
"status": "FINISHED",
"start-time": 1542082632465,
"end-time": 1542082632500,
"duration": 35,
"tasks": {
"SCHEDULED": 0,
"RUNNING": 0,
"CANCELING": 0,
"CANCELED": 0,
"FAILED": 0,
"CREATED": 0,
"DEPLOYING": 0,
"RECONCILING": 0,
"FINISHED": 1
},
"metrics": {
"read-bytes": 10724,
"read-bytes-complete": true,
"write-bytes": 0,
"write-bytes-complete": true,
"read-records": 12,
"read-records-complete": true,
"write-records": 12,
"write-records-complete": true
}
},
{
"id": "c6485ed8e08800c2f8ae0ef7e6411255",
"name": "CHAIN MapPartition (MapPartition at 64ReadData/Read/Reshuffle/ReshufflePerKey/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0) -> FlatMap (FlatMap at 64ReadData/Read/Reshuffle/ReshufflePerKey/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0/out.0)",
"parallelism": 1,
"status": "FINISHED",
"start-time": 1542082632499,
"end-time": 1542082670922,
"duration": 38423,
"tasks": {
"SCHEDULED": 0,
"RUNNING": 0,
"CANCELING": 0,
"CANCELED": 0,
"FAILED": 0,
"CREATED": 0,
"DEPLOYING": 0,
"RECONCILING": 0,
"FINISHED": 1
},
"metrics": {
"read-bytes": 10539,
"read-bytes-complete": true,
"write-bytes": 364773376,
"write-bytes-complete": true,
"read-records": 12,
"read-records-complete": true,
"write-records": 1125972,
"write-records-complete": true
}
},
{
"id": "4983cfb36a4f03aebd68726bad8c05a5",
"name": "CHAIN MapPartition (MapPartition at 26DecodeData/ParseCSVRecords.None/beam:env:docker:v1:1) -> FlatMap (FlatMap at 26DecodeData/ParseCSVRecords.None/beam:env:docker:v1:1/out.0) -> Map (Key Extractor) -> GroupCombine (GroupCombine at GroupCombine: DecodeData/InferFeatureTypes/CombinePerKey/GroupByKey) -> Map (Key Extractor)",
"parallelism": 1,
"status": "FINISHED",
"start-time": 1542082670924,
"end-time": 1542082692698,
"duration": 21774,
"tasks": {
"SCHEDULED": 0,
"RUNNING": 0,
"CANCELING": 0,
"CANCELED": 0,
"FAILED": 0,
"CREATED": 0,
"DEPLOYING": 0,
"RECONCILING": 0,
"FINISHED": 1
},
"metrics": {
"read-bytes": 182394152,
"read-bytes-complete": true,
"write-bytes": 163250176,
"write-bytes-complete": true,
"read-records": 1125972,
"read-records-complete": true,
"write-records": 1125979,
"write-records-complete": true
}
},
{
"id": "ab40a7291893738a7c7ecae2bcdd07a9",
"name": "GroupReduce (GroupReduce at DecodeData/InferFeatureTypes/CombinePerKey/GroupByKey)",
"parallelism": 1,
"status": "FINISHED",
"start-time": 1542082675389,
"end-time": 1542082695000,
"duration": 19611,
"tasks": {
"SCHEDULED": 0,
"RUNNING": 0,
"CANCELING": 0,
"CANCELED": 0,
"FAILED": 0,
"CREATED": 0,
"DEPLOYING": 0,
"RECONCILING": 0,
"FINISHED": 1
},
"metrics": {
"read-bytes": 163252831,
"read-bytes-complete": true,
"write-bytes": 163250176,
"write-bytes-complete": true,
"read-records": 7,
"read-records-complete": true,
"write-records": 1,
"write-records-complete": true
}
},
{
"id": "bf9097c62f41786b45bd4d64d00ecf33",
"name": "CHAIN MapPartition (MapPartition at 67DecodeData/InferFeatureTypes/CombinePerKey/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0) -> FlatMap (FlatMap at 67DecodeData/InferFeatureTypes/CombinePerKey/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0/out.0)",
"parallelism": 1,
"status": "FINISHED",
"start-time": 1542082694812,
"end-time": 1542082734034,
"duration": 39222,
"tasks": {
"SCHEDULED": 0,
"RUNNING": 0,
"CANCELING": 0,
"CANCELED": 0,
"FAILED": 0,
"CREATED": 0,
"DEPLOYING": 0,
"RECONCILING": 0,
"FINISHED": 1
},
"metrics": {
"read-bytes": 163252651,
"read-bytes-complete": true,
"write-bytes": 0,
"write-bytes-complete": true,
"read-records": 1,
"read-records-complete": true,
"write-records": 1,
"write-records-complete": true
}
},
{
"id": "7f96f31248e59e2ef247f45568169783",
"name": "CHAIN MapPartition (MapPartition at 47DecodeData/InferFeatureTypes/DoOnce/Map(decode).None/beam:env:docker:v1:0) -> FlatMap (FlatMap at 47DecodeData/InferFeatureTypes/DoOnce/Map(decode).None/beam:env:docker:v1:0/out.0)",
"parallelism": 1,
"status": "FINISHED",
"start-time": 1542082631308,
"end-time": 1542082734132,
"duration": 102824,
"tasks": {
"SCHEDULED": 0,
"RUNNING": 0,
"CANCELING": 0,
"CANCELED": 0,
"FAILED": 0,
"CREATED": 0,
"DEPLOYING": 0,
"RECONCILING": 0,
"FINISHED": 1
},
"metrics": {
"read-bytes": 1658,
"read-bytes-complete": true,
"write-bytes": 0,
"write-bytes-complete": true,
"read-records": 1,
"read-records-complete": true,
"write-records": 1,
"write-records-complete": true
}
},
{
"id": "9e5d4b9b9aee5486a04c9c6fa18cb684",
"name": "MapPartition (MapPartition at 26DecodeData/ParseCSVRecords.None/beam:env:docker:v1:0)",
"parallelism": 1,
"status": "FINISHED",
"start-time": 1542082670924,
"end-time": 1542083241328,
"duration": 570404,
"tasks": {
"SCHEDULED": 0,
"RUNNING": 0,
"CANCELING": 0,
"CANCELED": 0,
"FAILED": 0,
"CREATED": 0,
"DEPLOYING": 0,
"RECONCILING": 0,
"FINISHED": 1
},
"metrics": {
"read-bytes": 182395787,
"read-bytes-complete": true,
"write-bytes": 11357388800,
"write-bytes-complete": true,
"read-records": 1125972,
"read-records-complete": true,
"write-records": 3713276,
"write-records-complete": true
}
},
{
"id": "16d3da7754788feb2d6b67fcfc3d9f40",
"name": "CHAIN FlatMap (FlatMap at 26DecodeData/ParseCSVRecords.None/beam:env:docker:v1:0/out.1) -> Map (Key Extractor) -> GroupCombine (GroupCombine at GroupCombine: GenerateStatistics/RunStatsGenerators/NumericStatsGenerator/CombinePerKey/GroupByKey) -> Map (Key Extractor)",
"parallelism": 1,
"status": "CANCELED",
"start-time": 1542083241330,
"end-time": 1542083248774,
"duration": 7444,
"tasks": {
"SCHEDULED": 0,
"RUNNING": 0,
"CANCELING": 0,
"CANCELED": 1,
"FAILED": 0,
"CREATED": 0,
"DEPLOYING": 0,
"RECONCILING": 0,
"FINISHED": 0
},
"metrics": {
"read-bytes": 390037504,
"read-bytes-complete": true,
"write-bytes": 99876864,
"write-bytes-complete": true,
"read-records": 502044,
"read-records-complete": true,
"write-records": 1095,
"write-records-complete": true
}
},
{
"id": "b66363d1cd8ed075c6e4526d72af9176",
"name": "GroupReduce (GroupReduce at GenerateStatistics/RunStatsGenerators/NumericStatsGenerator/CombinePerKey/GroupByKey)",
"parallelism": 1,
"status": "CANCELED",
"start-time": 1542083242335,
"end-time": 1542083248772,
"duration": 6437,
"tasks": {
"SCHEDULED": 0,
"RUNNING": 0,
"CANCELING": 0,
"CANCELED": 1,
"FAILED": 0,
"CREATED": 0,
"DEPLOYING": 0,
"RECONCILING": 0,
"FINISHED": 0
},
"metrics": {
"read-bytes": 90013696,
"read-bytes-complete": true,
"write-bytes": 0,
"write-bytes-complete": true,
"read-records": 0,
"read-records-complete": true,
"write-records": 0,
"write-records-complete": true
}
},
{
"id": "d17e147a7808e6fffb562f5ad608348c",
"name": "CHAIN MapPartition (MapPartition at 98GenerateStatistics/RunStatsGenerators/NumericStatsGenerator/CombinePerKey/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0) -> FlatMap (FlatMap at 98GenerateStatistics/RunStatsGenerators/NumericStatsGenerator/CombinePerKey/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0/out.0)",
"parallelism": 1,
"status": "CANCELED",
"start-time": -1,
"end-time": -1,
"duration": -1,
"tasks": {
"SCHEDULED": 0,
"RUNNING": 0,
"CANCELING": 0,
"CANCELED": 1,
"FAILED": 0,
"CREATED": 0,
"DEPLOYING": 0,
"RECONCILING": 0,
"FINISHED": 0
},
"metrics": {
"read-bytes": 0,
"read-bytes-complete": true,
"write-bytes": 0,
"write-bytes-complete": true,
"read-records": 0,
"read-records-complete": true,
"write-records": 0,
"write-records-complete": true
}
},
{
"id": "510b5ba27b288217d588462cdfdb1c89",
"name": "CHAIN FlatMap (FlatMap at 26DecodeData/ParseCSVRecords.None/beam:env:docker:v1:0/out.3) -> Map (Key Extractor) -> GroupCombine (GroupCombine at GroupCombine: GenerateStatistics/RunStatsGenerators/TopKStatsGenerator/TopK_CountFeatureNameValueTuple/CombinePerKey(CountCombineFn)/GroupByKey) -> Map (Key Extractor)",
"parallelism": 1,
"status": "CANCELED",
"start-time": 1542083241330,
"end-time": 1542083248782,
"duration": 7452,
"tasks": {
"SCHEDULED": 0,
"RUNNING": 0,
"CANCELING": 0,
"CANCELED": 1,
"FAILED": 0,
"CREATED": 0,
"DEPLOYING": 0,
"RECONCILING": 0,
"FINISHED": 0
},
"metrics": {
"read-bytes": 615284736,
"read-bytes-complete": true,
"write-bytes": 589824,
"write-bytes-complete": true,
"read-records": 875071,
"read-records-complete": true,
"write-records": 375219,
"write-records-complete": true
}
},
{
"id": "59b4d70dcb043f75e8a71e0f60368044",
"name": "GroupReduce (GroupReduce at GenerateStatistics/RunStatsGenerators/TopKStatsGenerator/TopK_CountFeatureNameValueTuple/CombinePerKey(CountCombineFn)/GroupByKey)",
"parallelism": 1,
"status": "CANCELED",
"start-time": -1,
"end-time": -1,
"duration": -1,
"tasks": {
"SCHEDULED": 0,
"RUNNING": 0,
"CANCELING": 0,
"CANCELED": 1,
"FAILED": 0,
"CREATED": 0,
"DEPLOYING": 0,
"RECONCILING": 0,
"FINISHED": 0
},
"metrics": {
"read-bytes": 0,
"read-bytes-complete": true,
"write-bytes": 0,
"write-bytes-complete": true,
"read-records": 0,
"read-records-complete": true,
"write-records": 0,
"write-records-complete": true
}
},
{
"id": "b16f8e2ec206664fb6b95c49f57a04c2",
"name": "CHAIN MapPartition (MapPartition at 143GenerateStatistics/RunStatsGenerators/TopKStatsGenerator/TopK_CountFeatureNameValueTuple/CombinePerKey(CountCombineFn)/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0) -> FlatMap (FlatMap at 143GenerateStatistics/RunStatsGenerators/TopKStatsGenerator/TopK_CountFeatureNameValueTuple/CombinePerKey(CountCombineFn)/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0/out.0) -> Map (Key Extractor) -> GroupCombine (GroupCombine at GroupCombine: GenerateStatistics/RunStatsGenerators/TopKStatsGenerator/TopK_GetTopK/CombinePerKey(TopCombineFn)/GroupByKey) -> Map (Key Extractor)",
"parallelism": 1,
"status": "CANCELED",
"start-time": -1,
"end-time": -1,
"duration": -1,
"tasks": {
"SCHEDULED": 0,
"RUNNING": 0,
"CANCELING": 0,
"CANCELED": 1,
"FAILED": 0,
"CREATED": 0,
"DEPLOYING": 0,
"RECONCILING": 0,
"FINISHED": 0
},
"metrics": {
"read-bytes": 0,
"read-bytes-complete": true,
"write-bytes": 0,
"write-bytes-complete": true,
"read-records": 0,
"read-records-complete": true,
"write-records": 0,
"write-records-complete": true
}
},
{
"id": "24ef3f0717eb591a4a46823c63b305c3",
"name": "GroupReduce (GroupReduce at GenerateStatistics/RunStatsGenerators/TopKStatsGenerator/TopK_GetTopK/CombinePerKey(TopCombineFn)/GroupByKey)",
"parallelism": 1,
"status": "CANCELED",
"start-time": -1,
"end-time": -1,
"duration": -1,
"tasks": {
"SCHEDULED": 0,
"RUNNING": 0,
"CANCELING": 0,
"CANCELED": 1,
"FAILED": 0,
"CREATED": 0,
"DEPLOYING": 0,
"RECONCILING": 0,
"FINISHED": 0
},
"metrics": {
"read-bytes": 0,
"read-bytes-complete": true,
"write-bytes": 0,
"write-bytes-complete": true,
"read-records": 0,
"read-records-complete": true,
"write-records": 0,
"write-records-complete": true
}
},
{
"id": "18fc0b71278713b5c0f18c4299931157",
"name": "CHAIN MapPartition (MapPartition at 122GenerateStatistics/RunStatsGenerators/TopKStatsGenerator/TopK_GetTopK/CombinePerKey(TopCombineFn)/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0) -> FlatMap (FlatMap at 122GenerateStatistics/RunStatsGenerators/TopKStatsGenerator/TopK_GetTopK/CombinePerKey(TopCombineFn)/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0/out.0)",
"parallelism": 1,
"status": "CANCELED",
"start-time": -1,
"end-time": -1,
"duration": -1,
"tasks": {
"SCHEDULED": 0,
"RUNNING": 0,
"CANCELING": 0,
"CANCELED": 1,
"FAILED": 0,
"CREATED": 0,
"DEPLOYING": 0,
"RECONCILING": 0,
"FINISHED": 0
},
"metrics": {
"read-bytes": 0,
"read-bytes-complete": true,
"write-bytes": 0,
"write-bytes-complete": true,
"read-records": 0,
"read-records-complete": true,
"write-records": 0,
"write-records-complete": true
}
},
{
"id": "68f880071c774596985fcd67729f0f1a",
"name": "CHAIN FlatMap (FlatMap at 26DecodeData/ParseCSVRecords.None/beam:env:docker:v1:0/out.2) -> Map (Key Extractor) -> GroupCombine (GroupCombine at GroupCombine: GenerateStatistics/RunStatsGenerators/StringStatsGenerator/CombinePerKey/GroupByKey) -> Map (Key Extractor)",
"parallelism": 1,
"status": "CANCELED",
"start-time": 1542083241330,
"end-time": 1542083248778,
"duration": 7448,
"tasks": {
"SCHEDULED": 0,
"RUNNING": 0,
"CANCELING": 0,
"CANCELED": 1,
"FAILED": 0,
"CREATED": 0,
"DEPLOYING": 0,
"RECONCILING": 0,
"FINISHED": 0
},
"metrics": {
"read-bytes": 400556032,
"read-bytes-complete": true,
"write-bytes": 120324096,
"write-bytes-complete": true,
"read-records": 502127,
"read-records-complete": true,
"write-records": 1095,
"write-records-complete": true
}
},
{
"id": "2481c440912b84ca49e8f8b72da1752d",
"name": "GroupReduce (GroupReduce at GenerateStatistics/RunStatsGenerators/StringStatsGenerator/CombinePerKey/GroupByKey)",
"parallelism": 1,
"status": "CANCELED",
"start-time": 1542083242364,
"end-time": 1542083248770,
"duration": 6406,
"tasks": {
"SCHEDULED": 0,
"RUNNING": 0,
"CANCELING": 0,
"CANCELED": 1,
"FAILED": 0,
"CREATED": 0,
"DEPLOYING": 0,
"RECONCILING": 0,
"FINISHED": 0
},
"metrics": {
"read-bytes": 90013696,
"read-bytes-complete": true,
"write-bytes": 0,
"write-bytes-complete": true,
"read-records": 0,
"read-records-complete": true,
"write-records": 0,
"write-records-complete": true
}
},
{
"id": "e8d11f0a437ebd15b951c95626e4e052",
"name": "CHAIN MapPartition (MapPartition at 97GenerateStatistics/RunStatsGenerators/StringStatsGenerator/CombinePerKey/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0) -> FlatMap (FlatMap at 97GenerateStatistics/RunStatsGenerators/StringStatsGenerator/CombinePerKey/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0/out.0)",
"parallelism": 1,
"status": "CANCELED",
"start-time": -1,
"end-time": -1,
"duration": -1,
"tasks": {
"SCHEDULED": 0,
"RUNNING": 0,
"CANCELING": 0,
"CANCELED": 1,
"FAILED": 0,
"CREATED": 0,
"DEPLOYING": 0,
"RECONCILING": 0,
"FINISHED": 0
},
"metrics": {
"read-bytes": 0,
"read-bytes-complete": true,
"write-bytes": 0,
"write-bytes-complete": true,
"read-records": 0,
"read-records-complete": true,
"write-records": 0,
"write-records-complete": true
}
},
{
"id": "b5521a56f6f1fa2bc3a06a975a873c99",
"name": "CHAIN FlatMap (FlatMap at 26DecodeData/ParseCSVRecords.None/beam:env:docker:v1:0/out.0) -> Map (Key Extractor) -> GroupCombine (GroupCombine at GroupCombine: GenerateStatistics/RunStatsGenerators/CommonStatsGenerator/CombinePerKey/GroupByKey) -> Map (Key Extractor)",
"parallelism": 1,
"status": "CANCELED",
"start-time": 1542083241329,
"end-time": 1542083248777,
"duration": 7448,
"tasks": {
"SCHEDULED": 0,
"RUNNING": 0,
"CANCELING": 0,
"CANCELED": 1,
"FAILED": 0,
"CREATED": 0,
"DEPLOYING": 0,
"RECONCILING": 0,
"FINISHED": 0
},
"metrics": {
"read-bytes": 583434240,
"read-bytes-complete": true,
"write-bytes": 180944896,
"write-bytes-complete": true,
"read-records": 505313,
"read-records-complete": true,
"write-records": 2265,
"write-records-complete": true
}
},
{
"id": "18daf2682316240371747d029e2aa07f",
"name": "GroupReduce (GroupReduce at GenerateStatistics/RunStatsGenerators/CommonStatsGenerator/CombinePerKey/GroupByKey)",
"parallelism": 1,
"status": "FAILED",
"start-time": 1542083241961,
"end-time": 1542083247152,
"duration": 5191,
"tasks": {
"SCHEDULED": 0,
"RUNNING": 0,
"CANCELING": 0,
"CANCELED": 0,
"FAILED": 1,
"CREATED": 0,
"DEPLOYING": 0,
"RECONCILING": 0,
"FINISHED": 0
},
"metrics": {
"read-bytes": 90013696,
"read-bytes-complete": true,
"write-bytes": 0,
"write-bytes-complete": true,
"read-records": 0,
"read-records-complete": true,
"write-records": 0,
"write-records-complete": true
}
},
{
"id": "ee2a7ee51779773ead459855cd6fdf9a",
"name": "CHAIN MapPartition (MapPartition at 97GenerateStatistics/RunStatsGenerators/CommonStatsGenerator/CombinePerKey/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0) -> FlatMap (FlatMap at 97GenerateStatistics/RunStatsGenerators/CommonStatsGenerator/CombinePerKey/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0/out.0)",
"parallelism": 1,
"status": "CANCELED",
"start-time": -1,
"end-time": -1,
"duration": -1,
"tasks": {
"SCHEDULED": 0,
"RUNNING": 0,
"CANCELING": 0,
"CANCELED": 1,
"FAILED": 0,
"CREATED": 0,
"DEPLOYING": 0,
"RECONCILING": 0,
"FINISHED": 0
},
"metrics": {
"read-bytes": 0,
"read-bytes-complete": true,
"write-bytes": 0,
"write-bytes-complete": true,
"read-records": 0,
"read-records-complete": true,
"write-records": 0,
"write-records-complete": true
}
},
{
"id": "18c48f3b69f7182a426812bf0a129a91",
"name": "CHAIN FlatMap (FlatMap at 26DecodeData/ParseCSVRecords.None/beam:env:docker:v1:0/out.4) -> Map (Key Extractor) -> GroupCombine (GroupCombine at GroupCombine: GenerateStatistics/RunStatsGenerators/UniquesStatsGenerator/Uniques_CountFeatureNameValueTuple/CombinePerKey(CountCombineFn)/GroupByKey) -> Map (Key Extractor)",
"parallelism": 1,
"status": "CANCELED",
"start-time": 1542083241330,
"end-time": 1542083248779,
"duration": 7449,
"tasks": {
"SCHEDULED": 0,
"RUNNING": 0,
"CANCELING": 0,
"CANCELED": 1,
"FAILED": 0,
"CREATED": 0,
"DEPLOYING": 0,
"RECONCILING": 0,
"FINISHED": 0
},
"metrics": {
"read-bytes": 603586560,
"read-bytes-complete": true,
"write-bytes": 753664,
"write-bytes-complete": true,
"read-records": 631916,
"read-records-complete": true,
"write-records": 375889,
"write-records-complete": true
}
},
{
"id": "988381bcd910a9dabb5670e82d9fde7f",
"name": "GroupReduce (GroupReduce at GenerateStatistics/RunStatsGenerators/UniquesStatsGenerator/Uniques_CountFeatureNameValueTuple/CombinePerKey(CountCombineFn)/GroupByKey)",
"parallelism": 1,
"status": "CANCELED",
"start-time": -1,
"end-time": -1,
"duration": -1,
"tasks": {
"SCHEDULED": 0,
"RUNNING": 0,
"CANCELING": 0,
"CANCELED": 1,
"FAILED": 0,
"CREATED": 0,
"DEPLOYING": 0,
"RECONCILING": 0,
"FINISHED": 0
},
"metrics": {
"read-bytes": 0,
"read-bytes-complete": true,
"write-bytes": 0,
"write-bytes-complete": true,
"read-records": 0,
"read-records-complete": true,
"write-records": 0,
"write-records-complete": true
}
},
{
"id": "6fbe81468dce31c8ca2853fdba8a8760",
"name": "CHAIN MapPartition (MapPartition at 149GenerateStatistics/RunStatsGenerators/UniquesStatsGenerator/Uniques_CountFeatureNameValueTuple/CombinePerKey(CountCombineFn)/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0) -> FlatMap (FlatMap at 149GenerateStatistics/RunStatsGenerators/UniquesStatsGenerator/Uniques_CountFeatureNameValueTuple/CombinePerKey(CountCombineFn)/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0/out.0) -> Map (Key Extractor) -> GroupCombine (GroupCombine at GroupCombine: GenerateStatistics/RunStatsGenerators/UniquesStatsGenerator/Uniques_CountPerFeatureName/CombinePerKey(CountCombineFn)/GroupByKey) -> Map (Key Extractor)",
"parallelism": 1,
"status": "CANCELED",
"start-time": -1,
"end-time": -1,
"duration": -1,
"tasks": {
"SCHEDULED": 0,
"RUNNING": 0,
"CANCELING": 0,
"CANCELED": 1,
"FAILED": 0,
"CREATED": 0,
"DEPLOYING": 0,
"RECONCILING": 0,
"FINISHED": 0
},
"metrics": {
"read-bytes": 0,
"read-bytes-complete": true,
"write-bytes": 0,
"write-bytes-complete": true,
"read-records": 0,
"read-records-complete": true,
"write-records": 0,
"write-records-complete": true
}
},
{
"id": "3fe989337a8ccda9c573d1410ead4dd0",
"name": "GroupReduce (GroupReduce at GenerateStatistics/RunStatsGenerators/UniquesStatsGenerator/Uniques_CountPerFeatureName/CombinePerKey(CountCombineFn)/GroupByKey)",
"parallelism": 1,
"status": "CANCELED",
"start-time": -1,
"end-time": -1,
"duration": -1,
"tasks": {
"SCHEDULED": 0,
"RUNNING": 0,
"CANCELING": 0,
"CANCELED": 1,
"FAILED": 0,
"CREATED": 0,
"DEPLOYING": 0,
"RECONCILING": 0,
"FINISHED": 0
},
"metrics": {
"read-bytes": 0,
"read-bytes-complete": true,
"write-bytes": 0,
"write-bytes-complete": true,
"read-records": 0,
"read-records-complete": true,
"write-records": 0,
"write-records-complete": true
}
},
{
"id": "88b81a3cae9eab85c0ea89ba0c01f1e5",
"name": "CHAIN MapPartition (MapPartition at 142GenerateStatistics/RunStatsGenerators/UniquesStatsGenerator/Uniques_CountPerFeatureName/CombinePerKey(CountCombineFn)/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0) -> FlatMap (FlatMap at 142GenerateStatistics/RunStatsGenerators/UniquesStatsGenerator/Uniques_CountPerFeatureName/CombinePerKey(CountCombineFn)/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0/out.0)",
"parallelism": 1,
"status": "CANCELED",
"start-time": -1,
"end-time": -1,
"duration": -1,
"tasks": {
"SCHEDULED": 0,
"RUNNING": 0,
"CANCELING": 0,
"CANCELED": 1,
"FAILED": 0,
"CREATED": 0,
"DEPLOYING": 0,
"RECONCILING": 0,
"FINISHED": 0
},
"metrics": {
"read-bytes": 0,
"read-bytes-complete": true,
"write-bytes": 0,
"write-bytes-complete": true,
"read-records": 0,
"read-records-complete": true,
"write-records": 0,
"write-records-complete": true
}
},
{
"id": "42dee93d9fa1826c4fe5b74bbc2b20be",
"name": "CHAIN MapPartition (MapPartition at 74GenerateStatistics/RunStatsGenerators/NumericStatsGenerator/DoOnce/Impulse.None/beam:env:docker:v1:0) -> FlatMap (FlatMap at 74GenerateStatistics/RunStatsGenerators/NumericStatsGenerator/DoOnce/Impulse.None/beam:env:docker:v1:0/out.0)",
"parallelism": 1,
"status": "FINISHED",
"start-time": 1542082624867,
"end-time": 1542082631350,
"duration": 6483,
"tasks": {
"SCHEDULED": 0,
"RUNNING": 0,
"CANCELING": 0,
"CANCELED": 0,
"FAILED": 0,
"CREATED": 0,
"DEPLOYING": 0,
"RECONCILING": 0,
"FINISHED": 1
},
"metrics": {
"read-bytes": 22,
"read-bytes-complete": true,
"write-bytes": 0,
"write-bytes-complete": true,
"read-records": 1,
"read-records-complete": true,
"write-records": 1,
"write-records-complete": true
}
},
{
"id": "e8a3cc8c502ce75b685d3e2927a1c97f",
"name": "CHAIN MapPartition (MapPartition at 78GenerateStatistics/RunStatsGenerators/NumericStatsGenerator/DoOnce/Map(decode).None/beam:env:docker:v1:0) -> FlatMap (FlatMap at 78GenerateStatistics/RunStatsGenerators/NumericStatsGenerator/DoOnce/Map(decode).None/beam:env:docker:v1:0/out.0)",
"parallelism": 1,
"status": "CANCELED",
"start-time": 1542082631346,
"end-time": 1542083248779,
"duration": 617433,
"tasks": {
"SCHEDULED": 0,
"RUNNING": 0,
"CANCELING": 0,
"CANCELED": 1,
"FAILED": 0,
"CREATED": 0,
"DEPLOYING": 0,
"RECONCILING": 0,
"FINISHED": 0
},
"metrics": {
"read-bytes": 0,
"read-bytes-complete": true,
"write-bytes": 0,
"write-bytes-complete": true,
"read-records": 0,
"read-records-complete": true,
"write-records": 0,
"write-records-complete": true
}
},
{
"id": "3f481475a4117a27391074a639a24c77",
"name": "CHAIN MapPartition (MapPartition at 73GenerateStatistics/RunStatsGenerators/CommonStatsGenerator/DoOnce/Impulse.None/beam:env:docker:v1:0) -> FlatMap (FlatMap at 73GenerateStatistics/RunStatsGenerators/CommonStatsGenerator/DoOnce/Impulse.None/beam:env:docker:v1:0/out.0)",
"parallelism": 1,
"status": "FINISHED",
"start-time": 1542082624866,
"end-time": 1542082631332,
"duration": 6466,
"tasks": {
"SCHEDULED": 0,
"RUNNING": 0,
"CANCELING": 0,
"CANCELED": 0,
"FAILED": 0,
"CREATED": 0,
"DEPLOYING": 0,
"RECONCILING": 0,
"FINISHED": 1
},
"metrics": {
"read-bytes": 22,
"read-bytes-complete": true,
"write-bytes": 0,
"write-bytes-complete": true,
"read-records": 1,
"read-records-complete": true,
"write-records": 1,
"write-records-complete": true
}
},
{
"id": "3bd851095315b142e35707a497509f11",
"name": "CHAIN MapPartition (MapPartition at 77GenerateStatistics/RunStatsGenerators/CommonStatsGenerator/DoOnce/Map(decode).None/beam:env:docker:v1:0) -> FlatMap (FlatMap at 77GenerateStatistics/RunStatsGenerators/CommonStatsGenerator/DoOnce/Map(decode).None/beam:env:docker:v1:0/out.0)",
"parallelism": 1,
"status": "CANCELED",
"start-time": 1542082631308,
"end-time": 1542083248780,
"duration": 617472,
"tasks": {
"SCHEDULED": 0,
"RUNNING": 0,
"CANCELING": 0,
"CANCELED": 1,
"FAILED": 0,
"CREATED": 0,
"DEPLOYING": 0,
"RECONCILING": 0,
"FINISHED": 0
},
"metrics": {
"read-bytes": 0,
"read-bytes-complete": true,
"write-bytes": 0,
"write-bytes-complete": true,
"read-records": 0,
"read-records-complete": true,
"write-records": 0,
"write-records-complete": true
}
},
{
"id": "35bad9de5c3986d1aa37eb321996f9e1",
"name": "CHAIN MapPartition (MapPartition at 73GenerateStatistics/RunStatsGenerators/StringStatsGenerator/DoOnce/Impulse.None/beam:env:docker:v1:0) -> FlatMap (FlatMap at 73GenerateStatistics/RunStatsGenerators/StringStatsGenerator/DoOnce/Impulse.None/beam:env:docker:v1:0/out.0)",
"parallelism": 1,
"status": "FINISHED",
"start-time": 1542082624866,
"end-time": 1542082631223,
"duration": 6357,
"tasks": {
"SCHEDULED": 0,
"RUNNING": 0,
"CANCELING": 0,
"CANCELED": 0,
"FAILED": 0,
"CREATED": 0,
"DEPLOYING": 0,
"RECONCILING": 0,
"FINISHED": 1
},
"metrics": {
"read-bytes": 22,
"read-bytes-complete": true,
"write-bytes": 0,
"write-bytes-complete": true,
"read-records": 1,
"read-records-complete": true,
"write-records": 1,
"write-records-complete": true
}
},
{
"id": "6db900e63ce3b00bbed15ce6268976b0",
"name": "CHAIN MapPartition (MapPartition at 77GenerateStatistics/RunStatsGenerators/StringStatsGenerator/DoOnce/Map(decode).None/beam:env:docker:v1:0) -> FlatMap (FlatMap at 77GenerateStatistics/RunStatsGenerators/StringStatsGenerator/DoOnce/Map(decode).None/beam:env:docker:v1:0/out.0)",
"parallelism": 1,
"status": "CANCELED",
"start-time": 1542082631208,
"end-time": 1542083248781,
"duration": 617573,
"tasks": {
"SCHEDULED": 0,
"RUNNING": 0,
"CANCELING": 0,
"CANCELED": 1,
"FAILED": 0,
"CREATED": 0,
"DEPLOYING": 0,
"RECONCILING": 0,
"FINISHED": 0
},
"metrics": {
"read-bytes": 0,
"read-bytes-complete": true,
"write-bytes": 0,
"write-bytes-complete": true,
"read-records": 0,
"read-records-complete": true,
"write-records": 0,
"write-records-complete": true
}
},
{
"id": "bb036e3688efb54b9875caf74fe7017b",
"name": "CHAIN Filter (UnionFixFilter) -> Map (Key Extractor) -> GroupCombine (GroupCombine at GroupCombine: GenerateStatistics/RunStatsGenerators/MergeDatasetFeatureStatisticsProtos/CombinePerKey/GroupByKey) -> Map (Key Extractor)",
"parallelism": 1,
"status": "CANCELED",
"start-time": -1,
"end-time": -1,
"duration": -1,
"tasks": {
"SCHEDULED": 0,
"RUNNING": 0,
"CANCELING": 0,
"CANCELED": 1,
"FAILED": 0,
"CREATED": 0,
"DEPLOYING": 0,
"RECONCILING": 0,
"FINISHED": 0
},
"metrics": {
"read-bytes": 0,
"read-bytes-complete": true,
"write-bytes": 0,
"write-bytes-complete": true,
"read-records": 0,
"read-records-complete": true,
"write-records": 0,
"write-records-complete": true
}
},
{
"id": "1e4111a2c49845717e0275467fc1f09f",
"name": "GroupReduce (GroupReduce at GenerateStatistics/RunStatsGenerators/MergeDatasetFeatureStatisticsProtos/CombinePerKey/GroupByKey)",
"parallelism": 1,
"status": "CANCELED",
"start-time": -1,
"end-time": -1,
"duration": -1,
"tasks": {
"SCHEDULED": 0,
"RUNNING": 0,
"CANCELING": 0,
"CANCELED": 1,
"FAILED": 0,
"CREATED": 0,
"DEPLOYING": 0,
"RECONCILING": 0,
"FINISHED": 0
},
"metrics": {
"read-bytes": 0,
"read-bytes-complete": true,
"write-bytes": 0,
"write-bytes-complete": true,
"read-records": 0,
"read-records-complete": true,
"write-records": 0,
"write-records-complete": true
}
},
{
"id": "f6ef3a5be1f89eca633122d825dde2e4",
"name": "CHAIN MapPartition (MapPartition at 112GenerateStatistics/RunStatsGenerators/MergeDatasetFeatureStatisticsProtos/CombinePerKey/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0) -> FlatMap (FlatMap at 112GenerateStatistics/RunStatsGenerators/MergeDatasetFeatureStatisticsProtos/CombinePerKey/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0/out.0)",
"parallelism": 1,
"status": "CANCELED",
"start-time": -1,
"end-time": -1,
"duration": -1,
"tasks": {
"SCHEDULED": 0,
"RUNNING": 0,
"CANCELING": 0,
"CANCELED": 1,
"FAILED": 0,
"CREATED": 0,
"DEPLOYING": 0,
"RECONCILING": 0,
"FINISHED": 0
},
"metrics": {
"read-bytes": 0,
"read-bytes-complete": true,
"write-bytes": 0,
"write-bytes-complete": true,
"read-records": 0,
"read-records-complete": true,
"write-records": 0,
"write-records-complete": true
}
},
{
"id": "6948037a86b77d33fed35777dad327c3",
"name": "CHAIN MapPartition (MapPartition at 92GenerateStatistics/RunStatsGenerators/MergeDatasetFeatureStatisticsProtos/DoOnce/Map(decode).None/beam:env:docker:v1:0) -> FlatMap (FlatMap at 92GenerateStatistics/RunStatsGenerators/MergeDatasetFeatureStatisticsProtos/DoOnce/Map(decode).None/beam:env:docker:v1:0/out.0) -> Map (Key Extractor) -> GroupCombine (GroupCombine at GroupCombine: WriteStatsOutput/Write/WriteImpl/GroupByKey) -> Map (Key Extractor)",
"parallelism": 1,
"status": "CANCELED",
"start-time": 1542082631354,
"end-time": 1542083248781,
"duration": 617427,
"tasks": {
"SCHEDULED": 0,
"RUNNING": 0,
"CANCELING": 0,
"CANCELED": 1,
"FAILED": 0,
"CREATED": 0,
"DEPLOYING": 0,
"RECONCILING": 0,
"FINISHED": 0
},
"metrics": {
"read-bytes": 0,
"read-bytes-complete": true,
"write-bytes": 0,
"write-bytes-complete": true,
"read-records": 0,
"read-records-complete": true,
"write-records": 0,
"write-records-complete": true
}
},
{
"id": "a633dac13fbf4c7fd8bd521edc7ace83",
"name": "GroupReduce (GroupReduce at WriteStatsOutput/Write/WriteImpl/GroupByKey)",
"parallelism": 1,
"status": "CANCELED",
"start-time": -1,
"end-time": -1,
"duration": -1,
"tasks": {
"SCHEDULED": 0,
"RUNNING": 0,
"CANCELING": 0,
"CANCELED": 1,
"FAILED": 0,
"CREATED": 0,
"DEPLOYING": 0,
"RECONCILING": 0,
"FINISHED": 0
},
"metrics": {
"read-bytes": 0,
"read-bytes-complete": true,
"write-bytes": 0,
"write-bytes-complete": true,
"read-records": 0,
"read-records-complete": true,
"write-records": 0,
"write-records-complete": true
}
},
{
"id": "766e63063a9b847725557fe8646ec382",
"name": "CHAIN MapPartition (MapPartition at 47WriteStatsOutput/Write/WriteImpl/DoOnce/Impulse.None/beam:env:docker:v1:0) -> FlatMap (FlatMap at 47WriteStatsOutput/Write/WriteImpl/DoOnce/Impulse.None/beam:env:docker:v1:0/out.0)",
"parallelism": 1,
"status": "FINISHED",
"start-time": 1542082624864,
"end-time": 1542082631303,
"duration": 6439,
"tasks": {
"SCHEDULED": 0,
"RUNNING": 0,
"CANCELING": 0,
"CANCELED": 0,
"FAILED": 0,
"CREATED": 0,
"DEPLOYING": 0,
"RECONCILING": 0,
"FINISHED": 1
},
"metrics": {
"read-bytes": 22,
"read-bytes-complete": true,
"write-bytes": 0,
"write-bytes-complete": true,
"read-records": 1,
"read-records-complete": true,
"write-records": 1,
"write-records-complete": true
}
},
{
"id": "12012f2c6a9d768ad5651fd2d01f4276",
"name": "CHAIN MapPartition (MapPartition at 51WriteStatsOutput/Write/WriteImpl/DoOnce/Map(decode).None/beam:env:docker:v1:0) -> FlatMap (FlatMap at 51WriteStatsOutput/Write/WriteImpl/DoOnce/Map(decode).None/beam:env:docker:v1:0/out.0)",
"parallelism": 1,
"status": "FINISHED",
"start-time": 1542082631304,
"end-time": 1542082631370,
"duration": 66,
"tasks": {
"SCHEDULED": 0,
"RUNNING": 0,
"CANCELING": 0,
"CANCELED": 0,
"FAILED": 0,
"CREATED": 0,
"DEPLOYING": 0,
"RECONCILING": 0,
"FINISHED": 1
},
"metrics": {
"read-bytes": 23,
"read-bytes-complete": true,
"write-bytes": 0,
"write-bytes-complete": true,
"read-records": 1,
"read-records-complete": true,
"write-records": 1,
"write-records-complete": true
}
},
{
"id": "03641b59a0998396c201212176db6d3f",
"name": "CHAIN MapPartition (MapPartition at 57WriteStatsOutput/Write/WriteImpl/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0) -> FlatMap (FlatMap at 57WriteStatsOutput/Write/WriteImpl/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0/out.0)",
"parallelism": 1,
"status": "CANCELED",
"start-time": 1542082631371,
"end-time": 1542083249150,
"duration": 617779,
"tasks": {
"SCHEDULED": 0,
"RUNNING": 0,
"CANCELING": 0,
"CANCELED": 1,
"FAILED": 0,
"CREATED": 0,
"DEPLOYING": 0,
"RECONCILING": 0,
"FINISHED": 0
},
"metrics": {
"read-bytes": 177,
"read-bytes-complete": true,
"write-bytes": 0,
"write-bytes-complete": true,
"read-records": 0,
"read-records-complete": true,
"write-records": 0,
"write-records-complete": true
}
},
{
"id": "55d331be8cfc0f9ab69860e8289ad9c9",
"name": "CHAIN MapPartition (MapPartition at 51WriteStatsOutput/Write/WriteImpl/DoOnce/Map(decode).None/beam:env:docker:v1:1) -> FlatMap (FlatMap at 51WriteStatsOutput/Write/WriteImpl/DoOnce/Map(decode).None/beam:env:docker:v1:1/out.0)",
"parallelism": 1,
"status": "CANCELED",
"start-time": 1542082631304,
"end-time": 1542083248782,
"duration": 617478,
"tasks": {
"SCHEDULED": 0,
"RUNNING": 0,
"CANCELING": 0,
"CANCELED": 1,
"FAILED": 0,
"CREATED": 0,
"DEPLOYING": 0,
"RECONCILING": 0,
"FINISHED": 0
},
"metrics": {
"read-bytes": 177,
"read-bytes-complete": true,
"write-bytes": 0,
"write-bytes-complete": true,
"read-records": 0,
"read-records-complete": true,
"write-records": 0,
"write-records-complete": true
}
},
{
"id": "1e2ed63c2e5714f106bcbed1130c3957",
"name": "MapPartition (MapPartition at 51WriteStatsOutput/Write/WriteImpl/DoOnce/Map(decode).None/beam:env:docker:v1:2)",
"parallelism": 1,
"status": "CANCELED",
"start-time": 1542082631304,
"end-time": 1542083248783,
"duration": 617479,
"tasks": {
"SCHEDULED": 0,
"RUNNING": 0,
"CANCELING": 0,
"CANCELED": 1,
"FAILED": 0,
"CREATED": 0,
"DEPLOYING": 0,
"RECONCILING": 0,
"FINISHED": 0
},
"metrics": {
"read-bytes": 177,
"read-bytes-complete": true,
"write-bytes": 0,
"write-bytes-complete": true,
"read-records": 0,
"read-records-complete": true,
"write-records": 0,
"write-records-complete": true
}
},
{
"id": "27ea7d6f7814e0c565930b8fd6577fba",
"name": "DataSink (org.apache.flink.api.java.io.DiscardingOutputFormat@32ec80e7)",
"parallelism": 1,
"status": "CANCELED",
"start-time": -1,
"end-time": -1,
"duration": -1,
"tasks": {
"SCHEDULED": 0,
"RUNNING": 0,
"CANCELING": 0,
"CANCELED": 1,
"FAILED": 0,
"CREATED": 0,
"DEPLOYING": 0,
"RECONCILING": 0,
"FINISHED": 0
},
"metrics": {
"read-bytes": 0,
"read-bytes-complete": true,
"write-bytes": 0,
"write-bytes-complete": true,
"read-records": 0,
"read-records-complete": true,
"write-records": 0,
"write-records-complete": true
}
}
],
"status-counts": {
"SCHEDULED": 0,
"RUNNING": 0,
"CANCELING": 0,
"CANCELED": 30,
"FAILED": 1,
"CREATED": 0,
"DEPLOYING": 0,
"RECONCILING": 0,
"FINISHED": 22
},
"plan": {
"jid": "cbbca35225bdde85aeeb7146ef19ce69",
"name": "BeamApp-goenka-1113041702-c9ef27fe",
"nodes": [
{
"id": "c6485ed8e08800c2f8ae0ef7e6411255",
"parallelism": 1,
"operator": "MapPartition -> FlatMap",
"operator_strategy": "Map Partition<br/> -> FlatMap",
"description": "MapPartition (MapPartition at 64ReadData/Read/Reshuffle/ReshufflePerKey/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0)<br/> -> FlatMap (FlatMap at 64ReadData/Read/Reshuffle/ReshufflePerKey/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0/out.0)",
"inputs": [
{
"num": 0,
"id": "f2b94c0ad43c028476beca14f5b03871",
"ship_strategy": "Forward",
"exchange": "pipelined"
}
],
"optimizer_properties": {
"global_properties": [
{
"name": "Partitioning",
"value": "RANDOM_PARTITIONED"
},
{
"name": "Partitioning Order",
"value": "(none)"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"local_properties": [
{
"name": "Order",
"value": "(none)"
},
{
"name": "Grouping",
"value": "not grouped"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"estimates": [
{
"name": "Est. Output Size",
"value": "(unknown)"
},
{
"name": "Est. Cardinality",
"value": "(unknown)"
}
],
"costs": [
{
"name": "Network",
"value": "0.0"
},
{
"name": "Disk I/O",
"value": "0.0"
},
{
"name": "CPU",
"value": "0.0"
},
{
"name": "Cumulative Network",
"value": "(unknown)"
},
{
"name": "Cumulative Disk I/O",
"value": "(unknown)"
},
{
"name": "Cumulative CPU",
"value": "(unknown)"
}
],
"compiler_hints": [
{
"name": "Output Size (bytes)",
"value": "(none)"
},
{
"name": "Output Cardinality",
"value": "(none)"
},
{
"name": "Avg. Output Record Size (bytes)",
"value": "(none)"
},
{
"name": "Filter Factor",
"value": "(none)"
}
]
}
},
{
"id": "e8a3cc8c502ce75b685d3e2927a1c97f",
"parallelism": 1,
"operator": "MapPartition -> FlatMap",
"operator_strategy": "Map Partition<br/> -> FlatMap",
"description": "MapPartition (MapPartition at 78GenerateStatistics/RunStatsGenerators/NumericStatsGenerator/DoOnce/Map(decode).None/beam:env:docker:v1:0)<br/> -> FlatMap (FlatMap at 78GenerateStatistics/RunStatsGenerators/NumericStatsGenerator/DoOnce/Map(decode).None/beam:env:docker:v1:0/out.0)",
"inputs": [
{
"num": 0,
"id": "42dee93d9fa1826c4fe5b74bbc2b20be",
"ship_strategy": "Forward",
"exchange": "pipelined"
},
{
"num": 1,
"id": "d17e147a7808e6fffb562f5ad608348c",
"ship_strategy": "Broadcast",
"exchange": "pipelined"
}
],
"optimizer_properties": {
"global_properties": [
{
"name": "Partitioning",
"value": "RANDOM_PARTITIONED"
},
{
"name": "Partitioning Order",
"value": "(none)"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"local_properties": [
{
"name": "Order",
"value": "(none)"
},
{
"name": "Grouping",
"value": "not grouped"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"estimates": [
{
"name": "Est. Output Size",
"value": "(unknown)"
},
{
"name": "Est. Cardinality",
"value": "(unknown)"
}
],
"costs": [
{
"name": "Network",
"value": "0.0"
},
{
"name": "Disk I/O",
"value": "0.0"
},
{
"name": "CPU",
"value": "0.0"
},
{
"name": "Cumulative Network",
"value": "(unknown)"
},
{
"name": "Cumulative Disk I/O",
"value": "(unknown)"
},
{
"name": "Cumulative CPU",
"value": "(unknown)"
}
],
"compiler_hints": [
{
"name": "Output Size (bytes)",
"value": "(none)"
},
{
"name": "Output Cardinality",
"value": "(none)"
},
{
"name": "Avg. Output Record Size (bytes)",
"value": "(none)"
},
{
"name": "Filter Factor",
"value": "(none)"
}
]
}
},
{
"id": "bb036e3688efb54b9875caf74fe7017b",
"parallelism": 1,
"operator": "Filter -> Map -> GroupCombine -> Map",
"operator_strategy": "FlatMap<br/> -> Map<br/> -> Sorted Combine<br/> -> Map",
"description": "Filter (UnionFixFilter)<br/> -> Map (Key Extractor)<br/> -> GroupCombine (GroupCombine at GroupCombine: GenerateStatistics/RunStatsGenerators/MergeDatasetFeatureStatisticsProtos/CombinePerKey/GroupByKey)<br/> -> Map (Key Extractor)",
"inputs": [
{
"num": 0,
"id": "e8a3cc8c502ce75b685d3e2927a1c97f",
"ship_strategy": "Forward",
"exchange": "pipelined"
},
{
"num": 1,
"id": "18fc0b71278713b5c0f18c4299931157",
"ship_strategy": "Forward",
"exchange": "pipelined"
},
{
"num": 2,
"id": "6db900e63ce3b00bbed15ce6268976b0",
"ship_strategy": "Forward",
"exchange": "pipelined"
},
{
"num": 3,
"id": "3bd851095315b142e35707a497509f11",
"ship_strategy": "Forward",
"exchange": "pipelined"
},
{
"num": 4,
"id": "88b81a3cae9eab85c0ea89ba0c01f1e5",
"ship_strategy": "Forward",
"exchange": "pipelined"
}
],
"optimizer_properties": {
"global_properties": [
{
"name": "Partitioning",
"value": "RANDOM_PARTITIONED"
},
{
"name": "Partitioning Order",
"value": "(none)"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"local_properties": [
{
"name": "Order",
"value": "(none)"
},
{
"name": "Grouping",
"value": "not grouped"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"estimates": [
{
"name": "Est. Output Size",
"value": "(unknown)"
},
{
"name": "Est. Cardinality",
"value": "(unknown)"
}
],
"costs": [
{
"name": "Network",
"value": "0.0"
},
{
"name": "Disk I/O",
"value": "0.0"
},
{
"name": "CPU",
"value": "0.0"
},
{
"name": "Cumulative Network",
"value": "(unknown)"
},
{
"name": "Cumulative Disk I/O",
"value": "(unknown)"
},
{
"name": "Cumulative CPU",
"value": "(unknown)"
}
],
"compiler_hints": [
{
"name": "Output Size (bytes)",
"value": "(none)"
},
{
"name": "Output Cardinality",
"value": "(none)"
},
{
"name": "Avg. Output Record Size (bytes)",
"value": "(none)"
},
{
"name": "Filter Factor",
"value": "(none)"
}
]
}
},
{
"id": "88b81a3cae9eab85c0ea89ba0c01f1e5",
"parallelism": 1,
"operator": "MapPartition -> FlatMap",
"operator_strategy": "Map Partition<br/> -> FlatMap",
"description": "MapPartition (MapPartition at 142GenerateStatistics/RunStatsGenerators/UniquesStatsGenerator/Uniques_CountPerFeatureName/CombinePerKey(CountCombineFn)/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0)<br/> -> FlatMap (FlatMap at 142GenerateStatistics/RunStatsGenerators/UniquesStatsGenerator/Uniques_CountPerFeatureName/CombinePerKey(CountCombineFn)/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0/out.0)",
"inputs": [
{
"num": 0,
"id": "3fe989337a8ccda9c573d1410ead4dd0",
"ship_strategy": "Forward",
"exchange": "pipelined"
}
],
"optimizer_properties": {
"global_properties": [
{
"name": "Partitioning",
"value": "RANDOM_PARTITIONED"
},
{
"name": "Partitioning Order",
"value": "(none)"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"local_properties": [
{
"name": "Order",
"value": "(none)"
},
{
"name": "Grouping",
"value": "not grouped"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"estimates": [
{
"name": "Est. Output Size",
"value": "(unknown)"
},
{
"name": "Est. Cardinality",
"value": "(unknown)"
}
],
"costs": [
{
"name": "Network",
"value": "0.0"
},
{
"name": "Disk I/O",
"value": "0.0"
},
{
"name": "CPU",
"value": "0.0"
},
{
"name": "Cumulative Network",
"value": "(unknown)"
},
{
"name": "Cumulative Disk I/O",
"value": "(unknown)"
},
{
"name": "Cumulative CPU",
"value": "(unknown)"
}
],
"compiler_hints": [
{
"name": "Output Size (bytes)",
"value": "(none)"
},
{
"name": "Output Cardinality",
"value": "(none)"
},
{
"name": "Avg. Output Record Size (bytes)",
"value": "(none)"
},
{
"name": "Filter Factor",
"value": "(none)"
}
]
}
},
{
"id": "58c1d4699c3aef634832ff9b7ea3b704",
"parallelism": 1,
"operator": "Data Source",
"operator_strategy": "(none)",
"description": "DataSource (at DecodeData/InferFeatureTypes/DoOnce/Impulse (org.apache.beam.runners.flink.translation.wrappers.ImpulseInputFormat))",
"optimizer_properties": {
"global_properties": [
{
"name": "Partitioning",
"value": "RANDOM_PARTITIONED"
},
{
"name": "Partitioning Order",
"value": "(none)"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"local_properties": [
{
"name": "Order",
"value": "(none)"
},
{
"name": "Grouping",
"value": "not grouped"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"estimates": [
{
"name": "Est. Output Size",
"value": "1.00"
},
{
"name": "Est. Cardinality",
"value": "1.00"
}
],
"costs": [
{
"name": "Network",
"value": "0.0"
},
{
"name": "Disk I/O",
"value": "0.0"
},
{
"name": "CPU",
"value": "0.0"
},
{
"name": "Cumulative Network",
"value": "0.0"
},
{
"name": "Cumulative Disk I/O",
"value": "0.0"
},
{
"name": "Cumulative CPU",
"value": "0.0"
}
],
"compiler_hints": [
{
"name": "Output Size (bytes)",
"value": "(none)"
},
{
"name": "Output Cardinality",
"value": "(none)"
},
{
"name": "Avg. Output Record Size (bytes)",
"value": "(none)"
},
{
"name": "Filter Factor",
"value": "(none)"
}
]
}
},
{
"id": "3f481475a4117a27391074a639a24c77",
"parallelism": 1,
"operator": "MapPartition -> FlatMap",
"operator_strategy": "Map Partition<br/> -> FlatMap",
"description": "MapPartition (MapPartition at 73GenerateStatistics/RunStatsGenerators/CommonStatsGenerator/DoOnce/Impulse.None/beam:env:docker:v1:0)<br/> -> FlatMap (FlatMap at 73GenerateStatistics/RunStatsGenerators/CommonStatsGenerator/DoOnce/Impulse.None/beam:env:docker:v1:0/out.0)",
"inputs": [
{
"num": 0,
"id": "f8111ccb6bd1714295230f2f26e70526",
"ship_strategy": "Forward",
"exchange": "pipelined"
}
],
"optimizer_properties": {
"global_properties": [
{
"name": "Partitioning",
"value": "RANDOM_PARTITIONED"
},
{
"name": "Partitioning Order",
"value": "(none)"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"local_properties": [
{
"name": "Order",
"value": "(none)"
},
{
"name": "Grouping",
"value": "not grouped"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"estimates": [
{
"name": "Est. Output Size",
"value": "(unknown)"
},
{
"name": "Est. Cardinality",
"value": "(unknown)"
}
],
"costs": [
{
"name": "Network",
"value": "0.0"
},
{
"name": "Disk I/O",
"value": "0.0"
},
{
"name": "CPU",
"value": "0.0"
},
{
"name": "Cumulative Network",
"value": "0.0"
},
{
"name": "Cumulative Disk I/O",
"value": "0.0"
},
{
"name": "Cumulative CPU",
"value": "0.0"
}
],
"compiler_hints": [
{
"name": "Output Size (bytes)",
"value": "(none)"
},
{
"name": "Output Cardinality",
"value": "(none)"
},
{
"name": "Avg. Output Record Size (bytes)",
"value": "(none)"
},
{
"name": "Filter Factor",
"value": "(none)"
}
]
}
},
{
"id": "55d331be8cfc0f9ab69860e8289ad9c9",
"parallelism": 1,
"operator": "MapPartition -> FlatMap",
"operator_strategy": "Map Partition<br/> -> FlatMap",
"description": "MapPartition (MapPartition at 51WriteStatsOutput/Write/WriteImpl/DoOnce/Map(decode).None/beam:env:docker:v1:1)<br/> -> FlatMap (FlatMap at 51WriteStatsOutput/Write/WriteImpl/DoOnce/Map(decode).None/beam:env:docker:v1:1/out.0)",
"inputs": [
{
"num": 0,
"id": "766e63063a9b847725557fe8646ec382",
"ship_strategy": "Forward",
"exchange": "blocking"
},
{
"num": 1,
"id": "12012f2c6a9d768ad5651fd2d01f4276",
"ship_strategy": "Broadcast",
"exchange": "blocking"
},
{
"num": 2,
"id": "03641b59a0998396c201212176db6d3f",
"ship_strategy": "Broadcast",
"exchange": "blocking"
}
],
"optimizer_properties": {
"global_properties": [
{
"name": "Partitioning",
"value": "RANDOM_PARTITIONED"
},
{
"name": "Partitioning Order",
"value": "(none)"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"local_properties": [
{
"name": "Order",
"value": "(none)"
},
{
"name": "Grouping",
"value": "not grouped"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"estimates": [
{
"name": "Est. Output Size",
"value": "(unknown)"
},
{
"name": "Est. Cardinality",
"value": "(unknown)"
}
],
"costs": [
{
"name": "Network",
"value": "0.0"
},
{
"name": "Disk I/O",
"value": "0.0"
},
{
"name": "CPU",
"value": "0.0"
},
{
"name": "Cumulative Network",
"value": "(unknown)"
},
{
"name": "Cumulative Disk I/O",
"value": "(unknown)"
},
{
"name": "Cumulative CPU",
"value": "(unknown)"
}
],
"compiler_hints": [
{
"name": "Output Size (bytes)",
"value": "(none)"
},
{
"name": "Output Cardinality",
"value": "(none)"
},
{
"name": "Avg. Output Record Size (bytes)",
"value": "(none)"
},
{
"name": "Filter Factor",
"value": "(none)"
}
]
}
},
{
"id": "d17e147a7808e6fffb562f5ad608348c",
"parallelism": 1,
"operator": "MapPartition -> FlatMap",
"operator_strategy": "Map Partition<br/> -> FlatMap",
"description": "MapPartition (MapPartition at 98GenerateStatistics/RunStatsGenerators/NumericStatsGenerator/CombinePerKey/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0)<br/> -> FlatMap (FlatMap at 98GenerateStatistics/RunStatsGenerators/NumericStatsGenerator/CombinePerKey/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0/out.0)",
"inputs": [
{
"num": 0,
"id": "b66363d1cd8ed075c6e4526d72af9176",
"ship_strategy": "Forward",
"exchange": "pipelined"
}
],
"optimizer_properties": {
"global_properties": [
{
"name": "Partitioning",
"value": "RANDOM_PARTITIONED"
},
{
"name": "Partitioning Order",
"value": "(none)"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"local_properties": [
{
"name": "Order",
"value": "(none)"
},
{
"name": "Grouping",
"value": "not grouped"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"estimates": [
{
"name": "Est. Output Size",
"value": "(unknown)"
},
{
"name": "Est. Cardinality",
"value": "(unknown)"
}
],
"costs": [
{
"name": "Network",
"value": "0.0"
},
{
"name": "Disk I/O",
"value": "0.0"
},
{
"name": "CPU",
"value": "0.0"
},
{
"name": "Cumulative Network",
"value": "(unknown)"
},
{
"name": "Cumulative Disk I/O",
"value": "(unknown)"
},
{
"name": "Cumulative CPU",
"value": "(unknown)"
}
],
"compiler_hints": [
{
"name": "Output Size (bytes)",
"value": "(none)"
},
{
"name": "Output Cardinality",
"value": "(none)"
},
{
"name": "Avg. Output Record Size (bytes)",
"value": "(none)"
},
{
"name": "Filter Factor",
"value": "(none)"
}
]
}
},
{
"id": "d4ffa588405cb2c615c35edbb049faee",
"parallelism": 1,
"operator": "Data Source",
"operator_strategy": "(none)",
"description": "DataSource (at GenerateStatistics/RunStatsGenerators/MergeDatasetFeatureStatisticsProtos/DoOnce/Impulse (org.apache.beam.runners.flink.translation.wrappers.Impuls)",
"optimizer_properties": {
"global_properties": [
{
"name": "Partitioning",
"value": "RANDOM_PARTITIONED"
},
{
"name": "Partitioning Order",
"value": "(none)"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"local_properties": [
{
"name": "Order",
"value": "(none)"
},
{
"name": "Grouping",
"value": "not grouped"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"estimates": [
{
"name": "Est. Output Size",
"value": "1.00"
},
{
"name": "Est. Cardinality",
"value": "1.00"
}
],
"costs": [
{
"name": "Network",
"value": "0.0"
},
{
"name": "Disk I/O",
"value": "0.0"
},
{
"name": "CPU",
"value": "0.0"
},
{
"name": "Cumulative Network",
"value": "0.0"
},
{
"name": "Cumulative Disk I/O",
"value": "0.0"
},
{
"name": "Cumulative CPU",
"value": "0.0"
}
],
"compiler_hints": [
{
"name": "Output Size (bytes)",
"value": "(none)"
},
{
"name": "Output Cardinality",
"value": "(none)"
},
{
"name": "Avg. Output Record Size (bytes)",
"value": "(none)"
},
{
"name": "Filter Factor",
"value": "(none)"
}
]
}
},
{
"id": "16d3da7754788feb2d6b67fcfc3d9f40",
"parallelism": 1,
"operator": "FlatMap -> Map -> GroupCombine -> Map",
"operator_strategy": "FlatMap<br/> -> Map<br/> -> Sorted Combine<br/> -> Map",
"description": "FlatMap (FlatMap at 26DecodeData/ParseCSVRecords.None/beam:env:docker:v1:0/out.1)<br/> -> Map (Key Extractor)<br/> -> GroupCombine (GroupCombine at GroupCombine: GenerateStatistics/RunStatsGenerators/NumericStatsGenerator/CombinePerKey/GroupByKey)<br/> -> Map (Key Extractor)",
"inputs": [
{
"num": 0,
"id": "9e5d4b9b9aee5486a04c9c6fa18cb684",
"ship_strategy": "Forward",
"exchange": "blocking"
}
],
"optimizer_properties": {
"global_properties": [
{
"name": "Partitioning",
"value": "RANDOM_PARTITIONED"
},
{
"name": "Partitioning Order",
"value": "(none)"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"local_properties": [
{
"name": "Order",
"value": "(none)"
},
{
"name": "Grouping",
"value": "not grouped"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"estimates": [
{
"name": "Est. Output Size",
"value": "(unknown)"
},
{
"name": "Est. Cardinality",
"value": "(unknown)"
}
],
"costs": [
{
"name": "Network",
"value": "0.0"
},
{
"name": "Disk I/O",
"value": "0.0"
},
{
"name": "CPU",
"value": "0.0"
},
{
"name": "Cumulative Network",
"value": "(unknown)"
},
{
"name": "Cumulative Disk I/O",
"value": "(unknown)"
},
{
"name": "Cumulative CPU",
"value": "(unknown)"
}
],
"compiler_hints": [
{
"name": "Output Size (bytes)",
"value": "(none)"
},
{
"name": "Output Cardinality",
"value": "(none)"
},
{
"name": "Avg. Output Record Size (bytes)",
"value": "(none)"
},
{
"name": "Filter Factor",
"value": "(none)"
}
]
}
},
{
"id": "a633dac13fbf4c7fd8bd521edc7ace83",
"parallelism": 1,
"operator": "GroupReduce",
"operator_strategy": "Sorted Group Reduce",
"description": "GroupReduce (GroupReduce at WriteStatsOutput/Write/WriteImpl/GroupByKey)",
"inputs": [
{
"num": 0,
"id": "6948037a86b77d33fed35777dad327c3",
"ship_strategy": "Hash Partition on [0]",
"local_strategy": "Sort on [0:ASC]",
"exchange": "pipelined"
}
],
"optimizer_properties": {
"global_properties": [
{
"name": "Partitioning",
"value": "RANDOM_PARTITIONED"
},
{
"name": "Partitioning Order",
"value": "(none)"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"local_properties": [
{
"name": "Order",
"value": "(none)"
},
{
"name": "Grouping",
"value": "not grouped"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"estimates": [
{
"name": "Est. Output Size",
"value": "(unknown)"
},
{
"name": "Est. Cardinality",
"value": "(unknown)"
}
],
"costs": [
{
"name": "Network",
"value": "(unknown)"
},
{
"name": "Disk I/O",
"value": "(unknown)"
},
{
"name": "CPU",
"value": "(unknown)"
},
{
"name": "Cumulative Network",
"value": "(unknown)"
},
{
"name": "Cumulative Disk I/O",
"value": "(unknown)"
},
{
"name": "Cumulative CPU",
"value": "(unknown)"
}
],
"compiler_hints": [
{
"name": "Output Size (bytes)",
"value": "(none)"
},
{
"name": "Output Cardinality",
"value": "(none)"
},
{
"name": "Avg. Output Record Size (bytes)",
"value": "(none)"
},
{
"name": "Filter Factor",
"value": "(none)"
}
]
}
},
{
"id": "b5521a56f6f1fa2bc3a06a975a873c99",
"parallelism": 1,
"operator": "FlatMap -> Map -> GroupCombine -> Map",
"operator_strategy": "FlatMap<br/> -> Map<br/> -> Sorted Combine<br/> -> Map",
"description": "FlatMap (FlatMap at 26DecodeData/ParseCSVRecords.None/beam:env:docker:v1:0/out.0)<br/> -> Map (Key Extractor)<br/> -> GroupCombine (GroupCombine at GroupCombine: GenerateStatistics/RunStatsGenerators/CommonStatsGenerator/CombinePerKey/GroupByKey)<br/> -> Map (Key Extractor)",
"inputs": [
{
"num": 0,
"id": "9e5d4b9b9aee5486a04c9c6fa18cb684",
"ship_strategy": "Forward",
"exchange": "blocking"
}
],
"optimizer_properties": {
"global_properties": [
{
"name": "Partitioning",
"value": "RANDOM_PARTITIONED"
},
{
"name": "Partitioning Order",
"value": "(none)"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"local_properties": [
{
"name": "Order",
"value": "(none)"
},
{
"name": "Grouping",
"value": "not grouped"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"estimates": [
{
"name": "Est. Output Size",
"value": "(unknown)"
},
{
"name": "Est. Cardinality",
"value": "(unknown)"
}
],
"costs": [
{
"name": "Network",
"value": "0.0"
},
{
"name": "Disk I/O",
"value": "0.0"
},
{
"name": "CPU",
"value": "0.0"
},
{
"name": "Cumulative Network",
"value": "(unknown)"
},
{
"name": "Cumulative Disk I/O",
"value": "(unknown)"
},
{
"name": "Cumulative CPU",
"value": "(unknown)"
}
],
"compiler_hints": [
{
"name": "Output Size (bytes)",
"value": "(none)"
},
{
"name": "Output Cardinality",
"value": "(none)"
},
{
"name": "Avg. Output Record Size (bytes)",
"value": "(none)"
},
{
"name": "Filter Factor",
"value": "(none)"
}
]
}
},
{
"id": "3fe989337a8ccda9c573d1410ead4dd0",
"parallelism": 1,
"operator": "GroupReduce",
"operator_strategy": "Sorted Group Reduce",
"description": "GroupReduce (GroupReduce at GenerateStatistics/RunStatsGenerators/UniquesStatsGenerator/Uniques_CountPerFeatureName/CombinePerKey(CountCombineFn)/GroupByKey)",
"inputs": [
{
"num": 0,
"id": "6fbe81468dce31c8ca2853fdba8a8760",
"ship_strategy": "Hash Partition on [0]",
"local_strategy": "Sort on [0:ASC]",
"exchange": "pipelined"
}
],
"optimizer_properties": {
"global_properties": [
{
"name": "Partitioning",
"value": "RANDOM_PARTITIONED"
},
{
"name": "Partitioning Order",
"value": "(none)"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"local_properties": [
{
"name": "Order",
"value": "(none)"
},
{
"name": "Grouping",
"value": "not grouped"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"estimates": [
{
"name": "Est. Output Size",
"value": "(unknown)"
},
{
"name": "Est. Cardinality",
"value": "(unknown)"
}
],
"costs": [
{
"name": "Network",
"value": "(unknown)"
},
{
"name": "Disk I/O",
"value": "(unknown)"
},
{
"name": "CPU",
"value": "(unknown)"
},
{
"name": "Cumulative Network",
"value": "(unknown)"
},
{
"name": "Cumulative Disk I/O",
"value": "(unknown)"
},
{
"name": "Cumulative CPU",
"value": "(unknown)"
}
],
"compiler_hints": [
{
"name": "Output Size (bytes)",
"value": "(none)"
},
{
"name": "Output Cardinality",
"value": "(none)"
},
{
"name": "Avg. Output Record Size (bytes)",
"value": "(none)"
},
{
"name": "Filter Factor",
"value": "(none)"
}
]
}
},
{
"id": "f2b94c0ad43c028476beca14f5b03871",
"parallelism": 1,
"operator": "GroupReduce",
"operator_strategy": "Sorted Group Reduce",
"description": "GroupReduce (GroupReduce at ReadData/Read/Reshuffle/ReshufflePerKey/GroupByKey)",
"inputs": [
{
"num": 0,
"id": "cec5f17742db186f0123405e18da30b3",
"ship_strategy": "Hash Partition on [0]",
"local_strategy": "Sort on [0:ASC]",
"exchange": "pipelined"
}
],
"optimizer_properties": {
"global_properties": [
{
"name": "Partitioning",
"value": "RANDOM_PARTITIONED"
},
{
"name": "Partitioning Order",
"value": "(none)"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"local_properties": [
{
"name": "Order",
"value": "(none)"
},
{
"name": "Grouping",
"value": "not grouped"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"estimates": [
{
"name": "Est. Output Size",
"value": "(unknown)"
},
{
"name": "Est. Cardinality",
"value": "(unknown)"
}
],
"costs": [
{
"name": "Network",
"value": "(unknown)"
},
{
"name": "Disk I/O",
"value": "(unknown)"
},
{
"name": "CPU",
"value": "(unknown)"
},
{
"name": "Cumulative Network",
"value": "(unknown)"
},
{
"name": "Cumulative Disk I/O",
"value": "(unknown)"
},
{
"name": "Cumulative CPU",
"value": "(unknown)"
}
],
"compiler_hints": [
{
"name": "Output Size (bytes)",
"value": "(none)"
},
{
"name": "Output Cardinality",
"value": "(none)"
},
{
"name": "Avg. Output Record Size (bytes)",
"value": "(none)"
},
{
"name": "Filter Factor",
"value": "(none)"
}
]
}
},
{
"id": "cec5f17742db186f0123405e18da30b3",
"parallelism": 1,
"operator": "MapPartition -> FlatMap -> Map -> GroupCombine -> Map",
"operator_strategy": "Map Partition<br/> -> FlatMap<br/> -> Map<br/> -> Sorted Combine<br/> -> Map",
"description": "MapPartition (MapPartition at 21ReadData/Read/Impulse.None/beam:env:docker:v1:0)<br/> -> FlatMap (FlatMap at 21ReadData/Read/Impulse.None/beam:env:docker:v1:0/out.0)<br/> -> Map (Key Extractor)<br/> -> GroupCombine (GroupCombine at GroupCombine: ReadData/Read/Reshuffle/ReshufflePerKey/GroupByKey)<br/> -> Map (Key Extractor)",
"inputs": [
{
"num": 0,
"id": "2654a83491f298762b2651817b7a3fca",
"ship_strategy": "Forward",
"exchange": "pipelined"
}
],
"optimizer_properties": {
"global_properties": [
{
"name": "Partitioning",
"value": "RANDOM_PARTITIONED"
},
{
"name": "Partitioning Order",
"value": "(none)"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"local_properties": [
{
"name": "Order",
"value": "(none)"
},
{
"name": "Grouping",
"value": "not grouped"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"estimates": [
{
"name": "Est. Output Size",
"value": "(unknown)"
},
{
"name": "Est. Cardinality",
"value": "(unknown)"
}
],
"costs": [
{
"name": "Network",
"value": "0.0"
},
{
"name": "Disk I/O",
"value": "0.0"
},
{
"name": "CPU",
"value": "0.0"
},
{
"name": "Cumulative Network",
"value": "0.0"
},
{
"name": "Cumulative Disk I/O",
"value": "0.0"
},
{
"name": "Cumulative CPU",
"value": "0.0"
}
],
"compiler_hints": [
{
"name": "Output Size (bytes)",
"value": "(none)"
},
{
"name": "Output Cardinality",
"value": "(none)"
},
{
"name": "Avg. Output Record Size (bytes)",
"value": "(none)"
},
{
"name": "Filter Factor",
"value": "(none)"
}
]
}
},
{
"id": "18c48f3b69f7182a426812bf0a129a91",
"parallelism": 1,
"operator": "FlatMap -> Map -> GroupCombine -> Map",
"operator_strategy": "FlatMap<br/> -> Map<br/> -> Sorted Combine<br/> -> Map",
"description": "FlatMap (FlatMap at 26DecodeData/ParseCSVRecords.None/beam:env:docker:v1:0/out.4)<br/> -> Map (Key Extractor)<br/> -> GroupCombine (GroupCombine at GroupCombine: GenerateStatistics/RunStatsGenerators/UniquesStatsGenerator/Uniques_CountFeatureNameValueTuple/CombinePerKey(CountCombineFn)/GroupByKey)<br/> -> Map (Key Extractor)",
"inputs": [
{
"num": 0,
"id": "9e5d4b9b9aee5486a04c9c6fa18cb684",
"ship_strategy": "Forward",
"exchange": "blocking"
}
],
"optimizer_properties": {
"global_properties": [
{
"name": "Partitioning",
"value": "RANDOM_PARTITIONED"
},
{
"name": "Partitioning Order",
"value": "(none)"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"local_properties": [
{
"name": "Order",
"value": "(none)"
},
{
"name": "Grouping",
"value": "not grouped"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"estimates": [
{
"name": "Est. Output Size",
"value": "(unknown)"
},
{
"name": "Est. Cardinality",
"value": "(unknown)"
}
],
"costs": [
{
"name": "Network",
"value": "0.0"
},
{
"name": "Disk I/O",
"value": "0.0"
},
{
"name": "CPU",
"value": "0.0"
},
{
"name": "Cumulative Network",
"value": "(unknown)"
},
{
"name": "Cumulative Disk I/O",
"value": "(unknown)"
},
{
"name": "Cumulative CPU",
"value": "(unknown)"
}
],
"compiler_hints": [
{
"name": "Output Size (bytes)",
"value": "(none)"
},
{
"name": "Output Cardinality",
"value": "(none)"
},
{
"name": "Avg. Output Record Size (bytes)",
"value": "(none)"
},
{
"name": "Filter Factor",
"value": "(none)"
}
]
}
},
{
"id": "42dee93d9fa1826c4fe5b74bbc2b20be",
"parallelism": 1,
"operator": "MapPartition -> FlatMap",
"operator_strategy": "Map Partition<br/> -> FlatMap",
"description": "MapPartition (MapPartition at 74GenerateStatistics/RunStatsGenerators/NumericStatsGenerator/DoOnce/Impulse.None/beam:env:docker:v1:0)<br/> -> FlatMap (FlatMap at 74GenerateStatistics/RunStatsGenerators/NumericStatsGenerator/DoOnce/Impulse.None/beam:env:docker:v1:0/out.0)",
"inputs": [
{
"num": 0,
"id": "e1d5f5310bd2666573d2b7c2faf170cb",
"ship_strategy": "Forward",
"exchange": "pipelined"
}
],
"optimizer_properties": {
"global_properties": [
{
"name": "Partitioning",
"value": "RANDOM_PARTITIONED"
},
{
"name": "Partitioning Order",
"value": "(none)"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"local_properties": [
{
"name": "Order",
"value": "(none)"
},
{
"name": "Grouping",
"value": "not grouped"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"estimates": [
{
"name": "Est. Output Size",
"value": "(unknown)"
},
{
"name": "Est. Cardinality",
"value": "(unknown)"
}
],
"costs": [
{
"name": "Network",
"value": "0.0"
},
{
"name": "Disk I/O",
"value": "0.0"
},
{
"name": "CPU",
"value": "0.0"
},
{
"name": "Cumulative Network",
"value": "0.0"
},
{
"name": "Cumulative Disk I/O",
"value": "0.0"
},
{
"name": "Cumulative CPU",
"value": "0.0"
}
],
"compiler_hints": [
{
"name": "Output Size (bytes)",
"value": "(none)"
},
{
"name": "Output Cardinality",
"value": "(none)"
},
{
"name": "Avg. Output Record Size (bytes)",
"value": "(none)"
},
{
"name": "Filter Factor",
"value": "(none)"
}
]
}
},
{
"id": "ee2a7ee51779773ead459855cd6fdf9a",
"parallelism": 1,
"operator": "MapPartition -> FlatMap",
"operator_strategy": "Map Partition<br/> -> FlatMap",
"description": "MapPartition (MapPartition at 97GenerateStatistics/RunStatsGenerators/CommonStatsGenerator/CombinePerKey/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0)<br/> -> FlatMap (FlatMap at 97GenerateStatistics/RunStatsGenerators/CommonStatsGenerator/CombinePerKey/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0/out.0)",
"inputs": [
{
"num": 0,
"id": "18daf2682316240371747d029e2aa07f",
"ship_strategy": "Forward",
"exchange": "pipelined"
}
],
"optimizer_properties": {
"global_properties": [
{
"name": "Partitioning",
"value": "RANDOM_PARTITIONED"
},
{
"name": "Partitioning Order",
"value": "(none)"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"local_properties": [
{
"name": "Order",
"value": "(none)"
},
{
"name": "Grouping",
"value": "not grouped"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"estimates": [
{
"name": "Est. Output Size",
"value": "(unknown)"
},
{
"name": "Est. Cardinality",
"value": "(unknown)"
}
],
"costs": [
{
"name": "Network",
"value": "0.0"
},
{
"name": "Disk I/O",
"value": "0.0"
},
{
"name": "CPU",
"value": "0.0"
},
{
"name": "Cumulative Network",
"value": "(unknown)"
},
{
"name": "Cumulative Disk I/O",
"value": "(unknown)"
},
{
"name": "Cumulative CPU",
"value": "(unknown)"
}
],
"compiler_hints": [
{
"name": "Output Size (bytes)",
"value": "(none)"
},
{
"name": "Output Cardinality",
"value": "(none)"
},
{
"name": "Avg. Output Record Size (bytes)",
"value": "(none)"
},
{
"name": "Filter Factor",
"value": "(none)"
}
]
}
},
{
"id": "6948037a86b77d33fed35777dad327c3",
"parallelism": 1,
"operator": "MapPartition -> FlatMap -> Map -> GroupCombine -> Map",
"operator_strategy": "Map Partition<br/> -> FlatMap<br/> -> Map<br/> -> Sorted Combine<br/> -> Map",
"description": "MapPartition (MapPartition at 92GenerateStatistics/RunStatsGenerators/MergeDatasetFeatureStatisticsProtos/DoOnce/Map(decode).None/beam:env:docker:v1:0)<br/> -> FlatMap (FlatMap at 92GenerateStatistics/RunStatsGenerators/MergeDatasetFeatureStatisticsProtos/DoOnce/Map(decode).None/beam:env:docker:v1:0/out.0)<br/> -> Map (Key Extractor)<br/> -> GroupCombine (GroupCombine at GroupCombine: WriteStatsOutput/Write/WriteImpl/GroupByKey)<br/> -> Map (Key Extractor)",
"inputs": [
{
"num": 0,
"id": "d188f20c85ac7be031abc25e1fb5b148",
"ship_strategy": "Forward",
"exchange": "pipelined"
},
{
"num": 1,
"id": "f6ef3a5be1f89eca633122d825dde2e4",
"ship_strategy": "Broadcast",
"exchange": "pipelined"
}
],
"optimizer_properties": {
"global_properties": [
{
"name": "Partitioning",
"value": "RANDOM_PARTITIONED"
},
{
"name": "Partitioning Order",
"value": "(none)"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"local_properties": [
{
"name": "Order",
"value": "(none)"
},
{
"name": "Grouping",
"value": "not grouped"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"estimates": [
{
"name": "Est. Output Size",
"value": "(unknown)"
},
{
"name": "Est. Cardinality",
"value": "(unknown)"
}
],
"costs": [
{
"name": "Network",
"value": "0.0"
},
{
"name": "Disk I/O",
"value": "0.0"
},
{
"name": "CPU",
"value": "0.0"
},
{
"name": "Cumulative Network",
"value": "(unknown)"
},
{
"name": "Cumulative Disk I/O",
"value": "(unknown)"
},
{
"name": "Cumulative CPU",
"value": "(unknown)"
}
],
"compiler_hints": [
{
"name": "Output Size (bytes)",
"value": "(none)"
},
{
"name": "Output Cardinality",
"value": "(none)"
},
{
"name": "Avg. Output Record Size (bytes)",
"value": "(none)"
},
{
"name": "Filter Factor",
"value": "(none)"
}
]
}
},
{
"id": "bf9097c62f41786b45bd4d64d00ecf33",
"parallelism": 1,
"operator": "MapPartition -> FlatMap",
"operator_strategy": "Map Partition<br/> -> FlatMap",
"description": "MapPartition (MapPartition at 67DecodeData/InferFeatureTypes/CombinePerKey/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0)<br/> -> FlatMap (FlatMap at 67DecodeData/InferFeatureTypes/CombinePerKey/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0/out.0)",
"inputs": [
{
"num": 0,
"id": "ab40a7291893738a7c7ecae2bcdd07a9",
"ship_strategy": "Forward",
"exchange": "pipelined"
}
],
"optimizer_properties": {
"global_properties": [
{
"name": "Partitioning",
"value": "RANDOM_PARTITIONED"
},
{
"name": "Partitioning Order",
"value": "(none)"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"local_properties": [
{
"name": "Order",
"value": "(none)"
},
{
"name": "Grouping",
"value": "not grouped"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"estimates": [
{
"name": "Est. Output Size",
"value": "(unknown)"
},
{
"name": "Est. Cardinality",
"value": "(unknown)"
}
],
"costs": [
{
"name": "Network",
"value": "0.0"
},
{
"name": "Disk I/O",
"value": "0.0"
},
{
"name": "CPU",
"value": "0.0"
},
{
"name": "Cumulative Network",
"value": "(unknown)"
},
{
"name": "Cumulative Disk I/O",
"value": "(unknown)"
},
{
"name": "Cumulative CPU",
"value": "(unknown)"
}
],
"compiler_hints": [
{
"name": "Output Size (bytes)",
"value": "(none)"
},
{
"name": "Output Cardinality",
"value": "(none)"
},
{
"name": "Avg. Output Record Size (bytes)",
"value": "(none)"
},
{
"name": "Filter Factor",
"value": "(none)"
}
]
}
},
{
"id": "18daf2682316240371747d029e2aa07f",
"parallelism": 1,
"operator": "GroupReduce",
"operator_strategy": "Sorted Group Reduce",
"description": "GroupReduce (GroupReduce at GenerateStatistics/RunStatsGenerators/CommonStatsGenerator/CombinePerKey/GroupByKey)",
"inputs": [
{
"num": 0,
"id": "b5521a56f6f1fa2bc3a06a975a873c99",
"ship_strategy": "Hash Partition on [0]",
"local_strategy": "Sort on [0:ASC]",
"exchange": "pipelined"
}
],
"optimizer_properties": {
"global_properties": [
{
"name": "Partitioning",
"value": "RANDOM_PARTITIONED"
},
{
"name": "Partitioning Order",
"value": "(none)"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"local_properties": [
{
"name": "Order",
"value": "(none)"
},
{
"name": "Grouping",
"value": "not grouped"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"estimates": [
{
"name": "Est. Output Size",
"value": "(unknown)"
},
{
"name": "Est. Cardinality",
"value": "(unknown)"
}
],
"costs": [
{
"name": "Network",
"value": "(unknown)"
},
{
"name": "Disk I/O",
"value": "(unknown)"
},
{
"name": "CPU",
"value": "(unknown)"
},
{
"name": "Cumulative Network",
"value": "(unknown)"
},
{
"name": "Cumulative Disk I/O",
"value": "(unknown)"
},
{
"name": "Cumulative CPU",
"value": "(unknown)"
}
],
"compiler_hints": [
{
"name": "Output Size (bytes)",
"value": "(none)"
},
{
"name": "Output Cardinality",
"value": "(none)"
},
{
"name": "Avg. Output Record Size (bytes)",
"value": "(none)"
},
{
"name": "Filter Factor",
"value": "(none)"
}
]
}
},
{
"id": "24ef3f0717eb591a4a46823c63b305c3",
"parallelism": 1,
"operator": "GroupReduce",
"operator_strategy": "Sorted Group Reduce",
"description": "GroupReduce (GroupReduce at GenerateStatistics/RunStatsGenerators/TopKStatsGenerator/TopK_GetTopK/CombinePerKey(TopCombineFn)/GroupByKey)",
"inputs": [
{
"num": 0,
"id": "b16f8e2ec206664fb6b95c49f57a04c2",
"ship_strategy": "Hash Partition on [0]",
"local_strategy": "Sort on [0:ASC]",
"exchange": "pipelined"
}
],
"optimizer_properties": {
"global_properties": [
{
"name": "Partitioning",
"value": "RANDOM_PARTITIONED"
},
{
"name": "Partitioning Order",
"value": "(none)"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"local_properties": [
{
"name": "Order",
"value": "(none)"
},
{
"name": "Grouping",
"value": "not grouped"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"estimates": [
{
"name": "Est. Output Size",
"value": "(unknown)"
},
{
"name": "Est. Cardinality",
"value": "(unknown)"
}
],
"costs": [
{
"name": "Network",
"value": "(unknown)"
},
{
"name": "Disk I/O",
"value": "(unknown)"
},
{
"name": "CPU",
"value": "(unknown)"
},
{
"name": "Cumulative Network",
"value": "(unknown)"
},
{
"name": "Cumulative Disk I/O",
"value": "(unknown)"
},
{
"name": "Cumulative CPU",
"value": "(unknown)"
}
],
"compiler_hints": [
{
"name": "Output Size (bytes)",
"value": "(none)"
},
{
"name": "Output Cardinality",
"value": "(none)"
},
{
"name": "Avg. Output Record Size (bytes)",
"value": "(none)"
},
{
"name": "Filter Factor",
"value": "(none)"
}
]
}
},
{
"id": "988381bcd910a9dabb5670e82d9fde7f",
"parallelism": 1,
"operator": "GroupReduce",
"operator_strategy": "Sorted Group Reduce",
"description": "GroupReduce (GroupReduce at GenerateStatistics/RunStatsGenerators/UniquesStatsGenerator/Uniques_CountFeatureNameValueTuple/CombinePerKey(CountCombineFn)/GroupByKey)",
"inputs": [
{
"num": 0,
"id": "18c48f3b69f7182a426812bf0a129a91",
"ship_strategy": "Hash Partition on [0]",
"local_strategy": "Sort on [0:ASC]",
"exchange": "pipelined"
}
],
"optimizer_properties": {
"global_properties": [
{
"name": "Partitioning",
"value": "RANDOM_PARTITIONED"
},
{
"name": "Partitioning Order",
"value": "(none)"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"local_properties": [
{
"name": "Order",
"value": "(none)"
},
{
"name": "Grouping",
"value": "not grouped"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"estimates": [
{
"name": "Est. Output Size",
"value": "(unknown)"
},
{
"name": "Est. Cardinality",
"value": "(unknown)"
}
],
"costs": [
{
"name": "Network",
"value": "(unknown)"
},
{
"name": "Disk I/O",
"value": "(unknown)"
},
{
"name": "CPU",
"value": "(unknown)"
},
{
"name": "Cumulative Network",
"value": "(unknown)"
},
{
"name": "Cumulative Disk I/O",
"value": "(unknown)"
},
{
"name": "Cumulative CPU",
"value": "(unknown)"
}
],
"compiler_hints": [
{
"name": "Output Size (bytes)",
"value": "(none)"
},
{
"name": "Output Cardinality",
"value": "(none)"
},
{
"name": "Avg. Output Record Size (bytes)",
"value": "(none)"
},
{
"name": "Filter Factor",
"value": "(none)"
}
]
}
},
{
"id": "766e63063a9b847725557fe8646ec382",
"parallelism": 1,
"operator": "MapPartition -> FlatMap",
"operator_strategy": "Map Partition<br/> -> FlatMap",
"description": "MapPartition (MapPartition at 47WriteStatsOutput/Write/WriteImpl/DoOnce/Impulse.None/beam:env:docker:v1:0)<br/> -> FlatMap (FlatMap at 47WriteStatsOutput/Write/WriteImpl/DoOnce/Impulse.None/beam:env:docker:v1:0/out.0)",
"inputs": [
{
"num": 0,
"id": "b81b935c3dbeb33defbe2a8cc43faf2a",
"ship_strategy": "Forward",
"exchange": "pipelined"
}
],
"optimizer_properties": {
"global_properties": [
{
"name": "Partitioning",
"value": "RANDOM_PARTITIONED"
},
{
"name": "Partitioning Order",
"value": "(none)"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"local_properties": [
{
"name": "Order",
"value": "(none)"
},
{
"name": "Grouping",
"value": "not grouped"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"estimates": [
{
"name": "Est. Output Size",
"value": "(unknown)"
},
{
"name": "Est. Cardinality",
"value": "(unknown)"
}
],
"costs": [
{
"name": "Network",
"value": "0.0"
},
{
"name": "Disk I/O",
"value": "0.0"
},
{
"name": "CPU",
"value": "0.0"
},
{
"name": "Cumulative Network",
"value": "0.0"
},
{
"name": "Cumulative Disk I/O",
"value": "0.0"
},
{
"name": "Cumulative CPU",
"value": "0.0"
}
],
"compiler_hints": [
{
"name": "Output Size (bytes)",
"value": "(none)"
},
{
"name": "Output Cardinality",
"value": "(none)"
},
{
"name": "Avg. Output Record Size (bytes)",
"value": "(none)"
},
{
"name": "Filter Factor",
"value": "(none)"
}
]
}
},
{
"id": "2654a83491f298762b2651817b7a3fca",
"parallelism": 1,
"operator": "Data Source",
"operator_strategy": "(none)",
"description": "DataSource (at ReadData/Read/Impulse (org.apache.beam.runners.flink.translation.wrappers.ImpulseInputFormat))",
"optimizer_properties": {
"global_properties": [
{
"name": "Partitioning",
"value": "RANDOM_PARTITIONED"
},
{
"name": "Partitioning Order",
"value": "(none)"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"local_properties": [
{
"name": "Order",
"value": "(none)"
},
{
"name": "Grouping",
"value": "not grouped"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"estimates": [
{
"name": "Est. Output Size",
"value": "1.00"
},
{
"name": "Est. Cardinality",
"value": "1.00"
}
],
"costs": [
{
"name": "Network",
"value": "0.0"
},
{
"name": "Disk I/O",
"value": "0.0"
},
{
"name": "CPU",
"value": "0.0"
},
{
"name": "Cumulative Network",
"value": "0.0"
},
{
"name": "Cumulative Disk I/O",
"value": "0.0"
},
{
"name": "Cumulative CPU",
"value": "0.0"
}
],
"compiler_hints": [
{
"name": "Output Size (bytes)",
"value": "(none)"
},
{
"name": "Output Cardinality",
"value": "(none)"
},
{
"name": "Avg. Output Record Size (bytes)",
"value": "(none)"
},
{
"name": "Filter Factor",
"value": "(none)"
}
]
}
},
{
"id": "298e41af9e9ebc1b922447d1d6473eaf",
"parallelism": 1,
"operator": "MapPartition -> FlatMap",
"operator_strategy": "Map Partition<br/> -> FlatMap",
"description": "MapPartition (MapPartition at 43DecodeData/InferFeatureTypes/DoOnce/Impulse.None/beam:env:docker:v1:0)<br/> -> FlatMap (FlatMap at 43DecodeData/InferFeatureTypes/DoOnce/Impulse.None/beam:env:docker:v1:0/out.0)",
"inputs": [
{
"num": 0,
"id": "58c1d4699c3aef634832ff9b7ea3b704",
"ship_strategy": "Forward",
"exchange": "pipelined"
}
],
"optimizer_properties": {
"global_properties": [
{
"name": "Partitioning",
"value": "RANDOM_PARTITIONED"
},
{
"name": "Partitioning Order",
"value": "(none)"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"local_properties": [
{
"name": "Order",
"value": "(none)"
},
{
"name": "Grouping",
"value": "not grouped"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"estimates": [
{
"name": "Est. Output Size",
"value": "(unknown)"
},
{
"name": "Est. Cardinality",
"value": "(unknown)"
}
],
"costs": [
{
"name": "Network",
"value": "0.0"
},
{
"name": "Disk I/O",
"value": "0.0"
},
{
"name": "CPU",
"value": "0.0"
},
{
"name": "Cumulative Network",
"value": "0.0"
},
{
"name": "Cumulative Disk I/O",
"value": "0.0"
},
{
"name": "Cumulative CPU",
"value": "0.0"
}
],
"compiler_hints": [
{
"name": "Output Size (bytes)",
"value": "(none)"
},
{
"name": "Output Cardinality",
"value": "(none)"
},
{
"name": "Avg. Output Record Size (bytes)",
"value": "(none)"
},
{
"name": "Filter Factor",
"value": "(none)"
}
]
}
},
{
"id": "4983cfb36a4f03aebd68726bad8c05a5",
"parallelism": 1,
"operator": "MapPartition -> FlatMap -> Map -> GroupCombine -> Map",
"operator_strategy": "Map Partition<br/> -> FlatMap<br/> -> Map<br/> -> Sorted Combine<br/> -> Map",
"description": "MapPartition (MapPartition at 26DecodeData/ParseCSVRecords.None/beam:env:docker:v1:1)<br/> -> FlatMap (FlatMap at 26DecodeData/ParseCSVRecords.None/beam:env:docker:v1:1/out.0)<br/> -> Map (Key Extractor)<br/> -> GroupCombine (GroupCombine at GroupCombine: DecodeData/InferFeatureTypes/CombinePerKey/GroupByKey)<br/> -> Map (Key Extractor)",
"inputs": [
{
"num": 0,
"id": "c6485ed8e08800c2f8ae0ef7e6411255",
"ship_strategy": "Forward",
"exchange": "blocking"
}
],
"optimizer_properties": {
"global_properties": [
{
"name": "Partitioning",
"value": "RANDOM_PARTITIONED"
},
{
"name": "Partitioning Order",
"value": "(none)"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"local_properties": [
{
"name": "Order",
"value": "(none)"
},
{
"name": "Grouping",
"value": "not grouped"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"estimates": [
{
"name": "Est. Output Size",
"value": "(unknown)"
},
{
"name": "Est. Cardinality",
"value": "(unknown)"
}
],
"costs": [
{
"name": "Network",
"value": "0.0"
},
{
"name": "Disk I/O",
"value": "0.0"
},
{
"name": "CPU",
"value": "0.0"
},
{
"name": "Cumulative Network",
"value": "(unknown)"
},
{
"name": "Cumulative Disk I/O",
"value": "(unknown)"
},
{
"name": "Cumulative CPU",
"value": "(unknown)"
}
],
"compiler_hints": [
{
"name": "Output Size (bytes)",
"value": "(none)"
},
{
"name": "Output Cardinality",
"value": "(none)"
},
{
"name": "Avg. Output Record Size (bytes)",
"value": "(none)"
},
{
"name": "Filter Factor",
"value": "(none)"
}
]
}
},
{
"id": "03641b59a0998396c201212176db6d3f",
"parallelism": 1,
"operator": "MapPartition -> FlatMap",
"operator_strategy": "Map Partition<br/> -> FlatMap",
"description": "MapPartition (MapPartition at 57WriteStatsOutput/Write/WriteImpl/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0)<br/> -> FlatMap (FlatMap at 57WriteStatsOutput/Write/WriteImpl/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0/out.0)",
"inputs": [
{
"num": 0,
"id": "a633dac13fbf4c7fd8bd521edc7ace83",
"ship_strategy": "Forward",
"exchange": "pipelined"
},
{
"num": 1,
"id": "12012f2c6a9d768ad5651fd2d01f4276",
"ship_strategy": "Broadcast",
"exchange": "blocking"
}
],
"optimizer_properties": {
"global_properties": [
{
"name": "Partitioning",
"value": "RANDOM_PARTITIONED"
},
{
"name": "Partitioning Order",
"value": "(none)"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"local_properties": [
{
"name": "Order",
"value": "(none)"
},
{
"name": "Grouping",
"value": "not grouped"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"estimates": [
{
"name": "Est. Output Size",
"value": "(unknown)"
},
{
"name": "Est. Cardinality",
"value": "(unknown)"
}
],
"costs": [
{
"name": "Network",
"value": "0.0"
},
{
"name": "Disk I/O",
"value": "0.0"
},
{
"name": "CPU",
"value": "0.0"
},
{
"name": "Cumulative Network",
"value": "(unknown)"
},
{
"name": "Cumulative Disk I/O",
"value": "(unknown)"
},
{
"name": "Cumulative CPU",
"value": "(unknown)"
}
],
"compiler_hints": [
{
"name": "Output Size (bytes)",
"value": "(none)"
},
{
"name": "Output Cardinality",
"value": "(none)"
},
{
"name": "Avg. Output Record Size (bytes)",
"value": "(none)"
},
{
"name": "Filter Factor",
"value": "(none)"
}
]
}
},
{
"id": "510b5ba27b288217d588462cdfdb1c89",
"parallelism": 1,
"operator": "FlatMap -> Map -> GroupCombine -> Map",
"operator_strategy": "FlatMap<br/> -> Map<br/> -> Sorted Combine<br/> -> Map",
"description": "FlatMap (FlatMap at 26DecodeData/ParseCSVRecords.None/beam:env:docker:v1:0/out.3)<br/> -> Map (Key Extractor)<br/> -> GroupCombine (GroupCombine at GroupCombine: GenerateStatistics/RunStatsGenerators/TopKStatsGenerator/TopK_CountFeatureNameValueTuple/CombinePerKey(CountCombineFn)/GroupByKey)<br/> -> Map (Key Extractor)",
"inputs": [
{
"num": 0,
"id": "9e5d4b9b9aee5486a04c9c6fa18cb684",
"ship_strategy": "Forward",
"exchange": "blocking"
}
],
"optimizer_properties": {
"global_properties": [
{
"name": "Partitioning",
"value": "RANDOM_PARTITIONED"
},
{
"name": "Partitioning Order",
"value": "(none)"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"local_properties": [
{
"name": "Order",
"value": "(none)"
},
{
"name": "Grouping",
"value": "not grouped"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"estimates": [
{
"name": "Est. Output Size",
"value": "(unknown)"
},
{
"name": "Est. Cardinality",
"value": "(unknown)"
}
],
"costs": [
{
"name": "Network",
"value": "0.0"
},
{
"name": "Disk I/O",
"value": "0.0"
},
{
"name": "CPU",
"value": "0.0"
},
{
"name": "Cumulative Network",
"value": "(unknown)"
},
{
"name": "Cumulative Disk I/O",
"value": "(unknown)"
},
{
"name": "Cumulative CPU",
"value": "(unknown)"
}
],
"compiler_hints": [
{
"name": "Output Size (bytes)",
"value": "(none)"
},
{
"name": "Output Cardinality",
"value": "(none)"
},
{
"name": "Avg. Output Record Size (bytes)",
"value": "(none)"
},
{
"name": "Filter Factor",
"value": "(none)"
}
]
}
},
{
"id": "e1d5f5310bd2666573d2b7c2faf170cb",
"parallelism": 1,
"operator": "Data Source",
"operator_strategy": "(none)",
"description": "DataSource (at GenerateStatistics/RunStatsGenerators/NumericStatsGenerator/DoOnce/Impulse (org.apache.beam.runners.flink.translation.wrappers.ImpulseInputFormat))",
"optimizer_properties": {
"global_properties": [
{
"name": "Partitioning",
"value": "RANDOM_PARTITIONED"
},
{
"name": "Partitioning Order",
"value": "(none)"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"local_properties": [
{
"name": "Order",
"value": "(none)"
},
{
"name": "Grouping",
"value": "not grouped"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"estimates": [
{
"name": "Est. Output Size",
"value": "1.00"
},
{
"name": "Est. Cardinality",
"value": "1.00"
}
],
"costs": [
{
"name": "Network",
"value": "0.0"
},
{
"name": "Disk I/O",
"value": "0.0"
},
{
"name": "CPU",
"value": "0.0"
},
{
"name": "Cumulative Network",
"value": "0.0"
},
{
"name": "Cumulative Disk I/O",
"value": "0.0"
},
{
"name": "Cumulative CPU",
"value": "0.0"
}
],
"compiler_hints": [
{
"name": "Output Size (bytes)",
"value": "(none)"
},
{
"name": "Output Cardinality",
"value": "(none)"
},
{
"name": "Avg. Output Record Size (bytes)",
"value": "(none)"
},
{
"name": "Filter Factor",
"value": "(none)"
}
]
}
},
{
"id": "18fc0b71278713b5c0f18c4299931157",
"parallelism": 1,
"operator": "MapPartition -> FlatMap",
"operator_strategy": "Map Partition<br/> -> FlatMap",
"description": "MapPartition (MapPartition at 122GenerateStatistics/RunStatsGenerators/TopKStatsGenerator/TopK_GetTopK/CombinePerKey(TopCombineFn)/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0)<br/> -> FlatMap (FlatMap at 122GenerateStatistics/RunStatsGenerators/TopKStatsGenerator/TopK_GetTopK/CombinePerKey(TopCombineFn)/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0/out.0)",
"inputs": [
{
"num": 0,
"id": "24ef3f0717eb591a4a46823c63b305c3",
"ship_strategy": "Forward",
"exchange": "pipelined"
}
],
"optimizer_properties": {
"global_properties": [
{
"name": "Partitioning",
"value": "RANDOM_PARTITIONED"
},
{
"name": "Partitioning Order",
"value": "(none)"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"local_properties": [
{
"name": "Order",
"value": "(none)"
},
{
"name": "Grouping",
"value": "not grouped"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"estimates": [
{
"name": "Est. Output Size",
"value": "(unknown)"
},
{
"name": "Est. Cardinality",
"value": "(unknown)"
}
],
"costs": [
{
"name": "Network",
"value": "0.0"
},
{
"name": "Disk I/O",
"value": "0.0"
},
{
"name": "CPU",
"value": "0.0"
},
{
"name": "Cumulative Network",
"value": "(unknown)"
},
{
"name": "Cumulative Disk I/O",
"value": "(unknown)"
},
{
"name": "Cumulative CPU",
"value": "(unknown)"
}
],
"compiler_hints": [
{
"name": "Output Size (bytes)",
"value": "(none)"
},
{
"name": "Output Cardinality",
"value": "(none)"
},
{
"name": "Avg. Output Record Size (bytes)",
"value": "(none)"
},
{
"name": "Filter Factor",
"value": "(none)"
}
]
}
},
{
"id": "9e5d4b9b9aee5486a04c9c6fa18cb684",
"parallelism": 1,
"operator": "MapPartition",
"operator_strategy": "Map Partition",
"description": "MapPartition (MapPartition at 26DecodeData/ParseCSVRecords.None/beam:env:docker:v1:0)",
"inputs": [
{
"num": 0,
"id": "c6485ed8e08800c2f8ae0ef7e6411255",
"ship_strategy": "Forward",
"exchange": "blocking"
},
{
"num": 1,
"id": "7f96f31248e59e2ef247f45568169783",
"ship_strategy": "Broadcast",
"exchange": "pipelined"
}
],
"optimizer_properties": {
"global_properties": [
{
"name": "Partitioning",
"value": "RANDOM_PARTITIONED"
},
{
"name": "Partitioning Order",
"value": "(none)"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"local_properties": [
{
"name": "Order",
"value": "(none)"
},
{
"name": "Grouping",
"value": "not grouped"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"estimates": [
{
"name": "Est. Output Size",
"value": "(unknown)"
},
{
"name": "Est. Cardinality",
"value": "(unknown)"
}
],
"costs": [
{
"name": "Network",
"value": "0.0"
},
{
"name": "Disk I/O",
"value": "0.0"
},
{
"name": "CPU",
"value": "0.0"
},
{
"name": "Cumulative Network",
"value": "(unknown)"
},
{
"name": "Cumulative Disk I/O",
"value": "(unknown)"
},
{
"name": "Cumulative CPU",
"value": "(unknown)"
}
],
"compiler_hints": [
{
"name": "Output Size (bytes)",
"value": "(none)"
},
{
"name": "Output Cardinality",
"value": "(none)"
},
{
"name": "Avg. Output Record Size (bytes)",
"value": "(none)"
},
{
"name": "Filter Factor",
"value": "(none)"
}
]
}
},
{
"id": "1e2ed63c2e5714f106bcbed1130c3957",
"parallelism": 1,
"operator": "MapPartition",
"operator_strategy": "Map Partition",
"description": "MapPartition (MapPartition at 51WriteStatsOutput/Write/WriteImpl/DoOnce/Map(decode).None/beam:env:docker:v1:2)",
"inputs": [
{
"num": 0,
"id": "766e63063a9b847725557fe8646ec382",
"ship_strategy": "Forward",
"exchange": "blocking"
},
{
"num": 1,
"id": "12012f2c6a9d768ad5651fd2d01f4276",
"ship_strategy": "Broadcast",
"exchange": "blocking"
},
{
"num": 2,
"id": "03641b59a0998396c201212176db6d3f",
"ship_strategy": "Broadcast",
"exchange": "blocking"
},
{
"num": 3,
"id": "55d331be8cfc0f9ab69860e8289ad9c9",
"ship_strategy": "Broadcast",
"exchange": "pipelined"
}
],
"optimizer_properties": {
"global_properties": [
{
"name": "Partitioning",
"value": "RANDOM_PARTITIONED"
},
{
"name": "Partitioning Order",
"value": "(none)"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"local_properties": [
{
"name": "Order",
"value": "(none)"
},
{
"name": "Grouping",
"value": "not grouped"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"estimates": [
{
"name": "Est. Output Size",
"value": "(unknown)"
},
{
"name": "Est. Cardinality",
"value": "(unknown)"
}
],
"costs": [
{
"name": "Network",
"value": "0.0"
},
{
"name": "Disk I/O",
"value": "0.0"
},
{
"name": "CPU",
"value": "0.0"
},
{
"name": "Cumulative Network",
"value": "(unknown)"
},
{
"name": "Cumulative Disk I/O",
"value": "(unknown)"
},
{
"name": "Cumulative CPU",
"value": "(unknown)"
}
],
"compiler_hints": [
{
"name": "Output Size (bytes)",
"value": "(none)"
},
{
"name": "Output Cardinality",
"value": "(none)"
},
{
"name": "Avg. Output Record Size (bytes)",
"value": "(none)"
},
{
"name": "Filter Factor",
"value": "(none)"
}
]
}
},
{
"id": "d188f20c85ac7be031abc25e1fb5b148",
"parallelism": 1,
"operator": "MapPartition -> FlatMap",
"operator_strategy": "Map Partition<br/> -> FlatMap",
"description": "MapPartition (MapPartition at 88GenerateStatistics/RunStatsGenerators/MergeDatasetFeatureStatisticsProtos/DoOnce/Impulse.None/beam:env:docker:v1:0)<br/> -> FlatMap (FlatMap at 88GenerateStatistics/RunStatsGenerators/MergeDatasetFeatureStatisticsProtos/DoOnce/Impulse.None/beam:env:docker:v1:0/out.0)",
"inputs": [
{
"num": 0,
"id": "d4ffa588405cb2c615c35edbb049faee",
"ship_strategy": "Forward",
"exchange": "pipelined"
}
],
"optimizer_properties": {
"global_properties": [
{
"name": "Partitioning",
"value": "RANDOM_PARTITIONED"
},
{
"name": "Partitioning Order",
"value": "(none)"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"local_properties": [
{
"name": "Order",
"value": "(none)"
},
{
"name": "Grouping",
"value": "not grouped"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"estimates": [
{
"name": "Est. Output Size",
"value": "(unknown)"
},
{
"name": "Est. Cardinality",
"value": "(unknown)"
}
],
"costs": [
{
"name": "Network",
"value": "0.0"
},
{
"name": "Disk I/O",
"value": "0.0"
},
{
"name": "CPU",
"value": "0.0"
},
{
"name": "Cumulative Network",
"value": "0.0"
},
{
"name": "Cumulative Disk I/O",
"value": "0.0"
},
{
"name": "Cumulative CPU",
"value": "0.0"
}
],
"compiler_hints": [
{
"name": "Output Size (bytes)",
"value": "(none)"
},
{
"name": "Output Cardinality",
"value": "(none)"
},
{
"name": "Avg. Output Record Size (bytes)",
"value": "(none)"
},
{
"name": "Filter Factor",
"value": "(none)"
}
]
}
},
{
"id": "7f96f31248e59e2ef247f45568169783",
"parallelism": 1,
"operator": "MapPartition -> FlatMap",
"operator_strategy": "Map Partition<br/> -> FlatMap",
"description": "MapPartition (MapPartition at 47DecodeData/InferFeatureTypes/DoOnce/Map(decode).None/beam:env:docker:v1:0)<br/> -> FlatMap (FlatMap at 47DecodeData/InferFeatureTypes/DoOnce/Map(decode).None/beam:env:docker:v1:0/out.0)",
"inputs": [
{
"num": 0,
"id": "298e41af9e9ebc1b922447d1d6473eaf",
"ship_strategy": "Forward",
"exchange": "pipelined"
},
{
"num": 1,
"id": "bf9097c62f41786b45bd4d64d00ecf33",
"ship_strategy": "Broadcast",
"exchange": "pipelined"
}
],
"optimizer_properties": {
"global_properties": [
{
"name": "Partitioning",
"value": "RANDOM_PARTITIONED"
},
{
"name": "Partitioning Order",
"value": "(none)"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"local_properties": [
{
"name": "Order",
"value": "(none)"
},
{
"name": "Grouping",
"value": "not grouped"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"estimates": [
{
"name": "Est. Output Size",
"value": "(unknown)"
},
{
"name": "Est. Cardinality",
"value": "(unknown)"
}
],
"costs": [
{
"name": "Network",
"value": "0.0"
},
{
"name": "Disk I/O",
"value": "0.0"
},
{
"name": "CPU",
"value": "0.0"
},
{
"name": "Cumulative Network",
"value": "(unknown)"
},
{
"name": "Cumulative Disk I/O",
"value": "(unknown)"
},
{
"name": "Cumulative CPU",
"value": "(unknown)"
}
],
"compiler_hints": [
{
"name": "Output Size (bytes)",
"value": "(none)"
},
{
"name": "Output Cardinality",
"value": "(none)"
},
{
"name": "Avg. Output Record Size (bytes)",
"value": "(none)"
},
{
"name": "Filter Factor",
"value": "(none)"
}
]
}
},
{
"id": "59b4d70dcb043f75e8a71e0f60368044",
"parallelism": 1,
"operator": "GroupReduce",
"operator_strategy": "Sorted Group Reduce",
"description": "GroupReduce (GroupReduce at GenerateStatistics/RunStatsGenerators/TopKStatsGenerator/TopK_CountFeatureNameValueTuple/CombinePerKey(CountCombineFn)/GroupByKey)",
"inputs": [
{
"num": 0,
"id": "510b5ba27b288217d588462cdfdb1c89",
"ship_strategy": "Hash Partition on [0]",
"local_strategy": "Sort on [0:ASC]",
"exchange": "pipelined"
}
],
"optimizer_properties": {
"global_properties": [
{
"name": "Partitioning",
"value": "RANDOM_PARTITIONED"
},
{
"name": "Partitioning Order",
"value": "(none)"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"local_properties": [
{
"name": "Order",
"value": "(none)"
},
{
"name": "Grouping",
"value": "not grouped"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"estimates": [
{
"name": "Est. Output Size",
"value": "(unknown)"
},
{
"name": "Est. Cardinality",
"value": "(unknown)"
}
],
"costs": [
{
"name": "Network",
"value": "(unknown)"
},
{
"name": "Disk I/O",
"value": "(unknown)"
},
{
"name": "CPU",
"value": "(unknown)"
},
{
"name": "Cumulative Network",
"value": "(unknown)"
},
{
"name": "Cumulative Disk I/O",
"value": "(unknown)"
},
{
"name": "Cumulative CPU",
"value": "(unknown)"
}
],
"compiler_hints": [
{
"name": "Output Size (bytes)",
"value": "(none)"
},
{
"name": "Output Cardinality",
"value": "(none)"
},
{
"name": "Avg. Output Record Size (bytes)",
"value": "(none)"
},
{
"name": "Filter Factor",
"value": "(none)"
}
]
}
},
{
"id": "6fbe81468dce31c8ca2853fdba8a8760",
"parallelism": 1,
"operator": "MapPartition -> FlatMap -> Map -> GroupCombine -> Map",
"operator_strategy": "Map Partition<br/> -> FlatMap<br/> -> Map<br/> -> Sorted Combine<br/> -> Map",
"description": "MapPartition (MapPartition at 149GenerateStatistics/RunStatsGenerators/UniquesStatsGenerator/Uniques_CountFeatureNameValueTuple/CombinePerKey(CountCombineFn)/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0)<br/> -> FlatMap (FlatMap at 149GenerateStatistics/RunStatsGenerators/UniquesStatsGenerator/Uniques_CountFeatureNameValueTuple/CombinePerKey(CountCombineFn)/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0/out.0)<br/> -> Map (Key Extractor)<br/> -> GroupCombine (GroupCombine at GroupCombine: GenerateStatistics/RunStatsGenerators/UniquesStatsGenerator/Uniques_CountPerFeatureName/CombinePerKey(CountCombineFn)/GroupByKey)<br/> -> Map (Key Extractor)",
"inputs": [
{
"num": 0,
"id": "988381bcd910a9dabb5670e82d9fde7f",
"ship_strategy": "Forward",
"exchange": "pipelined"
}
],
"optimizer_properties": {
"global_properties": [
{
"name": "Partitioning",
"value": "RANDOM_PARTITIONED"
},
{
"name": "Partitioning Order",
"value": "(none)"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"local_properties": [
{
"name": "Order",
"value": "(none)"
},
{
"name": "Grouping",
"value": "not grouped"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"estimates": [
{
"name": "Est. Output Size",
"value": "(unknown)"
},
{
"name": "Est. Cardinality",
"value": "(unknown)"
}
],
"costs": [
{
"name": "Network",
"value": "0.0"
},
{
"name": "Disk I/O",
"value": "0.0"
},
{
"name": "CPU",
"value": "0.0"
},
{
"name": "Cumulative Network",
"value": "(unknown)"
},
{
"name": "Cumulative Disk I/O",
"value": "(unknown)"
},
{
"name": "Cumulative CPU",
"value": "(unknown)"
}
],
"compiler_hints": [
{
"name": "Output Size (bytes)",
"value": "(none)"
},
{
"name": "Output Cardinality",
"value": "(none)"
},
{
"name": "Avg. Output Record Size (bytes)",
"value": "(none)"
},
{
"name": "Filter Factor",
"value": "(none)"
}
]
}
},
{
"id": "68f880071c774596985fcd67729f0f1a",
"parallelism": 1,
"operator": "FlatMap -> Map -> GroupCombine -> Map",
"operator_strategy": "FlatMap<br/> -> Map<br/> -> Sorted Combine<br/> -> Map",
"description": "FlatMap (FlatMap at 26DecodeData/ParseCSVRecords.None/beam:env:docker:v1:0/out.2)<br/> -> Map (Key Extractor)<br/> -> GroupCombine (GroupCombine at GroupCombine: GenerateStatistics/RunStatsGenerators/StringStatsGenerator/CombinePerKey/GroupByKey)<br/> -> Map (Key Extractor)",
"inputs": [
{
"num": 0,
"id": "9e5d4b9b9aee5486a04c9c6fa18cb684",
"ship_strategy": "Forward",
"exchange": "blocking"
}
],
"optimizer_properties": {
"global_properties": [
{
"name": "Partitioning",
"value": "RANDOM_PARTITIONED"
},
{
"name": "Partitioning Order",
"value": "(none)"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"local_properties": [
{
"name": "Order",
"value": "(none)"
},
{
"name": "Grouping",
"value": "not grouped"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"estimates": [
{
"name": "Est. Output Size",
"value": "(unknown)"
},
{
"name": "Est. Cardinality",
"value": "(unknown)"
}
],
"costs": [
{
"name": "Network",
"value": "0.0"
},
{
"name": "Disk I/O",
"value": "0.0"
},
{
"name": "CPU",
"value": "0.0"
},
{
"name": "Cumulative Network",
"value": "(unknown)"
},
{
"name": "Cumulative Disk I/O",
"value": "(unknown)"
},
{
"name": "Cumulative CPU",
"value": "(unknown)"
}
],
"compiler_hints": [
{
"name": "Output Size (bytes)",
"value": "(none)"
},
{
"name": "Output Cardinality",
"value": "(none)"
},
{
"name": "Avg. Output Record Size (bytes)",
"value": "(none)"
},
{
"name": "Filter Factor",
"value": "(none)"
}
]
}
},
{
"id": "f8111ccb6bd1714295230f2f26e70526",
"parallelism": 1,
"operator": "Data Source",
"operator_strategy": "(none)",
"description": "DataSource (at GenerateStatistics/RunStatsGenerators/CommonStatsGenerator/DoOnce/Impulse (org.apache.beam.runners.flink.translation.wrappers.ImpulseInputFormat))",
"optimizer_properties": {
"global_properties": [
{
"name": "Partitioning",
"value": "RANDOM_PARTITIONED"
},
{
"name": "Partitioning Order",
"value": "(none)"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"local_properties": [
{
"name": "Order",
"value": "(none)"
},
{
"name": "Grouping",
"value": "not grouped"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"estimates": [
{
"name": "Est. Output Size",
"value": "1.00"
},
{
"name": "Est. Cardinality",
"value": "1.00"
}
],
"costs": [
{
"name": "Network",
"value": "0.0"
},
{
"name": "Disk I/O",
"value": "0.0"
},
{
"name": "CPU",
"value": "0.0"
},
{
"name": "Cumulative Network",
"value": "0.0"
},
{
"name": "Cumulative Disk I/O",
"value": "0.0"
},
{
"name": "Cumulative CPU",
"value": "0.0"
}
],
"compiler_hints": [
{
"name": "Output Size (bytes)",
"value": "(none)"
},
{
"name": "Output Cardinality",
"value": "(none)"
},
{
"name": "Avg. Output Record Size (bytes)",
"value": "(none)"
},
{
"name": "Filter Factor",
"value": "(none)"
}
]
}
},
{
"id": "e2e9488c10ee82bb3299dff6498221d6",
"parallelism": 1,
"operator": "Data Source",
"operator_strategy": "(none)",
"description": "DataSource (at GenerateStatistics/RunStatsGenerators/StringStatsGenerator/DoOnce/Impulse (org.apache.beam.runners.flink.translation.wrappers.ImpulseInputFormat))",
"optimizer_properties": {
"global_properties": [
{
"name": "Partitioning",
"value": "RANDOM_PARTITIONED"
},
{
"name": "Partitioning Order",
"value": "(none)"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"local_properties": [
{
"name": "Order",
"value": "(none)"
},
{
"name": "Grouping",
"value": "not grouped"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"estimates": [
{
"name": "Est. Output Size",
"value": "1.00"
},
{
"name": "Est. Cardinality",
"value": "1.00"
}
],
"costs": [
{
"name": "Network",
"value": "0.0"
},
{
"name": "Disk I/O",
"value": "0.0"
},
{
"name": "CPU",
"value": "0.0"
},
{
"name": "Cumulative Network",
"value": "0.0"
},
{
"name": "Cumulative Disk I/O",
"value": "0.0"
},
{
"name": "Cumulative CPU",
"value": "0.0"
}
],
"compiler_hints": [
{
"name": "Output Size (bytes)",
"value": "(none)"
},
{
"name": "Output Cardinality",
"value": "(none)"
},
{
"name": "Avg. Output Record Size (bytes)",
"value": "(none)"
},
{
"name": "Filter Factor",
"value": "(none)"
}
]
}
},
{
"id": "b16f8e2ec206664fb6b95c49f57a04c2",
"parallelism": 1,
"operator": "MapPartition -> FlatMap -> Map -> GroupCombine -> Map",
"operator_strategy": "Map Partition<br/> -> FlatMap<br/> -> Map<br/> -> Sorted Combine<br/> -> Map",
"description": "MapPartition (MapPartition at 143GenerateStatistics/RunStatsGenerators/TopKStatsGenerator/TopK_CountFeatureNameValueTuple/CombinePerKey(CountCombineFn)/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0)<br/> -> FlatMap (FlatMap at 143GenerateStatistics/RunStatsGenerators/TopKStatsGenerator/TopK_CountFeatureNameValueTuple/CombinePerKey(CountCombineFn)/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0/out.0)<br/> -> Map (Key Extractor)<br/> -> GroupCombine (GroupCombine at GroupCombine: GenerateStatistics/RunStatsGenerators/TopKStatsGenerator/TopK_GetTopK/CombinePerKey(TopCombineFn)/GroupByKey)<br/> -> Map (Key Extractor)",
"inputs": [
{
"num": 0,
"id": "59b4d70dcb043f75e8a71e0f60368044",
"ship_strategy": "Forward",
"exchange": "pipelined"
}
],
"optimizer_properties": {
"global_properties": [
{
"name": "Partitioning",
"value": "RANDOM_PARTITIONED"
},
{
"name": "Partitioning Order",
"value": "(none)"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"local_properties": [
{
"name": "Order",
"value": "(none)"
},
{
"name": "Grouping",
"value": "not grouped"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"estimates": [
{
"name": "Est. Output Size",
"value": "(unknown)"
},
{
"name": "Est. Cardinality",
"value": "(unknown)"
}
],
"costs": [
{
"name": "Network",
"value": "0.0"
},
{
"name": "Disk I/O",
"value": "0.0"
},
{
"name": "CPU",
"value": "0.0"
},
{
"name": "Cumulative Network",
"value": "(unknown)"
},
{
"name": "Cumulative Disk I/O",
"value": "(unknown)"
},
{
"name": "Cumulative CPU",
"value": "(unknown)"
}
],
"compiler_hints": [
{
"name": "Output Size (bytes)",
"value": "(none)"
},
{
"name": "Output Cardinality",
"value": "(none)"
},
{
"name": "Avg. Output Record Size (bytes)",
"value": "(none)"
},
{
"name": "Filter Factor",
"value": "(none)"
}
]
}
},
{
"id": "1e4111a2c49845717e0275467fc1f09f",
"parallelism": 1,
"operator": "GroupReduce",
"operator_strategy": "Sorted Group Reduce",
"description": "GroupReduce (GroupReduce at GenerateStatistics/RunStatsGenerators/MergeDatasetFeatureStatisticsProtos/CombinePerKey/GroupByKey)",
"inputs": [
{
"num": 0,
"id": "bb036e3688efb54b9875caf74fe7017b",
"ship_strategy": "Hash Partition on [0]",
"local_strategy": "Sort on [0:ASC]",
"exchange": "pipelined"
}
],
"optimizer_properties": {
"global_properties": [
{
"name": "Partitioning",
"value": "RANDOM_PARTITIONED"
},
{
"name": "Partitioning Order",
"value": "(none)"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"local_properties": [
{
"name": "Order",
"value": "(none)"
},
{
"name": "Grouping",
"value": "not grouped"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"estimates": [
{
"name": "Est. Output Size",
"value": "(unknown)"
},
{
"name": "Est. Cardinality",
"value": "(unknown)"
}
],
"costs": [
{
"name": "Network",
"value": "(unknown)"
},
{
"name": "Disk I/O",
"value": "(unknown)"
},
{
"name": "CPU",
"value": "(unknown)"
},
{
"name": "Cumulative Network",
"value": "(unknown)"
},
{
"name": "Cumulative Disk I/O",
"value": "(unknown)"
},
{
"name": "Cumulative CPU",
"value": "(unknown)"
}
],
"compiler_hints": [
{
"name": "Output Size (bytes)",
"value": "(none)"
},
{
"name": "Output Cardinality",
"value": "(none)"
},
{
"name": "Avg. Output Record Size (bytes)",
"value": "(none)"
},
{
"name": "Filter Factor",
"value": "(none)"
}
]
}
},
{
"id": "b66363d1cd8ed075c6e4526d72af9176",
"parallelism": 1,
"operator": "GroupReduce",
"operator_strategy": "Sorted Group Reduce",
"description": "GroupReduce (GroupReduce at GenerateStatistics/RunStatsGenerators/NumericStatsGenerator/CombinePerKey/GroupByKey)",
"inputs": [
{
"num": 0,
"id": "16d3da7754788feb2d6b67fcfc3d9f40",
"ship_strategy": "Hash Partition on [0]",
"local_strategy": "Sort on [0:ASC]",
"exchange": "pipelined"
}
],
"optimizer_properties": {
"global_properties": [
{
"name": "Partitioning",
"value": "RANDOM_PARTITIONED"
},
{
"name": "Partitioning Order",
"value": "(none)"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"local_properties": [
{
"name": "Order",
"value": "(none)"
},
{
"name": "Grouping",
"value": "not grouped"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"estimates": [
{
"name": "Est. Output Size",
"value": "(unknown)"
},
{
"name": "Est. Cardinality",
"value": "(unknown)"
}
],
"costs": [
{
"name": "Network",
"value": "(unknown)"
},
{
"name": "Disk I/O",
"value": "(unknown)"
},
{
"name": "CPU",
"value": "(unknown)"
},
{
"name": "Cumulative Network",
"value": "(unknown)"
},
{
"name": "Cumulative Disk I/O",
"value": "(unknown)"
},
{
"name": "Cumulative CPU",
"value": "(unknown)"
}
],
"compiler_hints": [
{
"name": "Output Size (bytes)",
"value": "(none)"
},
{
"name": "Output Cardinality",
"value": "(none)"
},
{
"name": "Avg. Output Record Size (bytes)",
"value": "(none)"
},
{
"name": "Filter Factor",
"value": "(none)"
}
]
}
},
{
"id": "2481c440912b84ca49e8f8b72da1752d",
"parallelism": 1,
"operator": "GroupReduce",
"operator_strategy": "Sorted Group Reduce",
"description": "GroupReduce (GroupReduce at GenerateStatistics/RunStatsGenerators/StringStatsGenerator/CombinePerKey/GroupByKey)",
"inputs": [
{
"num": 0,
"id": "68f880071c774596985fcd67729f0f1a",
"ship_strategy": "Hash Partition on [0]",
"local_strategy": "Sort on [0:ASC]",
"exchange": "pipelined"
}
],
"optimizer_properties": {
"global_properties": [
{
"name": "Partitioning",
"value": "RANDOM_PARTITIONED"
},
{
"name": "Partitioning Order",
"value": "(none)"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"local_properties": [
{
"name": "Order",
"value": "(none)"
},
{
"name": "Grouping",
"value": "not grouped"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"estimates": [
{
"name": "Est. Output Size",
"value": "(unknown)"
},
{
"name": "Est. Cardinality",
"value": "(unknown)"
}
],
"costs": [
{
"name": "Network",
"value": "(unknown)"
},
{
"name": "Disk I/O",
"value": "(unknown)"
},
{
"name": "CPU",
"value": "(unknown)"
},
{
"name": "Cumulative Network",
"value": "(unknown)"
},
{
"name": "Cumulative Disk I/O",
"value": "(unknown)"
},
{
"name": "Cumulative CPU",
"value": "(unknown)"
}
],
"compiler_hints": [
{
"name": "Output Size (bytes)",
"value": "(none)"
},
{
"name": "Output Cardinality",
"value": "(none)"
},
{
"name": "Avg. Output Record Size (bytes)",
"value": "(none)"
},
{
"name": "Filter Factor",
"value": "(none)"
}
]
}
},
{
"id": "ab40a7291893738a7c7ecae2bcdd07a9",
"parallelism": 1,
"operator": "GroupReduce",
"operator_strategy": "Sorted Group Reduce",
"description": "GroupReduce (GroupReduce at DecodeData/InferFeatureTypes/CombinePerKey/GroupByKey)",
"inputs": [
{
"num": 0,
"id": "4983cfb36a4f03aebd68726bad8c05a5",
"ship_strategy": "Hash Partition on [0]",
"local_strategy": "Sort on [0:ASC]",
"exchange": "pipelined"
}
],
"optimizer_properties": {
"global_properties": [
{
"name": "Partitioning",
"value": "RANDOM_PARTITIONED"
},
{
"name": "Partitioning Order",
"value": "(none)"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"local_properties": [
{
"name": "Order",
"value": "(none)"
},
{
"name": "Grouping",
"value": "not grouped"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"estimates": [
{
"name": "Est. Output Size",
"value": "(unknown)"
},
{
"name": "Est. Cardinality",
"value": "(unknown)"
}
],
"costs": [
{
"name": "Network",
"value": "(unknown)"
},
{
"name": "Disk I/O",
"value": "(unknown)"
},
{
"name": "CPU",
"value": "(unknown)"
},
{
"name": "Cumulative Network",
"value": "(unknown)"
},
{
"name": "Cumulative Disk I/O",
"value": "(unknown)"
},
{
"name": "Cumulative CPU",
"value": "(unknown)"
}
],
"compiler_hints": [
{
"name": "Output Size (bytes)",
"value": "(none)"
},
{
"name": "Output Cardinality",
"value": "(none)"
},
{
"name": "Avg. Output Record Size (bytes)",
"value": "(none)"
},
{
"name": "Filter Factor",
"value": "(none)"
}
]
}
},
{
"id": "3bd851095315b142e35707a497509f11",
"parallelism": 1,
"operator": "MapPartition -> FlatMap",
"operator_strategy": "Map Partition<br/> -> FlatMap",
"description": "MapPartition (MapPartition at 77GenerateStatistics/RunStatsGenerators/CommonStatsGenerator/DoOnce/Map(decode).None/beam:env:docker:v1:0)<br/> -> FlatMap (FlatMap at 77GenerateStatistics/RunStatsGenerators/CommonStatsGenerator/DoOnce/Map(decode).None/beam:env:docker:v1:0/out.0)",
"inputs": [
{
"num": 0,
"id": "3f481475a4117a27391074a639a24c77",
"ship_strategy": "Forward",
"exchange": "pipelined"
},
{
"num": 1,
"id": "ee2a7ee51779773ead459855cd6fdf9a",
"ship_strategy": "Broadcast",
"exchange": "pipelined"
}
],
"optimizer_properties": {
"global_properties": [
{
"name": "Partitioning",
"value": "RANDOM_PARTITIONED"
},
{
"name": "Partitioning Order",
"value": "(none)"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"local_properties": [
{
"name": "Order",
"value": "(none)"
},
{
"name": "Grouping",
"value": "not grouped"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"estimates": [
{
"name": "Est. Output Size",
"value": "(unknown)"
},
{
"name": "Est. Cardinality",
"value": "(unknown)"
}
],
"costs": [
{
"name": "Network",
"value": "0.0"
},
{
"name": "Disk I/O",
"value": "0.0"
},
{
"name": "CPU",
"value": "0.0"
},
{
"name": "Cumulative Network",
"value": "(unknown)"
},
{
"name": "Cumulative Disk I/O",
"value": "(unknown)"
},
{
"name": "Cumulative CPU",
"value": "(unknown)"
}
],
"compiler_hints": [
{
"name": "Output Size (bytes)",
"value": "(none)"
},
{
"name": "Output Cardinality",
"value": "(none)"
},
{
"name": "Avg. Output Record Size (bytes)",
"value": "(none)"
},
{
"name": "Filter Factor",
"value": "(none)"
}
]
}
},
{
"id": "b81b935c3dbeb33defbe2a8cc43faf2a",
"parallelism": 1,
"operator": "Data Source",
"operator_strategy": "(none)",
"description": "DataSource (at WriteStatsOutput/Write/WriteImpl/DoOnce/Impulse (org.apache.beam.runners.flink.translation.wrappers.ImpulseInputFormat))",
"optimizer_properties": {
"global_properties": [
{
"name": "Partitioning",
"value": "RANDOM_PARTITIONED"
},
{
"name": "Partitioning Order",
"value": "(none)"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"local_properties": [
{
"name": "Order",
"value": "(none)"
},
{
"name": "Grouping",
"value": "not grouped"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"estimates": [
{
"name": "Est. Output Size",
"value": "1.00"
},
{
"name": "Est. Cardinality",
"value": "1.00"
}
],
"costs": [
{
"name": "Network",
"value": "0.0"
},
{
"name": "Disk I/O",
"value": "0.0"
},
{
"name": "CPU",
"value": "0.0"
},
{
"name": "Cumulative Network",
"value": "0.0"
},
{
"name": "Cumulative Disk I/O",
"value": "0.0"
},
{
"name": "Cumulative CPU",
"value": "0.0"
}
],
"compiler_hints": [
{
"name": "Output Size (bytes)",
"value": "(none)"
},
{
"name": "Output Cardinality",
"value": "(none)"
},
{
"name": "Avg. Output Record Size (bytes)",
"value": "(none)"
},
{
"name": "Filter Factor",
"value": "(none)"
}
]
}
},
{
"id": "6db900e63ce3b00bbed15ce6268976b0",
"parallelism": 1,
"operator": "MapPartition -> FlatMap",
"operator_strategy": "Map Partition<br/> -> FlatMap",
"description": "MapPartition (MapPartition at 77GenerateStatistics/RunStatsGenerators/StringStatsGenerator/DoOnce/Map(decode).None/beam:env:docker:v1:0)<br/> -> FlatMap (FlatMap at 77GenerateStatistics/RunStatsGenerators/StringStatsGenerator/DoOnce/Map(decode).None/beam:env:docker:v1:0/out.0)",
"inputs": [
{
"num": 0,
"id": "35bad9de5c3986d1aa37eb321996f9e1",
"ship_strategy": "Forward",
"exchange": "pipelined"
},
{
"num": 1,
"id": "e8d11f0a437ebd15b951c95626e4e052",
"ship_strategy": "Broadcast",
"exchange": "pipelined"
}
],
"optimizer_properties": {
"global_properties": [
{
"name": "Partitioning",
"value": "RANDOM_PARTITIONED"
},
{
"name": "Partitioning Order",
"value": "(none)"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"local_properties": [
{
"name": "Order",
"value": "(none)"
},
{
"name": "Grouping",
"value": "not grouped"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"estimates": [
{
"name": "Est. Output Size",
"value": "(unknown)"
},
{
"name": "Est. Cardinality",
"value": "(unknown)"
}
],
"costs": [
{
"name": "Network",
"value": "0.0"
},
{
"name": "Disk I/O",
"value": "0.0"
},
{
"name": "CPU",
"value": "0.0"
},
{
"name": "Cumulative Network",
"value": "(unknown)"
},
{
"name": "Cumulative Disk I/O",
"value": "(unknown)"
},
{
"name": "Cumulative CPU",
"value": "(unknown)"
}
],
"compiler_hints": [
{
"name": "Output Size (bytes)",
"value": "(none)"
},
{
"name": "Output Cardinality",
"value": "(none)"
},
{
"name": "Avg. Output Record Size (bytes)",
"value": "(none)"
},
{
"name": "Filter Factor",
"value": "(none)"
}
]
}
},
{
"id": "35bad9de5c3986d1aa37eb321996f9e1",
"parallelism": 1,
"operator": "MapPartition -> FlatMap",
"operator_strategy": "Map Partition<br/> -> FlatMap",
"description": "MapPartition (MapPartition at 73GenerateStatistics/RunStatsGenerators/StringStatsGenerator/DoOnce/Impulse.None/beam:env:docker:v1:0)<br/> -> FlatMap (FlatMap at 73GenerateStatistics/RunStatsGenerators/StringStatsGenerator/DoOnce/Impulse.None/beam:env:docker:v1:0/out.0)",
"inputs": [
{
"num": 0,
"id": "e2e9488c10ee82bb3299dff6498221d6",
"ship_strategy": "Forward",
"exchange": "pipelined"
}
],
"optimizer_properties": {
"global_properties": [
{
"name": "Partitioning",
"value": "RANDOM_PARTITIONED"
},
{
"name": "Partitioning Order",
"value": "(none)"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"local_properties": [
{
"name": "Order",
"value": "(none)"
},
{
"name": "Grouping",
"value": "not grouped"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"estimates": [
{
"name": "Est. Output Size",
"value": "(unknown)"
},
{
"name": "Est. Cardinality",
"value": "(unknown)"
}
],
"costs": [
{
"name": "Network",
"value": "0.0"
},
{
"name": "Disk I/O",
"value": "0.0"
},
{
"name": "CPU",
"value": "0.0"
},
{
"name": "Cumulative Network",
"value": "0.0"
},
{
"name": "Cumulative Disk I/O",
"value": "0.0"
},
{
"name": "Cumulative CPU",
"value": "0.0"
}
],
"compiler_hints": [
{
"name": "Output Size (bytes)",
"value": "(none)"
},
{
"name": "Output Cardinality",
"value": "(none)"
},
{
"name": "Avg. Output Record Size (bytes)",
"value": "(none)"
},
{
"name": "Filter Factor",
"value": "(none)"
}
]
}
},
{
"id": "e8d11f0a437ebd15b951c95626e4e052",
"parallelism": 1,
"operator": "MapPartition -> FlatMap",
"operator_strategy": "Map Partition<br/> -> FlatMap",
"description": "MapPartition (MapPartition at 97GenerateStatistics/RunStatsGenerators/StringStatsGenerator/CombinePerKey/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0)<br/> -> FlatMap (FlatMap at 97GenerateStatistics/RunStatsGenerators/StringStatsGenerator/CombinePerKey/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0/out.0)",
"inputs": [
{
"num": 0,
"id": "2481c440912b84ca49e8f8b72da1752d",
"ship_strategy": "Forward",
"exchange": "pipelined"
}
],
"optimizer_properties": {
"global_properties": [
{
"name": "Partitioning",
"value": "RANDOM_PARTITIONED"
},
{
"name": "Partitioning Order",
"value": "(none)"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"local_properties": [
{
"name": "Order",
"value": "(none)"
},
{
"name": "Grouping",
"value": "not grouped"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"estimates": [
{
"name": "Est. Output Size",
"value": "(unknown)"
},
{
"name": "Est. Cardinality",
"value": "(unknown)"
}
],
"costs": [
{
"name": "Network",
"value": "0.0"
},
{
"name": "Disk I/O",
"value": "0.0"
},
{
"name": "CPU",
"value": "0.0"
},
{
"name": "Cumulative Network",
"value": "(unknown)"
},
{
"name": "Cumulative Disk I/O",
"value": "(unknown)"
},
{
"name": "Cumulative CPU",
"value": "(unknown)"
}
],
"compiler_hints": [
{
"name": "Output Size (bytes)",
"value": "(none)"
},
{
"name": "Output Cardinality",
"value": "(none)"
},
{
"name": "Avg. Output Record Size (bytes)",
"value": "(none)"
},
{
"name": "Filter Factor",
"value": "(none)"
}
]
}
},
{
"id": "27ea7d6f7814e0c565930b8fd6577fba",
"parallelism": 1,
"operator": "Data Sink",
"operator_strategy": "(none)",
"description": "DataSink (org.apache.flink.api.java.io.DiscardingOutputFormat@32ec80e7)",
"inputs": [
{
"num": 0,
"id": "1e2ed63c2e5714f106bcbed1130c3957",
"ship_strategy": "Forward",
"exchange": "pipelined"
}
],
"optimizer_properties": {
"global_properties": [
{
"name": "Partitioning",
"value": "RANDOM_PARTITIONED"
},
{
"name": "Partitioning Order",
"value": "(none)"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"local_properties": [
{
"name": "Order",
"value": "(none)"
},
{
"name": "Grouping",
"value": "not grouped"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"estimates": [
{
"name": "Est. Output Size",
"value": "(unknown)"
},
{
"name": "Est. Cardinality",
"value": "(unknown)"
}
],
"costs": [
{
"name": "Network",
"value": "0.0"
},
{
"name": "Disk I/O",
"value": "0.0"
},
{
"name": "CPU",
"value": "0.0"
},
{
"name": "Cumulative Network",
"value": "(unknown)"
},
{
"name": "Cumulative Disk I/O",
"value": "(unknown)"
},
{
"name": "Cumulative CPU",
"value": "(unknown)"
}
],
"compiler_hints": [
{
"name": "Output Size (bytes)",
"value": "(none)"
},
{
"name": "Output Cardinality",
"value": "(none)"
},
{
"name": "Avg. Output Record Size (bytes)",
"value": "(none)"
},
{
"name": "Filter Factor",
"value": "(none)"
}
]
}
},
{
"id": "f6ef3a5be1f89eca633122d825dde2e4",
"parallelism": 1,
"operator": "MapPartition -> FlatMap",
"operator_strategy": "Map Partition<br/> -> FlatMap",
"description": "MapPartition (MapPartition at 112GenerateStatistics/RunStatsGenerators/MergeDatasetFeatureStatisticsProtos/CombinePerKey/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0)<br/> -> FlatMap (FlatMap at 112GenerateStatistics/RunStatsGenerators/MergeDatasetFeatureStatisticsProtos/CombinePerKey/GroupByKey/GroupByWindow.None/beam:env:docker:v1:0/out.0)",
"inputs": [
{
"num": 0,
"id": "1e4111a2c49845717e0275467fc1f09f",
"ship_strategy": "Forward",
"exchange": "pipelined"
}
],
"optimizer_properties": {
"global_properties": [
{
"name": "Partitioning",
"value": "RANDOM_PARTITIONED"
},
{
"name": "Partitioning Order",
"value": "(none)"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"local_properties": [
{
"name": "Order",
"value": "(none)"
},
{
"name": "Grouping",
"value": "not grouped"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"estimates": [
{
"name": "Est. Output Size",
"value": "(unknown)"
},
{
"name": "Est. Cardinality",
"value": "(unknown)"
}
],
"costs": [
{
"name": "Network",
"value": "0.0"
},
{
"name": "Disk I/O",
"value": "0.0"
},
{
"name": "CPU",
"value": "0.0"
},
{
"name": "Cumulative Network",
"value": "(unknown)"
},
{
"name": "Cumulative Disk I/O",
"value": "(unknown)"
},
{
"name": "Cumulative CPU",
"value": "(unknown)"
}
],
"compiler_hints": [
{
"name": "Output Size (bytes)",
"value": "(none)"
},
{
"name": "Output Cardinality",
"value": "(none)"
},
{
"name": "Avg. Output Record Size (bytes)",
"value": "(none)"
},
{
"name": "Filter Factor",
"value": "(none)"
}
]
}
},
{
"id": "12012f2c6a9d768ad5651fd2d01f4276",
"parallelism": 1,
"operator": "MapPartition -> FlatMap",
"operator_strategy": "Map Partition<br/> -> FlatMap",
"description": "MapPartition (MapPartition at 51WriteStatsOutput/Write/WriteImpl/DoOnce/Map(decode).None/beam:env:docker:v1:0)<br/> -> FlatMap (FlatMap at 51WriteStatsOutput/Write/WriteImpl/DoOnce/Map(decode).None/beam:env:docker:v1:0/out.0)",
"inputs": [
{
"num": 0,
"id": "766e63063a9b847725557fe8646ec382",
"ship_strategy": "Forward",
"exchange": "blocking"
}
],
"optimizer_properties": {
"global_properties": [
{
"name": "Partitioning",
"value": "RANDOM_PARTITIONED"
},
{
"name": "Partitioning Order",
"value": "(none)"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"local_properties": [
{
"name": "Order",
"value": "(none)"
},
{
"name": "Grouping",
"value": "not grouped"
},
{
"name": "Uniqueness",
"value": "not unique"
}
],
"estimates": [
{
"name": "Est. Output Size",
"value": "(unknown)"
},
{
"name": "Est. Cardinality",
"value": "(unknown)"
}
],
"costs": [
{
"name": "Network",
"value": "0.0"
},
{
"name": "Disk I/O",
"value": "0.0"
},
{
"name": "CPU",
"value": "0.0"
},
{
"name": "Cumulative Network",
"value": "0.0"
},
{
"name": "Cumulative Disk I/O",
"value": "0.0"
},
{
"name": "Cumulative CPU",
"value": "0.0"
}
],
"compiler_hints": [
{
"name": "Output Size (bytes)",
"value": "(none)"
},
{
"name": "Output Cardinality",
"value": "(none)"
},
{
"name": "Avg. Output Record Size (bytes)",
"value": "(none)"
},
{
"name": "Filter Factor",
"value": "(none)"
}
]
}
}
]
}
}
Attachments
Issue Links
- blocks
-
BEAM-6015 Uber task for Portable Flink scalability
-
- Open
-