Apache Beam Metrics Counter giving incorrect count using SparkRunner

I am having source and target csv files with 10 million records and 250 columns. I am running an apache beam pipeline which joins all columns from source and target file. When I run this on spark cluster the pipeline executes correctly with no exceptions but, The join beam metrics counter returns double count when the following spark property is used. – executor-memory "2g" But, When I increase the excutor-memory to 11g then it returns the correct count.
Count doubles only when I dump the results to file but if I don't dump then counts are correct.

Note : https://stackoverflow.com/questions/59032734/apache-beam-metrics-counter-giving-incorrect-count-using-sparkrunner?noredirect=1#comment104344657_59032734