Missing SortedMap type in pyspark

Currently there is only MapType in pyspark API, which does not keep order of key-value map.

 

SortedMapType would fill this gap: a map with sorted key-value pairs (like TreeMap in Java).

 

For example:

SortedMapType would be very useful when user wants to persist row from DataFrame into Mongo-db.

row which consist a column with values of an ordered map type:

col : [1 -> 22, 2 -> 16, 3 -> 25]

will be persisted as following json:

{   "1": 22,   "2": 16,   "3": 25 }

 

instead of MapType which currently results in:

{   "2": 16,   "1": 22,   "3": 25 }