Pyspark DecisionTreeClassifier: results and tree structure in spark3 very different from that of the spark2 version on the same data and with the same hyperparameters.

dt  = DecisionTreeClassifier(maxBins=10000, seed=0)   
cv_dt_evaluator = BinaryClassificationEvaluator(
            metricName="", 
            rawPredictionCol="probability")

# Create param grid and cross validator for model selection
dt_grid = ParamGridBuilder()\
            .addGrid(
                dt.minInstancesPerNode, [100]
        )\
            .addGrid(
                dt.maxDepth, [10]
        )\
            .build()
cv = CrossValidator(
            estimator=dt, estimatorParamMaps=dt_grid, evaluator=cv_dt_evaluator,
            parallelism=4
            numFolds=4
        )