CREATE TABLE AS SELECT changes behaviour in SPARK 3.4.0

SPARK-41859 changes the behaviour for `CREATE TABLE AS SELECT ...` from OVERWRITE to APPEND when spark.sql.legacy.allowNonEmptyLocationInCTAS is set to true:

drop table if exists test_table;
create table test_table location '/tmp/test_table' stored as parquet as select 1 as col union all select 2 as col;
drop table if exists test_table;
create table test_table location '/tmp/test_table' stored as parquet as select 3 as col union all select 4 as col;
select * from test_table;

This produces

in Spark <3.4.0 and

in Spark 3.4.0 and later. This is a silent change in spark.sql.legacy.allowNonEmptyLocationInCTAS behaviour which introduces wrong results in the user application