Description
When you use a custom line delimiter spark.hadoop.textinputformat.record.delimiter that has a leading or a trailing whitespace character it's only possible when specified via --conf . Our pipeline consists of a highly customized generated jobs. Storing all the config in a properities file is not only better for readability but even necessary to avoid dealing with ARGS_MAX on different OS. Spark should uniformly avoid trimming conf values in both cases.