Optimize Utilities.getInputPaths() so each listStatus of a partition is done in parallel

When running on blobstores (like S3) where metadata operations (like listStatus) are costly, Utilities.getInputPaths() can add significant overhead when setting up the input paths for an MR / Spark / Tez job.

The method performs a listStatus on all input paths in order to check if the path is empty. If the path is empty, a dummy file is created for the given partition. This is all done sequentially. This can be really slow when there are a lot of empty partitions. Even when all partitions have input data, this can take a long time.

We should either:

(1) Just remove the logic to check if each input path is empty, and handle any edge cases accordingly.

(2) Multi-thread the listStatus calls