Shuffle Storage API: Dynamic updates of shuffle metadata

When using external storage for shuffles as part of the shuffle storage API mechanism, it is often desirable to update the metadata associated with shuffles that we have enabled plugin systems to implement via https://issues.apache.org/jira/browse/SPARK-31801. For example:

1. If data is stored in some replicated manner, and the number of replicas is updated - then we want the metadata stored on the driver to reflect the new number of replicas and where they are located.
2. If data is stored on the mapper's local disk, but is asynchronously backed up to some external storage medium, then we want to know when a backup is available externally.

To achieve this, we would need to pass a hook to updating the shuffle metadata to the shuffle executor components at the root of the plugin tree on the executor side. The executor would establish an RPC connection with the driver and send messages to update shuffle metadata accordingly.