Description
When interacting with Azure Blob Storage via the Hadoop FileSystem client, it's currently (as of a8bbd81) not possible to set custom metadata attributes.
Here is a snippet that demonstrates the missing behavior (throws an UnsupportedOperationException):
val blobAccount = "SET ME" val blobKey = "SET ME" val blobContainer = "SET ME" val blobFile = "SET ME" import org.apache.hadoop.conf.Configuration import org.apache.hadoop.fs.{FileSystem, Path} val conf = new Configuration() conf.set("fs.wasbs.impl", "org.apache.hadoop.fs.azure.NativeAzureFileSystem") conf.set(s"fs.azure.account.key.$blobAccount.blob.core.windows.net", blobKey) val path = new Path(s"wasbs://$blobContainer@$blobAccount.blob.core.windows.net/$blobFile") val fs = FileSystem.get(path, conf) fs.setXAttr(path, "somekey", "somevalue".getBytes)
Looking at the code in hadoop-tools/hadoop-azure, NativeAzureFileSystem inherits the default setXAttr from FileSystem which throws the UnsupportedOperationException.
The underlying Azure Blob Storage service does support custom metadata (service docs) as does the azure-storage SDK that's being used by NativeAzureFileSystem (SDK docs).
Is there another way that I should be setting custom metadata on Azure Blob Storage files? Is there a specific reason why setXAttr hasn't been implemented on NativeAzureFileSystem? If not, I can take a shot at implementing it.