LocalDirAllocator cannot recover from directory tree deletion during the life of a filesystem client

The s3a and abfs clients use LocalDirAllocator for allocating files in local (temporary) storage for buffering blocks to write, and, for the s3a staging committer, files being staged.
When initialized (or when the configuration key value is updated) LocalDirAllocator enumerates all directories in the list and calls mkdirs() to create them.

when you ask actually for a file, it will look for the parent dir, and will again call mkdirs().

But before it does that, it looks to see if the dir has any space...if not it is excluded from the list of directories with room for data.

And guess what: directories which don't exist report as having no space. So they get excluded -the recreation code doesn't get a chance to run.