Details
-
Bug
-
Status: Open
-
Minor
-
Resolution: Unresolved
-
3.1.3
-
None
-
None
Description
With regards to documentation - https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HdfsQuotaAdminGuide.html#Space_Quotas
- hdfs dfsadmin -setSpaceQuota <N> <directory>...<directory>
Set the space quota to be N bytes for each directory. This is a hard limit on total size of all the files under the directory tree. The space quota takes replication also into account, i.e. one GB of data with replication of 3 consumes 3GB of quota. N can also be specified with a binary prefix for convenience, for e.g. 50g for 50 gigabytes and 2t for 2 terabytes etc. Best effort for each directory, with faults reported if N is neither zero nor a positive integer, the directory does not exist or it is a file, or the directory would immediately exceed the new quota.
So, command to set Quota have to fail if quota already exceeded
Actual result – command finished with exit code 0
Steps to reproduce:
- put file 1GB into hdfs folder
- try to set quota of 512MB
hdfs dfs -ls /hdfs_dir
Exit code: 0
Found 1 items
- 3 user hadoop 1073741824 2024-01-17 18:57 /hdfs_dir/File
hdfs dfsadmin -setSpaceQuota 512m /hdfs_dir
result:
Exit code: 0
Quota report shows negative value for SPACE_REM_QUOTA
REM_SPACE_QUOTA -2.5GB QUOTA REM_QUOTA SPACE_QUOTA REM_SPACE_QUOTA DIR_COUNT FILE_COUNT CONTENT_SIZE PATHNAME none inf 512 M -2.5 G 1 1 1 G /hdfs_dir
The same situation with setQuota method, if we try to set name quota, for example, 3 for folder already contained 5 files – it doesn’t fail and shows negative value for REM_QUOTA
The reason for this behavior is absent quota check during SET operations