Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
0.10.0
-
None
Description
ERROR [2021-11-29 14:46:51,519] ({FIFOScheduler-org.apache.zeppelin.file.FileInterpreter1871691756-Worker-1} HDFSFileInterpreter.java[listAll]:249) - listall: listDir /user/*******
com.google.gson.JsonSyntaxException: java.lang.NumberFormatException: Expected an int but was 4947954640 at line 1 column 585 path $.FileStatuses.FileStatus[2].fileId
I found this error when using file(hdfs) interpreter.
Some paths are okay.
I met this exception when trying to access paths with high file id (above integer).
I think that "blockSize" & "fileId" should be declared as "long"
Refer to HdfsFileStatus.java,
class Builder { // Changing default values will affect cases where values are not // specified. Be careful! private long length = 0L; private boolean isdir = false; private int replication = 0; private long blocksize = 0L; private long mtime = 0L; private long atime = 0L; private FsPermission permission = null; private EnumSet<Flags> flags = EnumSet.noneOf(Flags.class); private String owner = null; private String group = null; private byte[] symlink = null; private byte[] path = EMPTY_NAME; private long fileId = -1L; private int childrenNum = 0; private FileEncryptionInfo feInfo = null; private byte storagePolicy = HdfsConstants.BLOCK_STORAGE_POLICY_ID_UNSPECIFIED; private ErasureCodingPolicy ecPolicy = null; private LocatedBlocks locations = null; ...
- HDFSFileInterpreter.java: https://github.com/apache/zeppelin/blob/85ed8e2e51e1ea10df38d4710216343efe218d60/file/src/main/java/org/apache/zeppelin/file/HDFSFileInterpreter.java#L87
- HdfsFileStatus.java: https://github.com/apache/hadoop/blob/branch-3.1.2/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/HdfsFileStatus.java#L62-L83
Attachments
Issue Links
- links to