Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
3.5.0, 4.0.0, 3.5.1, 3.5.2
Description
There are currently some race conditions between maintenance thread and task thread which can result in corrupted checkpoint state.
- The maintenance thread currently relies on class variable lastSnapshot to find the latest checkpoint and uploads it to DFS. This checkpoint can be modified at commit time by Task thread if a new snapshot is created.
- The task thread does not reset lastSnapshot at load time, which can result in newer snapshots (if a old version is loaded) being considered valid and uploaded to DFS. This results in VersionIdMismatch errors.
This issue proposes to fix these issues by guarding latestSnapshot variable modification, and setting latestSnapshot properly at load time.