Details
-
Improvement
-
Status: Closed
-
Major
-
Resolution: Implemented
-
None
-
None
Description
Make WAL a pluggable, configurable component, thus making it easier to write to different filesystems (including multiple filesystems).
From Stack:
Pluggable WAL component would need to check that the split can deal w/
multiple logs written by the one server concurrently (sort by sequence
edit id after sorting on all the rest that makes up a wal log key).
From Jesse Yates:
It would be nice to be able to tie pluggable WAL component into a service that logs directly to
disk, rather than go through HDFS giving some potentially awesome speedup at
the cost of having to write a logging service that handles replication, etc.
From Karthik Tunga:
Along with the log replaying part, logic is also needed for log roll.
This, I think, is easier compared to the merging of the logs. Any edits less
than the last sequence number on the file system can be removed from all
the WALs.