Details
-
Improvement
-
Status: Resolved
-
Major
-
Resolution: Duplicate
-
None
Description
On DSGC the blob references are collected from the individual segments. It would be better to also have them in the tar files (similar to the segment graph).
Scanning through all segments is troublesome for various reasons( e.g. cache coherence, IO thrashing etc). A similar change in the POC for OAK-3348 resulted in an speedup of 2 orders of magnitude. In that case the segment generation was written to the tar index instead being read from the individual segments.