Optimize SortedSet/SortedNumeric doc values writers for fields that are effectively single-valued

I was wondering how much overhead multi-valued doc-value types have over their single-valued counterparts, so I hacked IndexTaxis to index all doc-value fields via Sorted(Set|Numeric)DocValuesField instead of (Sorted|Numeric)DocValuesField and flush times increased by 30%. It should be easy to automatically detect such cases in the doc values writers?