Code improvements for 3.0: existentials edition

The Spark code base makes use of 'existential types' in Scala, a language feature which is quasi-deprecated – it generates a warning unless scala.language.existentials is imported, and there is talk of removing it from future Scala versions: https://contributors.scala-lang.org/t/proposal-to-remove-existential-types-from-the-language/2785

We can get rid of most usages of this feature with lots of minor changes to the code. A PR is coming to demonstrate what's involved.