[SPARK-28199] Move Trigger implementations to Triggers.scala and avoid exposing these to the end users - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Improvement
Status: Resolved
Priority: Minor
Resolution: Fixed
Affects Version/s: 3.0.0
Fix Version/s: 3.0.0
Component/s: Structured Streaming
Labels:
- release-notes

Docs Text:

Hide
In Spark 3.0, the deprecated class org.apache.spark.sql.streaming.ProcessingTime has been removed. Use org.apache.spark.sql.streaming.Trigger.ProcessingTime() instead. Likewise, org.apache.spark.sql.execution.streaming.continuous.ContinuousTrigger has been removed in favor of Trigger.Continuous(), and org.apache.spark.sql.execution.streaming.OneTimeTrigger has been hidden in favor of Trigger.Once().

Show
In Spark 3.0, the deprecated class org.apache.spark.sql.streaming.ProcessingTime has been removed. Use org.apache.spark.sql.streaming.Trigger.ProcessingTime() instead. Likewise, org.apache.spark.sql.execution.streaming.continuous.ContinuousTrigger has been removed in favor of Trigger.Continuous(), and org.apache.spark.sql.execution.streaming.OneTimeTrigger has been hidden in favor of Trigger.Once().

Description

Even ProcessingTime is deprecated in 2.2.0, it's being used in Spark codebase, and actually the alternative Spark proposes use deprecated methods which feels like circular - never be able to remove usage.

In fact, ProcessingTime is deprecated because we want to only expose Trigger.xxx instead of exposing actual implementations, and I think we miss some other implementations as well.

This issue targets to move all Trigger implementations to Triggers.scala, and hide from end users.