Details
-
New Feature
-
Status: Resolved
-
Major
-
Resolution: Duplicate
-
2.3.1
-
None
-
None
Description
We are doing video analytics for video streams using Spark. At present there is no direct way to stream video frames or image files to Spark and process them using Structured Streaming and Dataset. We are using Kafka to stream images and then doing processing at spark. We need a method in Spark to stream images from directory. Currently DataStreamReader doesn't support Image files. With the introduction of org.apache.spark.ml.image.ImageSchema class, we think streaming capabilities can be added for image files. It is fine if it won't support some of the structured streaming features as it is a binary file. This method could be similar to mmlspark streamImages method. https://github.com/Azure/mmlspark/blob/4413771a8830e4760f550084da60ea0616bf80b9/src/io/image/src/main/python/ImageReader.py
Attachments
Issue Links
- duplicates
-
SPARK-22666 Spark datasource for image format
- Resolved