[SPARK-14745] [Streaming] [WIP] CEP support in Spark Streaming

[agsachin]

What changes were proposed in this pull request?

This JIRA is about how/what can we add in Spark Streaming to support CEP out of the box.
The code in this PR needs to merge up with Structured Streaming changes in 2.0. The PR is issued for now to facilitate discussion and view by other users interested in CEP and asked on spark-user group

How was this patch tested?

Manually with KafkaDirect. WIP

[AmplabJenkins]

Can one of the admins verify this patch?

[tdas]

Hello @agsachin,

This is a cool idea, and we apologize that this has been sitting here for a while. Unfortunately, add-on features like this is unlikely to be core API for DStreams/DataFrames. So this is something that is best contributed to open-source as a spark-package.

[rxin]

@tdas I wouldn't rule it out as a possibility in the future, but I agree currently the project needs to focus more on structured streaming APIs first and then think about more complicated use cases like CEP.

[tdas]

@rxin thanks for the clarification!

[dpinol]

@agsachin Hi, do you plan to create a spark-package with this?

thanks

[opsnull]

@rxin HI，do you have any plan on CEP ？

[olad32]

Hi, is there any new plan on CEP ?

[smileyboy2019]

When to add CEP functionality to meet complex business needs