Details
-
New Feature
-
Status: Resolved
-
Major
-
Resolution: Incomplete
-
2.3.1
-
None
Description
Full-text search (i.e., keyword search) is widely used in search engines and relational databases such as MATCH() ... AGAINST operator in MySQL (https://dev.mysql.com/doc/en/fulltext-search.html), Text query in Oracle (https://docs.oracle.com/cd/B28359_01/text.111/b28303/query.htm#g1016054), and text search in PostgreSQL (https://www.postgresql.org/docs/9.5/static/textsearch.html). However, it is not natively supported in Spark SQL. We propose an approach to implement this full-text search in Spark SQL.
Our proposed approach is detailed at https://github.com/JerryLead/Misc/blob/master/FullTextSearch/Full-text-issue-2018.pdf
and the prototype is available at https://github.com/bigdata-iscas/SparkFullTextQuery/tree/like_explorer