前言
近期在做Oracle遷移到Spark平臺的項目上遇到了一些平臺公式翻譯為SparkSQL(on Hive)的需求，而Spark采用親媽語言Scala進行開發。下面是個意外，被論文查重了,移步至我的Leanote博客查看點我，先亂碼一段時間[分后，擬使中的EB式，進行基于@#@#@法解析。
平臺公式及翻譯后的SparkSQL
平臺公式的樣子如下所示：
if (XX1_m001[D003]="邢おb7骯α?薇" || XX1_m001[H003]
這里面字段值"邢おb7骯α?薇"為這個的目的是為了測試各種字符集是否都能匹配滿足。
那么對應的SparkSQL應該是這個樣子的,由于是使用的Hive on Spark，因而長得跟Oracle的SQL語句差不多：
SELECT COUNT(H022) FROM XX1_m001 WHERE (XX1_m001.D003='邢おb7骯α?薇' OR XX1_m001.H003
總體而言比較簡單，因為我只是想在這里做一個Demo。
平臺公式的EBNF范式及詞法解析設計
expr-condition ::= tableName "[" valueName "]" comparator Condition
expr-front ::= expr-condition (("&&"|"||")expr-front)*
expr-back ::= tableName "[" valueName "," operator "]"
expr ::= "if" expr-front "then" expr-back
其中詞法定義如下
operator => [SUM,COUNT]
tableName,valueName =>ident #ident為關鍵字
comparator => ["=",">=","<=",">","
Condition => stringLit #stringLit為字符串常量
使用Scala基于詞法單元的解析器解析上述EBNF文法
Scala基于詞法單元的解析器是需要繼承StandardTokenParsers這個類的，該類提供了很方便的解析函數，以及詞法集合。
我們可以通過使用lexical.delimiters列表來存放在文法翻譯器執行過程中遇到的分隔符，使用lexical.reserved列表來存放執行過程中的關鍵字。
比如，我們參照平臺公式，看到"=",">=","<=",">","=","<=",">","
表現在代碼中是醬紫的：
lexical.delimiters += ("=",">=","<=",">","
lexical.reserved += ("if","then","SUM","COUNT")
是不是so easy~。
我們再來看一下如何使用基于詞法單元的解析器解析前面我們設計的EBNF文法呢。我在這里先上代碼：
class ExprParsre extends StandardTokenParsers{
lexical.delimiters += ("=",">=","<=",">","
lexical.reserved += ("if","then","SUM","COUNT")
def expr: Parser[String] = "if" ~ expr_front ~ "then" ~ expr_back ^^{
case "if" ~ exp1 ~ "then" ~ exp2 => exp2 + " WHERE " +exp1
}
def expr_priority: Parser[String] = opt("(") ~ expr_condition ~ opt(")") ^^{
case Some("(") ~ conditions ~ Some(")") => "(" + conditions +")"
case Some("(") ~ conditions ~ None => "(" + conditions
case None ~ conditions ~ Some(")") => conditions +")"
case None ~ conditions ~ None => conditions
}
def expr_condition: Parser[String] = ident ~ "[" ~ ident ~ "]" ~ ("="|">="|"<="|">"|"
case ident1~"["~ident2~"]"~"="~stringList => ident1 + "." + ident2 +"='" + stringList +"'"
case ident1~"["~ident2~"]"~">="~stringList => ident1 + "." + ident2 +">='" + stringList +"'"
case ident1~"["~ident2~"]"~"<="~stringList => ident1 + "." + ident2 +"<='" + stringList +"'"
case ident1~"["~ident2~"]"~">"~stringList => ident1 + "." + ident2 +">'" + stringList +"'"
case ident1~"["~ident2~"]"~" ident1 + "." + ident2 +"
case ident1~"["~ident2~"]"~"!="~stringList => ident1 + "." + ident2 +"!='" + stringList +"'"
}
def comparator: Parser[String] = ("&&"|"||") ^^{
case "&&" => " AND "
case "||" => " OR "
}
def expr_front: Parser[String] = expr_priority ~ rep(comparator ~ expr_priority) ^^{
case exp1 ~ exp2 => exp1 + exp2.map(x =>{x._1 + " " + x._2}).mkString(" ")
}
def expr_back: Parser[String] = ident ~ "[" ~ ident ~ "," ~ ("SUM"|"COUNT") ~ "]" ^^ {
case ident1~"["~ident2~","~"COUNT"~"]" => "SELECT COUNT("+ ident2.toString() +") FROM " + ident1.toString()
case ident1~"["~ident2~","~"SUM"~"]" => "SELECT SUM("+ ident2.toString() +") FROM " + ident1.toString()
}
def parserAll[T]( p : Parser[T], input :String) = {
phrase(p)( new lexical.Scanner(input))
}
}
