Pig on Spark

Setting up your development environment:
0. download spark release package(currently pig on spark only support spark 1.6).
1. Check out Pig Spark branch.

2. Build Pig by running "ant jar" and "ant -Dhadoopversion=23 jar" for hadoop-2.x versions

3. Configure these environmental variables:
export HADOOP_USER_CLASSPATH_FIRST="true"
Now we support “local” and "yarn-client" mode, you can export system variable “SPARK_MASTER” like:
export SPARK_MASTER=local or export SPARK_MASTER="yarn-client"

4. In local mode: ./pig -x spark_local xxx.pig
In yarn-client mode:
export SPARK_HOME=xx;
export SPARK_JAR=hdfs://example.com:8020/xxxx (the hdfs location where you upload the spark-assembly*.jar)
./pig -x spark xxx.pig