Long running spark jobs fail because of HDFS delegation token expires

When the spark job runs on a secured cluster for longer then time that is mentioned in the dfs.namenode.delegation.token.renew-interval property of hdfs-site.xml the spark job fails. ** 

Following command was used to submit the spark job

bin/spark-submit --principal acekrbuser --keytab ~/keytabs/acekrbuser.keytab --master yarn --deploy-mode cluster examples/src/main/python/wordcount.py /tmp/ff1.txt

Application Logs attached