21/09/28 11:18:51 WARN YarnScheduler: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources

Hi, I am running a spark job with yarn programmatically using Eclipse IDE. 
Here I

1: open the spark session passing a SparkConf as input parameter,

 

System.setProperty("hadoop.home.dir", "/home/hadoop/hadoop"); System.setProperty("hadoop.home.dir", "/home/hadoop/hadoop");        System.setProperty("SPARK_YARN_MODE", "yarn");        System.setProperty("HADOOP_USER_NAME", "hadoop");
SparkConf sparkConf = new SparkConf().setAppName("simpleTest2").setMaster("yarn") .set("spark.executor.memory", "1g")
.set("deploy.mode", "cluster")
.set("spark.yarn.stagingDir", "hdfs://localhost:9000/user/hadoop/")
.set("spark.yarn.am.memory", "512m") .set("spark.dynamicAllocation.minExecutors","1") .set("spark.dynamicAllocation.maxExecutors","40") .set("spark.dynamicAllocation.initialExecutors","2")         .set("spark.shuffle.service.enabled", "true")         .set("spark.dynamicAllocation.enabled", "false")
.set("spark.cores.max", "1")
 .set("spark.yarn.executor.memoryOverhead", "500m")         .set("spark.executor.instances","2")
.set("spark.executor.memory","500m")
.set("spark.num.executors","2")
.set("spark.executor.cores","1")
.set("spark.worker.instances","1")
.set("spark.worker.memory","512m")
.set("spark.worker.max.heapsize","512m")
.set("spark.worker.cores","1")
.set("maximizeResourceAllocation", "true") .set("spark.yarn.nodemanager.resource.cpu-vcores","4") .set("spark.yarn.submit.file.replication", "1")
SparkSession spark = SparkSession.builder().config(sparkConf).getOrCreate(); 

 

2: Create a dataset of two Rows and i Show them

List<Row> rows = new ArrayList<>(); List<Row> rows = new ArrayList<>(); rows.add(RowFactory.create("a", "b")); rows.add(RowFactory.create("b", "c")); rows.add(RowFactory.create("a", "a"));
 StructType structType = new StructType(); structType = structType.add("edge_1", DataTypes.StringType, false); structType = structType.add("edge_2", DataTypes.StringType, false); ExpressionEncoder<Row> edgeEncoder = RowEncoder.apply(structType);
 Dataset<Row> edge = spark.createDataset(rows, edgeEncoder); edge.show();
 

From now it is all Ok, the job is submitted on hadoop and the rows are showed correctly

3: I perform a Map that upper cases the elements in the row

Dataset<Row> edge2 = edge.map(new MyFunction2(), edgeEncoder); Dataset<Row> edge2 = edge.map(new MyFunction2(), edgeEncoder);

{{ public static class MyFunction2 implements MapFunction<Row, Row> { public static class MyFunction2 implements MapFunction<Row, Row> {}}
{{ /** *  */ private static final long serialVersionUID = 1L;}}
{{ @Override public Row call(Row v1) throws Exception { String el1 = v1.get(0).toString().toUpperCase(); String el2 = v1.get(1).toString().toUpperCase(); return RowFactory.create(el1,el2); }}}
{{ }}}

4: Then I show the dataset after map is performed

edge2.show();

And precisely here the log Loops saying 

21/09/28 11:18:51 WARN YarnScheduler: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources

Here is the Full log

log4j:WARN No appenders could be found for logger (org.apache.hadoop.metrics2.lib.MutableMetricsFactory).log4j:WARN No appenders could be found for logger (org.apache.hadoop.metrics2.lib.MutableMetricsFactory).log4j:WARN Please initialize the log4j system properly.log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties21/09/28 03:05:16 WARN Utils: Your hostname, davben-lubuntu resolves to a loopback address: 127.0.1.1; using 192.168.1.36 instead (on interface wlo1)21/09/28 03:05:16 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to another addressWARNING: An illegal reflective access operation has occurredWARNING: Illegal reflective access by org.apache.spark.unsafe.Platform (file:/home/davben/.m2/repository/org/apache/spark/spark-unsafe_2.12/3.1.2/spark-unsafe_2.12-3.1.2.jar) to constructor java.nio.DirectByteBuffer(long,int)WARNING: Please consider reporting this to the maintainers of org.apache.spark.unsafe.PlatformWARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operationsWARNING: All illegal access operations will be denied in a future release21/09/28 03:05:16 WARN SparkConf: The configuration key 'spark.yarn.executor.memoryOverhead' has been deprecated as of Spark 2.3 and may be removed in the future. Please use the new key 'spark.executor.memoryOverhead' instead.21/09/28 03:05:16 WARN SparkConf: The configuration key 'spark.yarn.executor.memoryOverhead' has been deprecated as of Spark 2.3 and may be removed in the future. Please use the new key 'spark.executor.memoryOverhead' instead.21/09/28 03:05:16 INFO SparkContext: Running Spark version 3.1.221/09/28 03:05:16 INFO ResourceUtils: ==============================================================21/09/28 03:05:16 INFO ResourceUtils: No custom resources configured for spark.driver.21/09/28 03:05:16 INFO ResourceUtils: ==============================================================21/09/28 03:05:16 INFO SparkContext: Submitted application: simpleTest221/09/28 03:05:16 INFO ResourceProfile: Default ResourceProfile created, executor resources: Map(memoryOverhead -> name: memoryOverhead, amount: 500, script: , vendor: , cores -> name: cores, amount: 1, script: , vendor: , memory -> name: memory, amount: 500, script: , vendor: , offHeap -> name: offHeap, amount: 0, script: , vendor: ), task resources: Map(cpus -> name: cpus, amount: 1.0)21/09/28 03:05:16 INFO ResourceProfile: Limiting resource is cpus at 1 tasks per executor21/09/28 03:05:16 INFO ResourceProfileManager: Added ResourceProfile id: 021/09/28 03:05:16 INFO SecurityManager: Changing view acls to: davben,hadoop21/09/28 03:05:16 INFO SecurityManager: Changing modify acls to: davben,hadoop21/09/28 03:05:16 INFO SecurityManager: Changing view acls groups to: 21/09/28 03:05:16 INFO SecurityManager: Changing modify acls groups to: 21/09/28 03:05:16 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users  with view permissions: Set(davben, hadoop); groups with view permissions: Set(); users  with modify permissions: Set(davben, hadoop); groups with modify permissions: Set()21/09/28 03:05:16 INFO Utils: Successfully started service 'sparkDriver' on port 38955.21/09/28 03:05:17 INFO SparkEnv: Registering MapOutputTracker21/09/28 03:05:17 INFO SparkEnv: Registering BlockManagerMaster21/09/28 03:05:17 INFO BlockManagerMasterEndpoint: Using org.apache.spark.storage.DefaultTopologyMapper for getting topology information21/09/28 03:05:17 INFO BlockManagerMasterEndpoint: BlockManagerMasterEndpoint up21/09/28 03:05:17 INFO SparkEnv: Registering BlockManagerMasterHeartbeat21/09/28 03:05:17 INFO DiskBlockManager: Created local directory at /tmp/blockmgr-33cfa046-2c7b-45b5-9cf1-c0f20195a98421/09/28 03:05:17 INFO MemoryStore: MemoryStore started with capacity 994.8 MiB21/09/28 03:05:17 INFO SparkEnv: Registering OutputCommitCoordinator21/09/28 03:05:17 INFO Utils: Successfully started service 'SparkUI' on port 4040.21/09/28 03:05:17 INFO SparkUI: Bound SparkUI to 0.0.0.0, and started at http://davben-lubuntu.home:404021/09/28 03:05:17 INFO RMProxy: Connecting to ResourceManager at /0.0.0.0:803221/09/28 03:05:18 INFO Client: Requesting a new application from cluster with 1 NodeManagers21/09/28 03:05:18 INFO Configuration: resource-types.xml not found21/09/28 03:05:18 INFO ResourceUtils: Unable to find 'resource-types.xml'.21/09/28 03:05:18 INFO Client: Verifying our application has not requested more than the maximum memory capability of the cluster (8192 MB per container)21/09/28 03:05:18 INFO Client: Will allocate AM container, with 896 MB memory including 384 MB overhead21/09/28 03:05:18 INFO Client: Setting up container launch context for our AM21/09/28 03:05:18 INFO Client: Setting up the launch environment for our AM container21/09/28 03:05:18 INFO Client: Preparing resources for our AM container21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/HikariCP-2.5.1.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/JLargeArrays-1.5.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/JTransforms-3.1.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/RoaringBitmap-0.9.0.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/ST4-4.0.4.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/accessors-smart-1.2.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/activation-1.1.1.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/aircompressor-0.10.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/algebra_2.12-2.0.0-M2.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/antlr-runtime-3.5.2.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/antlr4-runtime-4.8-1.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/aopalliance-1.0.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/aopalliance-repackaged-2.6.1.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/arpack_combined_all-0.1.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/arrow-format-2.0.0.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/arrow-memory-core-2.0.0.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/arrow-memory-netty-2.0.0.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/arrow-vector-2.0.0.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/audience-annotations-0.5.0.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/automaton-1.11-8.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/avro-1.8.2.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/avro-ipc-1.8.2.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/avro-mapred-1.8.2-hadoop2.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/bonecp-0.8.0.RELEASE.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/breeze-macros_2.12-1.0.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/breeze_2.12-1.0.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/cats-kernel_2.12-2.0.0-M4.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/chill-java-0.9.5.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/chill_2.12-0.9.5.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/commons-beanutils-1.9.4.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/commons-cli-1.2.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/commons-codec-1.10.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/commons-collections-3.2.2.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/commons-compiler-3.0.16.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/commons-compress-1.20.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/commons-configuration2-2.1.1.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/commons-crypto-1.1.0.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/commons-daemon-1.0.13.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/commons-dbcp-1.4.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/commons-httpclient-3.1.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/commons-io-2.5.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/commons-lang-2.6.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/commons-lang3-3.10.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/commons-logging-1.1.3.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/commons-math3-3.4.1.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/commons-net-3.1.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/commons-pool-1.5.4.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/commons-text-1.6.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/compress-lzf-1.0.3.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/core-1.1.2.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/curator-client-2.13.0.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/curator-framework-2.13.0.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/curator-recipes-2.13.0.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/datanucleus-api-jdo-4.2.4.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/datanucleus-core-4.1.17.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/datanucleus-rdbms-4.1.19.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/derby-10.12.1.1.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/dnsjava-2.1.7.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/dropwizard-metrics-hadoop-metrics2-reporter-0.1.2.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/ehcache-3.3.1.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/flatbuffers-java-1.9.0.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/generex-1.0.2.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/geronimo-jcache_1.0_spec-1.0-alpha-1.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/gson-2.2.4.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/guava-14.0.1.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/guice-4.0.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/guice-servlet-4.0.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/hadoop-annotations-3.2.0.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/hadoop-auth-3.2.0.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/hadoop-client-3.2.0.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/hadoop-common-3.2.0.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/hadoop-hdfs-client-3.2.0.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/hadoop-mapreduce-client-common-3.2.0.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/hadoop-mapreduce-client-core-3.2.0.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/hadoop-mapreduce-client-jobclient-3.2.0.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/hadoop-yarn-api-3.2.0.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/hadoop-yarn-client-3.2.0.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/hadoop-yarn-common-3.2.0.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/hadoop-yarn-registry-3.2.0.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/hadoop-yarn-server-common-3.2.0.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/hadoop-yarn-server-web-proxy-3.2.0.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/hive-beeline-2.3.7.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/hive-cli-2.3.7.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/hive-common-2.3.7.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/hive-exec-2.3.7-core.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/hive-jdbc-2.3.7.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/hive-llap-common-2.3.7.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/hive-metastore-2.3.7.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/hive-serde-2.3.7.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/hive-service-rpc-3.1.2.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/hive-shims-0.23-2.3.7.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/hive-shims-2.3.7.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/hive-shims-common-2.3.7.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/hive-shims-scheduler-2.3.7.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/hive-storage-api-2.7.2.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/hive-vector-code-gen-2.3.7.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/hk2-api-2.6.1.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/hk2-locator-2.6.1.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/hk2-utils-2.6.1.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/htrace-core4-4.1.0-incubating.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/httpclient-4.5.6.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/httpcore-4.4.12.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/istack-commons-runtime-3.0.8.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/ivy-2.4.0.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/jackson-annotations-2.10.0.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/jackson-core-2.10.0.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/jackson-core-asl-1.9.13.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/jackson-databind-2.10.0.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/jackson-dataformat-yaml-2.10.0.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/jackson-datatype-jsr310-2.11.2.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/jackson-jaxrs-base-2.9.5.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/jackson-jaxrs-json-provider-2.9.5.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/jackson-mapper-asl-1.9.13.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/jackson-module-jaxb-annotations-2.10.0.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/jackson-module-paranamer-2.10.0.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/jackson-module-scala_2.12-2.10.0.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/jakarta.activation-api-1.2.1.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/jakarta.annotation-api-1.3.5.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/jakarta.inject-2.6.1.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/jakarta.servlet-api-4.0.3.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/jakarta.validation-api-2.0.2.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/jakarta.ws.rs-api-2.1.6.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/jakarta.xml.bind-api-2.3.2.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/janino-3.0.16.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/javassist-3.25.0-GA.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/javax.inject-1.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/javax.jdo-3.2.0-m3.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/javolution-5.5.1.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/jaxb-api-2.2.11.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/jaxb-runtime-2.3.2.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/jcip-annotations-1.0-1.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/jcl-over-slf4j-1.7.30.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/jdo-api-3.0.1.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/jersey-client-2.30.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/jersey-common-2.30.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/jersey-container-servlet-2.30.jar21/09/28 03:05:18 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/jersey-container-servlet-core-2.30.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/jersey-hk2-2.30.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/jersey-media-jaxb-2.30.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/jersey-server-2.30.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/jline-2.14.6.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/joda-time-2.10.5.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/jodd-core-3.5.2.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/jpam-1.1.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/json-1.8.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/json-smart-2.3.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/json4s-ast_2.12-3.7.0-M5.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/json4s-core_2.12-3.7.0-M5.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/json4s-jackson_2.12-3.7.0-M5.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/json4s-scalap_2.12-3.7.0-M5.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/jsp-api-2.1.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/jsr305-3.0.0.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/jta-1.1.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/jul-to-slf4j-1.7.30.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/kerb-admin-1.0.1.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/kerb-client-1.0.1.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/kerb-common-1.0.1.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/kerb-core-1.0.1.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/kerb-crypto-1.0.1.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/kerb-identity-1.0.1.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/kerb-server-1.0.1.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/kerb-simplekdc-1.0.1.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/kerb-util-1.0.1.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/kerby-asn1-1.0.1.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/kerby-config-1.0.1.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/kerby-pkix-1.0.1.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/kerby-util-1.0.1.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/kerby-xdr-1.0.1.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/kryo-shaded-4.0.2.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/kubernetes-client-4.12.0.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/kubernetes-model-admissionregistration-4.12.0.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/kubernetes-model-apiextensions-4.12.0.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/kubernetes-model-apps-4.12.0.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/kubernetes-model-autoscaling-4.12.0.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/kubernetes-model-batch-4.12.0.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/kubernetes-model-certificates-4.12.0.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/kubernetes-model-common-4.12.0.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/kubernetes-model-coordination-4.12.0.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/kubernetes-model-core-4.12.0.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/kubernetes-model-discovery-4.12.0.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/kubernetes-model-events-4.12.0.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/kubernetes-model-extensions-4.12.0.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/kubernetes-model-metrics-4.12.0.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/kubernetes-model-networking-4.12.0.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/kubernetes-model-policy-4.12.0.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/kubernetes-model-rbac-4.12.0.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/kubernetes-model-scheduling-4.12.0.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/kubernetes-model-settings-4.12.0.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/kubernetes-model-storageclass-4.12.0.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/leveldbjni-all-1.8.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/libfb303-0.9.3.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/libthrift-0.12.0.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/log4j-1.2.17.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/logging-interceptor-3.12.12.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/lz4-java-1.7.1.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/machinist_2.12-0.6.8.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/macro-compat_2.12-1.1.1.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/mesos-1.4.0-shaded-protobuf.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/metrics-core-4.1.1.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/metrics-graphite-4.1.1.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/metrics-jmx-4.1.1.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/metrics-json-4.1.1.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/metrics-jvm-4.1.1.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/minlog-1.3.0.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/netty-all-4.1.51.Final.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/nimbus-jose-jwt-4.41.1.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/objenesis-2.6.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/okhttp-2.7.5.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/okhttp-3.12.12.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/okio-1.14.0.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/opencsv-2.3.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/orc-core-1.5.12.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/orc-mapreduce-1.5.12.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/orc-shims-1.5.12.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/oro-2.0.8.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/osgi-resource-locator-1.0.3.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/paranamer-2.8.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/parquet-column-1.10.1.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/parquet-common-1.10.1.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/parquet-encoding-1.10.1.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/parquet-format-2.4.0.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/parquet-hadoop-1.10.1.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/parquet-jackson-1.10.1.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/protobuf-java-2.5.0.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/py4j-0.10.9.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/pyrolite-4.30.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/re2j-1.1.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/scala-collection-compat_2.12-2.1.1.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/scala-compiler-2.12.10.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/scala-library-2.12.10.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/scala-parser-combinators_2.12-1.1.2.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/scala-reflect-2.12.10.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/scala-xml_2.12-1.2.0.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/shapeless_2.12-2.3.3.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/shims-0.9.0.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/slf4j-api-1.7.30.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/slf4j-log4j12-1.7.30.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/snakeyaml-1.24.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/snappy-java-1.1.8.2.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/spark-catalyst_2.12-3.1.2.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/spark-core_2.12-3.1.2.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/spark-graphx_2.12-3.1.2.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/spark-hive-thriftserver_2.12-3.1.2.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/spark-hive_2.12-3.1.2.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/spark-kubernetes_2.12-3.1.2.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/spark-kvstore_2.12-3.1.2.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/spark-launcher_2.12-3.1.2.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/spark-libs.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/spark-mesos_2.12-3.1.2.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/spark-mllib-local_2.12-3.1.2.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/spark-mllib_2.12-3.1.2.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/spark-network-common_2.12-3.1.2.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/spark-network-shuffle_2.12-3.1.2.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/spark-repl_2.12-3.1.2.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/spark-sketch_2.12-3.1.2.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/spark-sql_2.12-3.1.2.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/spark-streaming_2.12-3.1.2.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/spark-tags_2.12-3.1.2-tests.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/spark-tags_2.12-3.1.2.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/spark-unsafe_2.12-3.1.2.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/spark-yarn_2.12-3.1.2.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/spire-macros_2.12-0.17.0-M1.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/spire-platform_2.12-0.17.0-M1.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/spire-util_2.12-0.17.0-M1.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/spire_2.12-0.17.0-M1.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/stax-api-1.0.1.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/stax2-api-3.1.4.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/stream-2.9.6.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/super-csv-2.2.0.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/threeten-extra-1.5.0.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/token-provider-1.0.1.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/transaction-api-1.1.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/univocity-parsers-2.9.1.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/velocity-1.5.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/woodstox-core-5.0.3.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/xbean-asm7-shaded-4.15.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/xz-1.5.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/zjsonpatch-0.3.0.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/zookeeper-3.4.14.jar21/09/28 03:05:19 INFO Client: Source and destination file systems are the same. Not copying hdfs://localhost:9000/user/hadoop/davben/jars/zstd-jni-1.4.8-1.jar21/09/28 03:05:19 INFO Client: Uploading resource file:/tmp/spark-3b097b0a-1f72-4cb1-9dd1-f419cdbd645f/__spark_conf__17045972699920737203.zip -> hdfs://localhost:9000/user/hadoop/hadoop/.sparkStaging/application_1632774234019_0016/_spark_conf_.zip21/09/28 03:05:19 INFO SecurityManager: Changing view acls to: davben,hadoop21/09/28 03:05:19 INFO SecurityManager: Changing modify acls to: davben,hadoop21/09/28 03:05:19 INFO SecurityManager: Changing view acls groups to: 21/09/28 03:05:19 INFO SecurityManager: Changing modify acls groups to: 21/09/28 03:05:19 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users  with view permissions: Set(davben, hadoop); groups with view permissions: Set(); users  with modify permissions: Set(davben, hadoop); groups with modify permissions: Set()21/09/28 03:05:19 INFO Client: Submitting application application_1632774234019_0016 to ResourceManager21/09/28 03:05:19 INFO YarnClientImpl: Submitted application application_1632774234019_001621/09/28 03:05:20 INFO Client: Application report for application_1632774234019_0016 (state: ACCEPTED)21/09/28 03:05:20 INFO Client:  client token: N/A diagnostics: AM container is launched, waiting for AM container to Register with RM ApplicationMaster host: N/A ApplicationMaster RPC port: -1 queue: default start time: 1632791119751 final status: UNDEFINED tracking URL: http://davben-lubuntu:8088/proxy/application_1632774234019_0016/ user: hadoop21/09/28 03:05:21 INFO Client: Application report for application_1632774234019_0016 (state: ACCEPTED)21/09/28 03:05:22 INFO Client: Application report for application_1632774234019_0016 (state: ACCEPTED)21/09/28 03:05:23 INFO Client: Application report for application_1632774234019_0016 (state: RUNNING)21/09/28 03:05:23 INFO Client:  client token: N/A diagnostics: N/A ApplicationMaster host: 192.168.1.36 ApplicationMaster RPC port: -1 queue: default start time: 1632791119751 final status: UNDEFINED tracking URL: http://davben-lubuntu:8088/proxy/application_1632774234019_0016/ user: hadoop21/09/28 03:05:23 INFO YarnClientSchedulerBackend: Application application_1632774234019_0016 has started running.21/09/28 03:05:23 INFO Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 33377.21/09/28 03:05:23 INFO NettyBlockTransferService: Server created on davben-lubuntu.home:3337721/09/28 03:05:23 INFO BlockManager: Using org.apache.spark.storage.RandomBlockReplicationPolicy for block replication policy21/09/28 03:05:23 INFO BlockManagerMaster: Registering BlockManager BlockManagerId(driver, davben-lubuntu.home, 33377, None)21/09/28 03:05:23 INFO BlockManagerMasterEndpoint: Registering block manager davben-lubuntu.home:33377 with 994.8 MiB RAM, BlockManagerId(driver, davben-lubuntu.home, 33377, None)21/09/28 03:05:23 INFO BlockManagerMaster: Registered BlockManager BlockManagerId(driver, davben-lubuntu.home, 33377, None)21/09/28 03:05:23 INFO BlockManager: external shuffle service port = 733721/09/28 03:05:23 INFO BlockManager: Initialized BlockManager: BlockManagerId(driver, davben-lubuntu.home, 33377, None)21/09/28 03:05:23 INFO YarnClientSchedulerBackend: Add WebUI Filter. org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter, Map(PROXY_HOSTS -> davben-lubuntu, PROXY_URI_BASES -> http://davben-lubuntu:8088/proxy/application_1632774234019_0016), /proxy/application_1632774234019_001621/09/28 03:05:24 INFO ServerInfo: Adding filter to /metrics/json: org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter21/09/28 03:05:25 INFO YarnSchedulerBackend$YarnSchedulerEndpoint: ApplicationMaster registered as NettyRpcEndpointRef(spark-client://YarnAM)21/09/28 03:05:47 INFO YarnClientSchedulerBackend: SchedulerBackend is ready for scheduling beginning after waiting maxRegisteredResourcesWaitingTime: 30000000000(ns)21/09/28 03:05:49 INFO SharedState: Setting hive.metastore.warehouse.dir ('null') to the value of spark.sql.warehouse.dir ('file:/home/davben/prometheux/projects/spark-reasoner/spark-warehouse').21/09/28 03:05:49 INFO SharedState: Warehouse path is 'file:/home/davben/prometheux/projects/spark-reasoner/spark-warehouse'.21/09/28 03:05:49 INFO ServerInfo: Adding filter to /SQL: org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter21/09/28 03:05:49 INFO ServerInfo: Adding filter to /SQL/json: org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter21/09/28 03:05:49 INFO ServerInfo: Adding filter to /SQL/execution: org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter21/09/28 03:05:49 INFO ServerInfo: Adding filter to /SQL/execution/json: org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter21/09/28 03:05:49 INFO ServerInfo: Adding filter to /static/sql: org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter21/09/28 03:05:50 INFO CodeGenerator: Code generated in 169.2098 ms21/09/28 03:05:50 INFO CodeGenerator: Code generated in 13.789217 ms21/09/28 03:05:51 INFO CodeGenerator: Code generated in 17.287752 ms

----------+

edge_1

edge_2

----------+|     

{{a|     b||    }}

a|     a|

----------+
SHOW21/09/28 03:05:51 INFO CodeGenerator: Code generated in 45.072695 ms21/09/28 03:05:51 INFO SparkContext: Starting job: show at TrivialTests.java:124621/09/28 03:05:51 INFO DAGScheduler: Got job 0 (show at TrivialTests.java:1246) with 1 output partitions21/09/28 03:05:51 INFO DAGScheduler: Final stage: ResultStage 0 (show at TrivialTests.java:1246)21/09/28 03:05:51 INFO DAGScheduler: Parents of final stage: List()21/09/28 03:05:51 INFO DAGScheduler: Missing parents: List()21/09/28 03:05:51 INFO DAGScheduler: Submitting ResultStage 0 (MapPartitionsRDD[2] at show at TrivialTests.java:1246), which has no missing parents21/09/28 03:05:51 INFO MemoryStore: Block broadcast_0 stored as values in memory (estimated size 13.3 KiB, free 994.8 MiB)21/09/28 03:05:51 INFO MemoryStore: Block broadcast_0_piece0 stored as bytes in memory (estimated size 5.3 KiB, free 994.8 MiB)21/09/28 03:05:51 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on davben-lubuntu.home:33377 (size: 5.3 KiB, free: 994.8 MiB)21/09/28 03:05:51 INFO SparkContext: Created broadcast 0 from broadcast at DAGScheduler.scala:138821/09/28 03:05:51 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 0 (MapPartitionsRDD[2] at show at TrivialTests.java:1246) (first 15 tasks are for partitions Vector(0))21/09/28 03:05:51 INFO YarnScheduler: Adding task set 0.0 with 1 tasks resource profile 021/09/28 03:06:06 WARN YarnScheduler: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources21/09/28 03:06:21 WARN YarnScheduler: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources21/09/28 03:06:36 WARN YarnScheduler: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources21/09/28 03:06:51 WARN YarnScheduler: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources

It is strange because if the same job is run locally (setMaster(local[*])) all works properly.