From @sabbyanandan on May 6, 2016 22:34

From @rejurs on April 29, 2016 20:9

Hi Team,

Could you please help me to pass the spark-defaults.conf values when we are running java -jar sparkapp-yarn/target/sparkapp-yarn-task-1.0.0.BUILD-SNAPSHOT-exec.jar.

Copied from original issue: spring-cloud/spring-cloud-task-modules#36

Copied from original issue: spring-cloud/spring-cloud-task-app-starters#18

Hi, I'm running spring-cloud-dataflow-server-yarn-1.2.0.RELEASE server in docker cloudera container with SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904-el6.parcel and hadoop-2.6.0-cdh5.11.1.

Server starts without any issues, but when I'am submitting spark task it fails with:
Unrecognized option: --spark.app-jar=hdfs://app/spark/basic-spark-0.1-SNAPSHOT-jar-with-dependencies.jar Error: Could not create the Java Virtual Machine. Error: A fatal exception has occurred. Program will exit.

I'm registering app and creating task using:

app register --type task --name spark-yarn --uri hdfs:///app/spark/spring-cloud-starter-task-spark-yarn-1.2.0.RELEASE.jar

task create spark --definition "spark-yarn --spark.app-name=my-test-pi --spark.app-class=app.SparkESIngester --spark.app-jar=hdfs://app/spark/basic-spark-0.1-SNAPSHOT-jar-with-dependencies.jar --sark.assembly-jar=hdfs://app/spark/spark-assembly-1.6.1-hadoop2.6.0.jar --spark.app-args=10"

task launch spark

If anyone is able to provide some assistance would me immensely appreciated, as nothing on the web makes sense to resolve this and get the spark task going!!!

Thank you!

Fails fast on startup:

$ java -jar spark-yarn-task-1.1.0.RELEASE.jar
...
2017-02-06 15:56:02.682 ERROR 6683 --- [           main] o.s.boot.SpringApplication               : Application startup failed

org.springframework.context.ApplicationContextException: Unable to start embedded container; nested exception is org.springframework.context.ApplicationContextException: Unable to start EmbeddedWebApplicationContext due to missing EmbeddedServletContainerFactory bean.
	at org.springframework.boot.context.embedded.EmbeddedWebApplicationContext.onRefresh(EmbeddedWebApplicationContext.java:137) ~[spring-boot-1.4.1.RELEASE.jar!/:1.4.1.RELEASE]
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:535) ~[spring-context-4.3.3.RELEASE.jar!/:4.3.3.RELEASE]
	at org.springframework.boot.context.embedded.EmbeddedWebApplicationContext.refresh(EmbeddedWebApplicationContext.java:122) ~[spring-boot-1.4.1.RELEASE.jar!/:1.4.1.RELEASE]
	at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:761) [spring-boot-1.4.1.RELEASE.jar!/:1.4.1.RELEASE]