Support running on yarn cluster about ispark HOT 6 CLOSED

It should be running smoothly without extra effort (just change the
ipython profile to submit it to yarn), have you tried it?

Yours Peng

On 02/13/2015 12:57 AM, Lev wrote:

Hi,

are there plans to support yarn cluster?

—
Reply to this email directly or view it on GitHub
#9.

from ispark.

I did,
and I managed to run it,
but it would fail after being submitted to yarn.

I will recreate this and post the error I got

Thanks,
Lev.

from ispark.

Hi all, I'm really interested about this issue. I would like to be able to do what I'm doing now with "spark-submit --master yarn-cluster" with ISpark. Is there any possibility to configure ISpark to submit operation in cluster mode?
Cheers,
Luca

from ispark.

Sorry for the late reply.
The original Spark-shell and all derivative projects (including Zeppelin) cannot be run in cluster mode. Client mode is the only option (if you try you'll get an error message, furthermore, how could you interact with a driver that is randomly dispatched?)

from ispark.

Thank you very much for the answer!

from ispark.

You are welcome. This will be marked as 'solved', I'm still not sure if running on YARN is bug-free, but please open a new ticket in that case.

from ispark.