bdutil - on server stop/start - hdfs not able to find fs about bdutil HOT 6 CLOSED

I can try to repro this. While bdutil doesn't have built-in commands for "stopping" and "starting" a cluster, there are various flavors of stopping/starting which are indeed supported.

How did you "stop" and "restart" your cluster? Did you only stop/restart your master, or did you do it on every node? Are you using Local SSDs, and are you using USE_ATTACHED_PDS?

from bdutil.

This example was based on single node using the following command:
./bdutil deploy -e hadoop2_env.sh --bucket anintelcluster -e single_node_env.sh -e spark_env.sh -P anintelcluster
All I did was top the single node. However, when I bring it up, even though the hadoop and spark services are running, my application is not able to access them via configured URL e.g. hadoop URL is hdfs://anintelcluster:8020.
The way I stopped was using the developer console - Compute>VM>select VM>Stop/Start.
Re: HDD - I use whatever is default for the bdutil script - no changes to that.

from bdutil.

Alright, I did manage to repro. There might be some race-condition that's more specific to a single-node environment during startup. I was able to repair HDFS simply by running:

/home/hadoop/hadoop-install/sbin/stop-all.sh
/home/hadoop/hadoop-install/sbin/start-all.sh

When I did that I was able to access my old data inside HDFS again that I added prior to the stop/start. I'll check on why the automatic restart didn't work.

EDIT: Forgot to mention, you have to run those scripts as user hadoop, so first run sudo su hadoop.

from bdutil.

Alright, tracked down the bug to configure_startup_processes.sh and spark_configure_startup_processes.sh not correctly adding worker daemons in the case of single-node clusters. This bug only affects clusters which use single_node_env.sh. I've successfully tested a fix and sent a pull request which we'll merge shortly: #42

You can optionally clone from my fork in the meantime to check it out: https://github.com/dennishuo/bdutil

Or it should it make it into the master within a couple days.

from bdutil.

#42 has been merged, issue should be fixed now if you use bdutil at head revision.

from bdutil.

Confirm bug fixed ..

from bdutil.