zenoss / zing-hdfs Goto Github PK

Meant to standup a Hadoop 2.7.3 HDFS HA on multiple machines inside Docker containers. On the "active" namenode, the 1st time it starts up, it will format the namenode as well as format the ZK for failover. The "standby" namenode will bootStrapStandby.

• active: Start the 1st namenode with a zkfc service.
• standby: Start the 2nd namenode and bootstrap from an already formatted/running namenode. This will also start a zkfc service.
• zkfc: Start the zkfc in as a separate service
• journalnode: Start a journalnode
• datanode: Starts a datanode
• bash: allows you to jump in and check things out

CLUSTER_NAME: the HDFS URI default filesystem name

NNODE1_IP: Namenode #1 IP/hostname

NNODE2_IP: Namenode #2 IP/hostname

JNODES: semicolon separated list of journal node IPS, e.g jn01:8485;jn02:8485;jn03:8485

ZK_NODES: comma separated list of zookeeper IPS, e.g. zk01:2181,zk02:2181,zk03:2181

/data/hdfs/nn: inside the container, where the fsimage/namenode metadata exists

/data/hdfs/journal: inside the container, where the journal node keeps the edits

/data/jdfs/dn: inside the container, where the datanode keeps the blocks

• docker run --name nn01 --net=host --env-file hdfs-envs --env NNODE_ID=nn1 hdfs active