Add an additional Data Node about azure-marketplace HOT 1 CLOSED

Using the portal UI, it is not possible to add another data node to an existing Elasticsearch cluster. You used to be able to deploy to an existing resource group from within the portal UI but this feature was removed by Microsoft, so the portal UI only allows deployments into a new resource group.

The deployment template itself however does allow you to deploy into an existing resource group, so using the Azure CLI command line tools, it is possible to scale up (or down) an existing Elasticsearch cluster. This scaling mechanism however is not intended to be a foolhardy solution but can be an easy way to add additional nodes to an existing cluster, and tends to work best when you have dedicated master nodes and simply wish to add additional data nodes. For a cluster in production, I would recommend developing automation with something like puppet to suit your organisational requirements.

Some important points to highlight for how scaling with the template works:

The ARM template deploys in incremental mode meaning that it

• leaves unchanged resources that exist in the resource group but are not specified in the template
• adds resources that are specified in the template but do not exist in the resource group
• does not re-provision resources that exist in the resource group in the same condition defined in the template
• re-provisions existing resources that have updated settings in the template

so when you use the deployment template to add an additional data node, be sure to specify all of the same settings as you did when you originally deployed the template, changing only the number of data nodes.

If you are not using dedicated master nodes i.e. you have master eligible data nodes only, the template should update the elasticsearch yaml configuration file on each existing and new data node to set discovery.zen.minimum_master_nodes to a quorum of the number of master eligible nodes. I would recommend checking that this is the case after deployment has finished by running

sudo grep discovery.zen.minimum_master_nodes /etc/elasticsearch/elasticsearch.yml

The number assigned to discovery.zen.minimum_master_nodes should be the same across all nodes and be equal to (number of master eligible nodes / 2) + 1 so for 3 master eligible nodes, discovery.zen.minimum_master_nodes should be 2.

Similarly, discovery.zen.ping.unicast.hosts should also be updated on each node and be seeded with the host names of either the master nodes if using dedicated master nodes, or the host names of all master eligible nodes.

from azure-marketplace.