This terraform module creates an Azure Kubernetes Service and its associated Azure Application Gateway as ingress controller.
Inside the cluster default node pool, velero and cert-manager are installed.
Inside each node pool, Kured is installed as a daemonset.
This module also configures logging to a Log Analytics Workspace, deploys the Azure Active Directory Pod Identity and creates some Storage Classes with different types of Azure managed disks (Standard HDD retain and delete, Premium SSD retain and delete).
- Azurerm Terraform provider >= 2.10.0
- Helm Terraform provider >= 1.1.1
- Kubernetes Terraform provider >= 1.11.1
- Kubectl command
- A Microsoft.Storage service endpoint into the nodes subnet
Module version | Terraform version | AzureRM version |
---|---|---|
>= 4.x.x | 0.13.x | >= 2.10.0 |
>= 3.x.x | 0.12.x | >= 2.10.0 |
>= 2.x.x | 0.12.x | < 2.0 |
< 2.x.x | 0.11.x | < 2.0 |
This module is optimized to work with the Claranet terraform-wrapper too which set some terraform variables in the environment needed by this module.
More details about variables set by the terraform wrapper
available in the documentation.
You can use this module by including it this way:
locals {
allowed_cidr = ["x.x.x.x", "y.y.y.y"]
}
module "azure-region" {
source = "claranet/regions/azurerm"
version = "x.x.x"
azure_region = var.azure_region
}
module "rg" {
source = "claranet/rg/azurerm"
version = "x.x.x"
location = module.azure-region.location
client_name = var.client_name
environment = var.environment
stack = var.stack
}
module "azure-virtual-network" {
source = "claranet/vnet/azurerm"
version = "x.x.x"
environment = var.environment
location = module.azure-region.location
location_short = module.azure-region.location_short
client_name = var.client_name
stack = var.stack
resource_group_name = module.rg.resource_group_name
vnet_cidr = ["10.0.0.0/19"]
}
module "azure-network-subnet" {
source = "claranet/subnet/azurerm"
version = "x.x.x"
environment = var.environment
location_short = module.azure-region.location_short
client_name = var.client_name
stack = var.stack
resource_group_name = module.rg.resource_group_name
virtual_network_name = module.azure-virtual-network.virtual_network_name
subnet_cidr_list = ["10.0.0.0/20", "10.0.20.0/24"]
service_endpoints = ["Microsoft.Storage"]
}
module "global_run" {
source = "claranet/run-common/azurerm"
version = "x.x.x"
client_name = var.client_name
location = module.azure-region.location
location_short = module.azure-region.location_short
environment = var.environment
stack = var.stack
resource_group_name = module.rg.resource_group_name
tenant_id = var.azure_tenant_id
}
module "aks" {
source = "claranet/aks/azurerm"
version = "x.x.x"
client_name = var.client_name
environment = var.environment
stack = var.stack
resource_group_name = module.rg.resource_group_name
location = module.azure-region.location
location_short = module.azure-region.location_short
service_cidr = "10.0.16.0/22"
kubernetes_version = "1.19.7"
vnet_id = module.azure-virtual-network.virtual_network_id
nodes_subnet_id = module.azure-network-subnet.subnet_ids[0]
nodes_pools = [
{
name = "pool1"
count = 1
vm_size = "Standard_D1_v2"
os_type = "Linux"
os_disk_size_gb = 30
vnet_subnet_id = module.azure-network-subnet.subnet_ids[0]
},
{
name = "bigpool1"
count = 3
vm_size = "Standard_F8s_v2"
os_type = "Linux"
os_disk_size_gb = 30
vnet_subnet_id = module.azure-network-subnet.subnet_ids[0]
enable_auto_scaling = true
min_count = 3
max_count = 9
}
]
linux_profile = {
username = "user"
ssh_key = file("~/.ssh/id_rsa.pub")
}
addons = {
dashboard = false
oms_agent = true
oms_agent_workspace_id = var.log_analytic_workspace_id
policy = false
}
diagnostic_settings_logs_destination_ids = [var.log_analytic_workspace_id]
appgw_subnet_id = module.azure-network-subnet.subnet_ids[1]
appgw_ingress_controller_values = { "verbosityLevel" = "5", "appgw.shared" = "true" }
cert_manager_settings = { "cainjector.nodeSelector.agentpool" = "default", "nodeSelector.agentpool" = "default", "webhook.nodeSelector.agentpool" = "default" }
velero_storage_settings = { allowed_cidrs = local.allowed_cidrs }
}
module "acr" {
source = "claranet/acr/azurerm"
version = "x.x.x"
location = module.azure-region.location
location_short = module.azure-region.location_short
resource_group_name = module.rg.resource_group_name
sku = "Standard"
client_name = var.client_name
environment = var.environment
stack = var.stack
}
resource "azurerm_role_assignment" "allow_ACR" {
principal_id = module.aks.aks_user_managed_identity.0.object_id
scope = module.acr.acr_id
role_definition_name = "AcrPull"
}
Name | Description | Type | Default | Required |
---|---|---|---|---|
aadpodidentity_chart_repository | AAD Pod Identity Helm chart repository URL | string |
"https://vmware-tanzu.github.io/helm-charts" |
no |
aadpodidentity_chart_version | AAD Pod Identity helm chart version to use | string |
"2.0.0" |
no |
aadpodidentity_namespace | Kubernetes namespace in which to deploy AAD Pod Identity | string |
"system-aadpodid" |
no |
aadpodidentity_values | Settings for AAD Pod identity helm Chart:map(object({ |
map(string) |
{} |
no |
addons | Kubernetes addons to enable /disable | object({ |
{ |
no |
agic_chart_repository | Helm chart repository URL | string |
"https://appgwingress.blob.core.windows.net/ingress-azure-helm-package/" |
no |
agic_chart_version | Version of the Helm chart | string |
"1.2.0" |
no |
agic_helm_version | [DEPRECATED] Version of Helm chart to deploy | string |
null |
no |
api_server_authorized_ip_ranges | Ip ranges allowed to interract with Kubernetes API. Default no restrictions | list(string) |
[] |
no |
appgw_ingress_controller_values | Application Gateway Ingress Controller settings | map(string) |
{} |
no |
appgw_private_ip | Private IP for Application Gateway. Used when variable private_ingress is set to true . |
string |
null |
no |
appgw_settings | Application gateway configuration settings. Default dummy configuration | map(any) |
{} |
no |
appgw_subnet_id | Application gateway subnet id | string |
"" |
no |
cert_manager_chart_repository | Helm chart repository URL | string |
"https://charts.jetstack.io" |
no |
cert_manager_chart_version | Cert Manager helm chart version to use | string |
"v0.13.0" |
no |
cert_manager_namespace | Kubernetes namespace in which to deploy Cert Manager | string |
"system-cert-manager" |
no |
cert_manager_settings | Settings for cert-manager helm chart | map(string) |
{} |
no |
client_name | Client name/account used in naming | string |
n/a | yes |
container_registries | List of Azure Container Registries ids where AKS needs pull access. | list(string) |
[] |
no |
custom_aks_name | Custom AKS name | string |
"" |
no |
custom_appgw_name | Custom name for AKS ingress application gateway | string |
"" |
no |
default_node_pool | Default node pool configuration:map(object({ |
map(any) |
{} |
no |
diagnostic_settings_event_hub_name | Event hub name used with diagnostics settings | string |
null |
no |
diagnostic_settings_log_analytics_destination_type | When set to 'Dedicated' logs sent to a Log Analytics workspace will go into resource specific tables, instead of the legacy AzureDiagnostics table. This only includes Azure Data Factory | string |
"AzureDiagnostics" |
no |
diagnostic_settings_log_categories | List of log categories | list(string) |
null |
no |
diagnostic_settings_logs_destination_ids | List of destination resources IDs for logs diagnostic destination. Can be Storage Account, Log Analytics Workspace and Event Hub. No more than one of each can be set. | list(string) |
null |
no |
diagnostic_settings_metric_categories | List of metric categories | list(string) |
null |
no |
diagnostic_settings_retention_days | The number of days to keep diagnostic logs. | number |
30 |
no |
diagnostic_settings_custom_name | Custom name for Azure Diagnostics for AKS. | string |
"default" |
no |
docker_bridge_cidr | IP address for docker with Network CIDR. | string |
"172.16.0.1/16" |
no |
enable_agic | Enable Application gateway ingress controller | bool |
true |
no |
enable_cert_manager | Enable cert-manager on AKS cluster | bool |
true |
no |
enable_kured | Enable kured daemon on AKS cluster | bool |
true |
no |
enable_pod_security_policy | Enable pod security policy or not. https://docs.microsoft.com/fr-fr/azure/AKS/use-pod-security-policies | bool |
false |
no |
enable_velero | Enable velero on AKS cluster | bool |
true |
no |
environment | Project environment | string |
n/a | yes |
extra_tags | Extra tags to add | map(string) |
{} |
no |
kubernetes_version | Version of Kubernetes to deploy | string |
"1.17.9" |
no |
kured_chart_repository | Helm chart repository URL | string |
"https://weaveworks.github.io/kured" |
no |
kured_chart_version | Version of the Helm chart | string |
"1.5.0" |
no |
kured_settings | Settings for kured helm chart:map(object({ |
map(string) |
{} |
no |
linux_profile | Username and ssh key for accessing AKS Linux nodes with ssh. | object({ |
null |
no |
location | Azure region to use | string |
n/a | yes |
location_short | Short name of Azure regions to use | string |
n/a | yes |
managed_identities | List of managed identities where the AKS service principal should have access. | list(string) |
[] |
no |
name_prefix | Prefix used in naming | string |
"" |
no |
node_resource_group | Name of the resource group in which to put AKS nodes. If null default to MC_ | string |
null |
no |
nodes_pools | A list of nodes pools to create, each item supports same properties as local.default_agent_profile |
list(any) |
n/a | yes |
nodes_subnet_id | Id of the subnet used for nodes | string |
n/a | yes |
outbound_type | The outbound (egress) routing method which should be used for this Kubernetes Cluster. Possible values are loadBalancer and userDefinedRouting . |
string |
"loadBalancer" |
no |
private_ingress | Private ingress boolean variable. When true , the default http listener will listen on private IP instead of the public IP. |
bool |
false |
no |
resource_group_name | Name of the AKS resource group | string |
n/a | yes |
service_cidr | CIDR used by kubernetes services (kubectl get svc). | string |
n/a | yes |
stack | Project stack name | string |
n/a | yes |
storage_contributor | List of storage accounts ids where the AKS service principal should have access. | list(string) |
[] |
no |
velero_chart_repository | URL of the Helm chart repository | string |
"https://vmware-tanzu.github.io/helm-charts" |
no |
velero_chart_version | Velero helm chart version to use | string |
"2.12.13" |
no |
velero_namespace | Kubernetes namespace in which to deploy Velero | string |
"system-velero" |
no |
velero_storage_settings | Settings for Storage account and blob container for Veleromap(object({ |
map(any) |
{} |
no |
velero_values | Settings for Velero helm chart:map(object({ |
map(string) |
{} |
no |
vnet_id | Id of the vnet used for AKS | string |
n/a | yes |
Name | Description |
---|---|
aad_pod_identity_azure_identity | Identity object for AAD Pod Identity |
aad_pod_identity_namespace | Namespace used for AAD Pod Identity |
agic_namespace | Namespace used for AGIC |
aks_id | AKS resource id |
aks_kube_config | Kube configuration of AKS Cluster |
aks_kube_config_raw | Raw kube config to be used by kubectl command |
aks_name | Name of the AKS cluster |
aks_nodes_pools_ids | Ids of AKS nodes pools |
aks_nodes_pools_names | Names of AKS nodes pools |
aks_nodes_rg | Name of the resource group in which AKS nodes are deployed |
aks_user_managed_identity | The User Managed Identity used by AKS Agents |
application_gateway_id | Id of the application gateway used by AKS |
application_gateway_name | Name of the application gateway used by AKS |
cert_manager_namespace | Namespace used for Cert Manager |
kured_namespace | Namespace used for Kured |
public_ip_id | Id of the public ip used by AKS application gateway |
public_ip_name | Name of the public ip used by AKS application gateway |
velero_identity | Azure Identity used for Velero pods |
velero_namespace | Namespace used for Velero |
velero_storage_account | Storage Account on which Velero data is stored. |
velero_storage_account_container | Container in Storage Account on which Velero data is stored. |
- Azure Kubernetes Service documentation : docs.microsoft.com/en-us/azure/aks/
- Azure Kubernetes Service MSI Usage : docs.microsoft.com/en-us/azure/aks/use-managed-identity
- Azure Kubernetes Service User-Defined Route usage : docs.microsoft.com/en-us/azure/aks/egress-outboundtype
- Terraform AKS resource documentation: www.terraform.io/docs/providers/azurerm/r/kubernetes_cluster.html
- Terraform AKS Node pool resource documentation: www.terraform.io/docs/providers/azurerm/r/kubernetes_cluster_node_pool.html
- Terraform Kubernetes provider documentation: www.terraform.io/docs/providers/kubernetes/index.html
- Terraform Helm provider documentation: www.terraform.io/docs/providers/helm/index.html
- Kured documentation: github.com/weaveworks/kured
- Velero documentation: velero.io/docs/v1.2.0/
- Velero Azure specific documentation: github.com/vmware-tanzu/velero-plugin-for-microsoft-azure
- cert-manager documentation : cert-manager.io/docs/