Azure Kubernetes Service (AKS) CNI networking overview

Kubernetes uses Container Networking Interface (CNI) plugins to manage networking in Kubernetes clusters. CNI plugins handle assigning IP addresses to pods, routing network traffic between pods, routing Kubernetes service traffic, and more.

Azure Kubernetes Service (AKS) provides multiple CNI networking configurations that you can use in your clusters, depending on your networking requirements. When you plan pod networking, you choose an IP address management (IPAM) option and a routing and transport technology for the network data plane.

Networking models in AKS

Choosing an IPAM option for your AKS cluster largely depends on which networking model fits your needs best. Each model has its own advantages and disadvantages that you should consider when planning your AKS cluster.

AKS uses two main networking models:

• Overlay network:

  • Conserves IP address space for virtual networks (VNets) by using logically separate Classless Inter-Domain Routing (CIDR) ranges for pods.
  • Provides maximum cluster scale support.
  • Provides simple management of IP addresses.

• Flat network:

  • Provides full VNet connectivity for pods. Pods can be directly reached via their private IP address from connected networks.
  • Requires large, non-fragmented IP address space for VNets.

Both networking models support multiple IPAM options. The main differences between the models are how you assign pod IP addresses and how traffic leaves the cluster.

For Azure CNI, the IPAM option is separate from the network data plane. You can use the Azure CNI Powered by Cilium data plane with Azure CNI Overlay, Azure CNI Pod Subnet, or Azure CNI Node Subnet. For more information about IPAM and data-plane options, see Plan pod networking for AKS.

Overlay networks

Overlay networking in AKS assigns pod IP addresses from a separate pod CIDR that's distinct from the node subnet in the VNet. This configuration allows for simpler and often better scalability than the flat network model.

In overlay networks, pods can communicate with each other directly. Traffic that leaves the cluster is Source Network Address Translated (SNAT'd) to the node's IP address. Inbound pod IP traffic is routed through a service, such as a load balancer. The pod IP address is then "hidden" behind the node's IP address. This approach reduces the number of IP addresses required for virtual networks in your clusters.

For overlay networking, AKS provides Azure CNI Overlay. Use this IPAM option for most scenarios.

Flat networks

Unlike an overlay network, a flat network model in AKS assigns IP addresses to pods from a subnet in the same Azure virtual network as the AKS nodes. For private network traffic, the source IP address that a destination sees depends on the IPAM option. Azure CNI Pod Subnet preserves the pod IP address across connected virtual networks. With Azure CNI Node Subnet, destinations in the cluster virtual network see the pod IP address, but destinations outside the cluster virtual network see the node IP address. When internet egress is enabled, the cluster's configured outbound method determines the public source IP address that internet destinations see.

AKS provides two Azure CNI IPAM options for flat networking:

• Azure CNI Pod Subnet, the recommended IPAM option for flat networking scenarios.
• Azure CNI Node Subnet, a legacy CNI model for flat networks. In general, we recommend that you use it only if you need a managed virtual network for your cluster.

Choose an IPAM option for AKS

When choosing an IPAM option, consider several factors. Each networking model has its own advantages and disadvantages. The best choice for your cluster depends on your specific requirements.

Use case comparison

Feature comparison

Support scope between network models

Depending on the IPAM option that you use, you can deploy the virtual network resources for your cluster in one of the following ways:

• The Azure platform can automatically create and configure the virtual network resources when you create an AKS cluster.
• You can manually create and configure the virtual network resources and attach to those resources when you create your AKS cluster.

Although capabilities like service endpoints or UDRs are supported, the support policies for AKS define what changes you can make. For example:

• If you manually create the virtual network resources for an AKS cluster, you're supported when configuring your own UDRs or service endpoints.
• If the Azure platform automatically creates the virtual network resources for your AKS cluster, you can't manually change those AKS-managed resources to configure your own UDRs or service endpoints.

AKS CNI networking prerequisites

When you're planning your network configuration for AKS, keep these requirements and considerations in mind:

• Unless you use a network isolated cluster, the virtual network for the AKS cluster must allow outbound internet connectivity to required endpoints. Network isolated clusters can bootstrap without outbound internet connectivity.

• AKS address ranges have the following restrictions:

  Reserved CIDR range	Applies to	Condition
  169.254.0.0/16	Kubernetes service, pod, and cluster virtual network address ranges	All AKS clusters
  192.0.2.0/24	Kubernetes service, pod, and cluster virtual network address ranges	All AKS clusters
  172.30.0.0/16	Kubernetes service, pod, and cluster virtual network address ranges	All AKS clusters
  172.31.0.0/16	Kubernetes service, pod, and cluster virtual network address ranges	All AKS clusters

  AKS rejects pod CIDRs that overlap a reserved range during cluster creation or update. For example, 172.16.0.0/12 isn't valid because the range includes 172.30.0.0/16 and 172.31.0.0/16.

• In scenarios where you bring your own virtual network, the cluster identity that the AKS cluster uses must have at least Network Contributor permissions on the subnet within your virtual network.

• If you define a custom role instead of using the built-in Network Contributor role, include the following permissions:

  Permission	When required
  Microsoft.Network/virtualNetworks/subnets/join/action	Always when using a custom role
  Microsoft.Authorization/roleAssignments/write	Always when using a custom role
  Microsoft.Network/virtualNetworks/subnets/read	Only when defining your own subnets and CIDRs

• The subnet assigned to the AKS node pool can't be a delegated subnet.

• AKS doesn't apply network security groups (NSGs) to its subnet and doesn't modify any of the NSGs associated with that subnet. If you provide your own subnet and add NSGs associated with that subnet, you must ensure that the security rules in the NSGs allow traffic within the node CIDR range. With Azure CNI Overlay, if an NSG deny rule affects pod CIDR traffic, you must also allow traffic from the node CIDR to the pod CIDR and from the pod CIDR to the pod CIDR on all ports and protocols. For more information, see Network security groups with Azure CNI Overlay.

• Container Network Service (CNS) is a node local service within Azure Kubernetes Service CNI networking which allocates, tracks and programs networking for pods. This component sends telemetry (metrics and logs) by default out of the cluster to a Microsoft managed Application Insights endpoint to enable faster troubleshooting of any pod networking IP address assignment issues.

Related content

• Azure CNI Overlay
• Azure CNI Pod Subnet
• Legacy CNI options
• IP address planning for your clusters