[As of March 31, 2021]

Objectives

Ingress controller of Azure Kubernetes Service (AKS) can allow ingress traffic from specified public IP addresses, but how about egress traffic? Without any configuration, source public IP addresses of egress traffic from AKS are chosen at random. This behavior is not good for some API providers, because they would like to allow only incoming traffic from specified public IP addresses. In this entry, I summarized three ways to restrict IP addresses used for egress traffic from AKS.

1. Azure Firewall

Control egress traffic for cluster nodes in Azure Kubernetes Service (AKS)
https://docs.microsoft.com/azure/aks/limit-egress-traffic

This topology is similar to an environment where proxy is located on premises for internet access. As you see the diagram below, egress traffic are routed to subnet of firewall with UDR (user defined route). In this case, we should not block any internal subnet traffic within AKS cluster.

We have to pay attention to blocking internal subnet traffic.

Blocking internal subnet traffic using network security groups (NSGs) and firewalls is not supported. To control and block the traffic within the cluster, use Network Policies.

https://docs.microsoft.com/azure/aks/limit-egress-traffic

2. Standard/Basic Load Balancer

In this case, you configure outbound rules for load balancer.

Provide your own public IPs or prefixes for egress
https://docs.microsoft.com/azure/aks/load-balancer-standard#provide-your-own-public-ips-or-prefixes-for-egress
Use a static public IP address for egress traffic with a Basic SKU load balancer in Azure Kubernetes Service (AKS)
https://docs.microsoft.com/azure/aks/egress

3. Virtual Network NAT

NAT gateway was introduced in March 2020 and we can deploy NAT gateway onto subnet. NAT gateway can have at least one public IP address. When a pod sends egress traffic, one of public IP assigned to NAT gateway is used as source IP.

What is Virtual Network NAT?
https://docs.microsoft.com/azure/virtual-network/nat-overview
Designing virtual networks with NAT gateway resources
https://docs.microsoft.com/azure/virtual-network/nat-gateway-resource

Typically, each node pool, which consists of VMSS (Virtual Machine Scale Sets), is connected to single subnet. However, we can assign unique subnet to each node pool (preview). When assigning NAT gateway to each subnet, a pod running on each node pool uses different source IP.

Add a node pool with a unique subnet (preview)
https://docs.microsoft.com/azure/aks/use-multiple-node-pools#add-a-node-pool-with-a-unique-subnet-preview

4. Assign Public IP to each node in node pool

We can also assign public IP to each node in node pool (preview). When a pod running on a node sends egress traffic, public IP assigned to the node is used as source IP.

Assign a public IP per node for your node pools
https://docs.microsoft.com/azure/aks/use-multiple-node-pools#assign-a-public-ip-per-node-for-your-node-pools-preview

Resources

Integrate Azure Firewall with Azure Standard Load Balancer
https://docs.microsoft.com/azure/firewall/integrate-lb
Use a static public IP address for egress traffic in Azure Kubernetes Service (AKS)
https://docs.microsoft.com/azure/aks/egress