Install AWS Load Balancer Controller on EKS cluster and set up ALB Ingress

2024-02-26 kubernetes aws

AWS Load Balancer Controller is a controller that creates and manages ELBs for Kubernetes resources. It can be installed with a service account granted the required policies and Helm Chart, but if you create the cluster with CDK, this is done by specifying the version in albController.

new eks.Cluster(this, 'Cluster', {
  ...
  albController: {
    version: eks.AlbControllerVersion.V2_6_2,
  },
})

An ALB is created by creating the Ingress as follows. Subnets can be specified with alb.ingress.kubernetes.io/subnets, but if subnets have following tags, auto discovery works.

kubernetes.io/role/elb : 1
kubernetes.io/cluster/${cluster-name}: owned or shared

alb.ingress.kubernetes.io/target-type is instance or ip, and directs traffic to instances or pods directly.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: test
  annotations:
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
    alb.ingress.kubernetes.io/subnets: testpublicsubnet1, testpublicsubnet2
    alb.ingress.kubernetes.io/healthcheck-path: /health
    alb.ingress.kubernetes.io/target-group-attributes: deregistration_delay.timeout_seconds=5
spec:
  ingressClassName: alb
	rules:
    - http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: test
              port:
                number: 80

In case ip, pods may stop before ALB deregistration, resulting in a connection failure and a 5xx error, so you should shorten the Deregistration delay and also wait longer.

 lifecycle:
  preStop:
    exec:
      command: ["sleep", "15s"]

Additionally, if pods become ready before the ALB health check passes during a rolling update, previous pods will terminate and resources will be insufficient.To prevent this, add the following label to the namespace and make Pod readiness gate inserted automatically.

apiVersion: v1
kind: Namespace
metadata:
  name: app
  labels:
    elbv2.k8s.aws/pod-readiness-gate-inject: enabled

If load is concentrated on specific pods and the health check fails, requests will no longer be routed to them, causing the remaining pods also can be down due to traffic spike. It may be possible to resolve the concentrated load by using Least Outstanding Requests (LOR), which routes to the target with the fewest requests being processed instead of the default round robin. However, note that if it has a sidecar whose load increases by the number of requests, the load on that sidecar can be biased.

alb.ingress.kubernetes.io/target-group-attributes: load_balancing.algorithm.type=least_outstanding_requests

If an ALB is not created due to AccessDenied etc., you should check that the policies is up to date and that there are no errors in the annotations of ServiceAccount.

You can specify custom security groups at alb.ingress.kubernetes.io/security-groups, but if you do that, unless you also set alb.ingress.kubernetes.io/manage-backend-security-group-rules to true, the Backend SG that included in instance’s inbound rule won’t be added to the ALB, so it can’t access to the instances.