Mikhail Alekseev
Senior Cloud Engineer

AWS EKS Security. IAM. Part 1

Today, we are pleased to announce that we have set up a series of AWS EKS security-related blog posts. In these posts, we will cover the main principles of securing an AWS Elastic Kubernetes Service (EKS) cluster. The first topic in this series is Identity and Access Management (IAM) in AWS EKS. The topic is split into 2 parts:

1. How to grant access from an EKS cluster to other AWS services
2. How to grant access to an EKS cluster

Granting access from an EKS cluster to AWS services

The methods of accessing Kubernetes (K8s)/AWS Elastic Kubernetes Service (EKS) clusters are Service Account

A service account is a type of non-human account that, in Kubernetes, provides a distinct identity in a Kubernetes cluster.
Application Pods, system components, and entities inside and outside the cluster can use a specific ServiceAccount’s credentials to identify as that ServiceAccount.
This identity is useful in various situations, including authenticating to the API server or implementing identity-based security policies.

All further actions and considerations are applied exclusively to service accounts.

IAM Roles for Service Accounts (IRSA)

Applications in a Pod’s containers can use an AWS SDK or the AWS CLI to make API requests to AWS services using AWS Identity and Access Management (IAM) permissions.
Applications must sign their AWS API requests with AWS credentials.
IAM roles for service accounts provide the ability to manage credentials for your applications, similar to the way that Amazon EC2 instance profiles provide credentials to Amazon EC2 instances. Instead of creating and distributing your AWS credentials to the containers or using the Amazon EC2 instance’s role, you associate an IAM role with a Kubernetes service account and configure your Pods to use the service account.

IAM roles for service accounts provide the following benefits:

• Least privilege - following the principle of the least privilege IAM role can (and should!) have only the required scope of permissions and resources to interact with.

• Credential isolation - A Pod’s containers can only retrieve credentials for the IAM role that’s associated with the service account that the container uses. A container never has access to credentials that are used by other containers in other Pods. When using IAM roles for service accounts, the Pod’s containers also have the permissions assigned to the Amazon EKS node IAM role, unless you block Pod access to the Amazon EC2 Instance Metadata Service ( IMDS).

• Auditability - Access and event logging is available through AWS CloudTrail to help ensure retrospective auditing

To start using IRSA several of configuration options must be applied:

• Create an OIDC provider for the cluster
• Create and configure IAM roles specifying the IRSA-related trust policy - Use supported AWS SDK in the containers using IRSA
• Annotate the service account and use it with the POD

Figure 1 - IRSA Architecture

How IAM Roles for Service Accounts (IRSA) work:

1. The service account (SA) is annotated with an IAM role

   
   1
   2
   3
   4
   5
   6
   
   apiVersion: v1
   kind: ServiceAccount
   metadata:
   annotations:
   eks.amazonaws.com/role-arn: arn:aws:iam::<account_id>:role/<role_name>
   ...
   
   

2. The SA is attached to the POD
3. An AWS SDK supporting AWS IRSA sends AssumeRoleWithWebIdentity request to AWS STS with a reference to the IAM role
4. AWS STS sends request further using configured OIDC Identity Provider (OIDC IdP)
5. The OIDC IdP has trust relationships with the IAM Role
6. The AWS STS generates temporary security credentials
7. The temporary security credentials are sent back and the POD has permissions attached to the IAM role

DEMO

To demonstrate IRSA we need to deploy an EKS cluster and all related services first. Source code is located here.

Deploy infrastructure

cd ${REPOSITORY_ROOT}/02_assume_iam_roles/01_irsa/infra

Deploy the application

cd ${REPOSITORY_ROOT}/02_assume_iam_roles/01_irsa/app

After the infrastructure has been deployed, the application can be deployed.
In this demo, we are going to deploy a simple application and expose its port to the Internet through a Network Load Balancer (NLB).
The AWS Application Load Balancer controller (AWS ALB Controller) is in charge of creating Load Balancers.

Figure 2 - Architecture of the demo setup

1
2
3
4
5
6
7
8
9
10
11
12
13
14

module "alb_controller" {
  source                  = "./alb_controller"

  cluster_name            = local.cluster_name
  cluster_oidc_issuer_url = var.cluster_oidc_issuer_url
  name_prefix             = var.name_prefix
  oidc_provider_arn       = var.oidc_provider_arn
  service_account_name    = var.service_account_name
}

module "sample_app" {
  depends_on = [module.alb_controller]
  source     = "./sample_app"
}

1

terraform apply

The code has been deployed. Let’s check if an NLB was created.
Go to AWS Account > EC2 > Load Balancers

Figure 3 - Network Load Balancer

Yes, the Network Load Balancer (NLB) is provisioned with the specified name pattern. It confirms that the network load balancer has been created by the controller using the IAM role.

IAM Pod Identities

In November 2023, AWS introduced a feature simplifying the configuration process for cluster administrators to grant AWS Identity and Access Management (IAM) permissions to Kubernetes applications.

IAM Pod Identities offer the same benefits as IAM Roles for Service Accounts (IRSA):

• Least privilege – You can scope IAM permissions to a service account, and only Pods that use that service account have access to those permissions.

• Credential isolation – A Pod’s containers can only retrieve credentials for the IAM role that’s associated with the service account that the container uses. A container never has access to credentials that are used by other containers in other Pods. When using Pod Identities, the Pod’s containers also have the permissions assigned to the Amazon EKS node IAM role, unless you block Pod access to the Amazon EC2 Instance Metadata Service (IMDS).

• Auditability – Access and event logging is available through AWS CloudTrail to help facilitate retrospective auditing.

Despite the benefits, there are differences between IAM Roles for Service Accounts (IRSA) and IAM Pod Identities.
Let’s examine the implementation. To start using EKS Pod Identities some requirements and configuration options must be met/applied:

• EKS Pod Identities work only with EKS version higher or equal to 1.24 (>= 1.24)
• Install the agent matching the EKS cluster version
• Create and configure IAM roles specifying the Pod Identities-related trust policy
• Use supported AWS SDK in containers using EKS Pod Identities
• Associate a Pod Identity with the IAM role
• Use a service account attached to the POD

Figure 4 - AWS Pod Identities Architecture

How EKS Pod Identities work:

1. The EKS Pod Identities add-on is installed
2. The IAM role has trust policies with the EKS Pod Identities service
3. The association between the service account and the role is established
4. After the POD has been deployed, the EKS Pod Identities add-on mutates the POD specs adding the next section

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

...
env:
  - name: AWS_CONTAINER_AUTHORIZATION_TOKEN_FILE
    value: /var/run/secrets/pods.eks.amazonaws.com/serviceaccount/eks-pod-identity-token
  - name: AWS_CONTAINER_CREDENTIALS_FULL_URI
    value: http://169.254.170.23/v1/credentials
volumeMounts:
  - mountPath: /var/run/secrets/pods.eks.amazonaws.com/serviceaccount/
    name: eks-pod-identity-token
    volumes:
  - name: eks-pod-identity-token
    projected:
    defaultMode: 420
    sources:
  - serviceAccountToken:
    audience: pods.eks.amazonaws.com
    expirationSeconds: 86400
    path: eks-pod-identity-token
...

1. The Pod retrieves temporary credentials
2. The pod assumes the role using temporary credentials

Demo

Source code is located here.

Deploy infrastructure

cd ${REPOSITORY_ROOT}/02_assume_iam_roles/02_pod_identities/infra

Let’s take the same infrastructure code and adapt it to EKS Pod Identities.
First, the agent must be deployed. There are several options for how to install it.
Our decision is to use the EKS add-on for the agent.

1
2
3
4
5
6
7
8
9
10
11

module "eks-addons" {
  depends_on = [module.eks]
  source     = "./eks_addons"

  cluster_addons = {
    eks-pod-identity-agent = {
      most_recent = true
    }
  }
  cluster_name = module.eks.eks_cluster_name
}

Apply changes

terraform apply

Verify that the addon has been installed

aws eks list-addons --cluster-name secure-eks-cluster

1
2
3
4
5

{
    "addons": [
        "eks-pod-identity-agent"
    ]
}

Figure 5 - AWS EKS Pod Identities Agent

Deploy the application

We will deploy the same application but with the difference in the way of assuming a role.

cd ${REPOSITORY_ROOT}/02_assume_iam_roles/02_pod_identities/app

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

module "alb_controller" {
  source               = "./alb_controller"

  cluster_name         = local.cluster_name
  name_prefix          = local.name_prefix
  service_account_name = var.service_account_name
}

resource "aws_eks_pod_identity_association" "alb_controller" {
  cluster_name    = local.cluster_name
  namespace       = var.namespace
  service_account = var.service_account_name
  role_arn        = module.alb_controller.iam_role_arn
}

module "sample_app" {
  depends_on = [module.alb_controller]
  source     = "./sample_app"
}

Below is a list of changes in the application deployment section:

• OIDC data is not required anymore since it’s not the case
• A Pod Identity association must be created first to provide access to the service account
  Also, as it was mentioned above the IAM role should establish trust relationships with Pod Identities.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

resource "aws_iam_role" "alb_controller" {
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
-       Action = "sts:AssumeRoleWithWebIdentity"
+       Action = [
+         "sts:AssumeRole",
+         "sts:TagSession"
+       ]
        Effect = "Allow"
-       Principal = {
-         Federated = var.oidc_provider_arn #"${data.aws_ssm_parameter.oidc_provider_arn.value}"
-       }
+       Principal = {
+         Service = "pods.eks.amazonaws.com"
+       }
-       Condition = {
-         StringEquals = {
-           "${replace(var.cluster_oidc_issuer_url, "https://", "")}:sub" = [
-             "system:serviceaccount:${var.namespace}:${var.service_account_name}"
-           ]
-         }
        }
      },
    ]
  })
  name = local.alb_controller_role_name
}

Let’s apply this version

terraform apply

After the deployment has been successfully finished, go to AWS Account > EC2 > Load Balancers

Figure 6 - Network Load Balancer

We see the same outcome as with IAM Roles for Service Accounts (IRSA). The Network Load Balancer (NLB) is provisioned with the specified name pattern. It confirms that the network load balancer has been created by the controller using the IAM role.

Comparison

Conclusion

In this post, we showed ways to grant IAM permissions to containerized workloads in EKS clusters. We started with the first AWS solution, AWS IAM Roles for Service Accounts (IRSA), and continued with a new feature called EKS Pod Identities. The text is complemented by a demo showing how to migrate from AWS IRSA to EKS Pod Identities. In the end, we added a comparison of both features to show the fundamental differences and limitations in features.

See you in the next blog post!