Understanding credentials using the AWS CLI

The python based AWS Command Line Interface (CLI) is a unified tool to interact with and manage AWS services programmatically via the terminal. This post will show you a set-up to handling multiple identities scattered across multiple accounts, roles and also sometimes requiring multi-factor authentication (MFA).

Quick set-up

You can install the AWS CLI in a myriad of ways. The thing that might work for most set-ups (depending on how your Python interpreter is set up) is just running pip install awscli. Refer to the installation instructions in the AWS CLI GitHub Repo for more options.

Once installed, getting started is quite easy. Just run aws configure, put in your AWS Access Key Id and Secret Access Key and you are good to go. This will create two INI-formatted files (config and credentials) in the .aws directory of your home folder:

$ ls ~/.aws/
config
credentials

The credentials file will contain a [default] section containing the access key id and secret access key information you put in above:

[default]
aws_access_key_id = ABCDEFGHIJKLMNOP0123
aws_secret_access_key = AEyao7CklmK3Ye7hT5+//KlkjKHGt765

The config file will also contain a [default] section, but this contains the default region and output format you chose during aws configure:

[default]
region = eu-central-1
output = json

If all stays the same these are the default credentials the CLI will use every time you use it (e.g. for listing files in an S3 bucket).

Having multiple identities

Being in an enterprise environment there will come a point where you might have to juggle multiple identities (think multiple accounts or roles). There are a few ways to go about this.

New profiles using aws configure

This simplest way to set up a new identity is to just set up a new profile and provide credentials (in the form of access key id and secret access key) accordingly.

To set up a new profile named prod-account do

aws configure --profile prod-account

and enter the credentials for this account (if you don’t have credentials and just a role see below …). You should now have a new section [prod-account] in ~/.aws/credentials and also a section [profile prod-account] in ~/.aws/config (the prefix profile is mandatory for all entries except default).

New profiles by directly modifying config and credentials

The same as above can be achieved by directly editing ~/.aws/credentials:

[default]
aws_access_key_id = ABCDEFGHIJKLMNOP0123
aws_secret_access_key = AEyao7CklmK3Ye7hT5+//KlkjKHGt765
[prod-account]
aws_access_key_id = ...
aws_secret_access_key = ...

and ~/.aws/config:

[default]
region = eu-central-1
output = json
[profile prod-account]
region = eu-central-1
output = json

This is basically the gist of it. You need to add a new identity just have your access key id and secret access key ready for each account…

…unless there is another way, without having to juggle countless account logins and secrets.

Roles for granular access

Roles within IAM are a concept to temporarily delegate permissions to anyone (i.e. users) or anything (i.e. ec2 instances, lambdas etc.) that is allowed to use (assume) this role. (see IAM roles terms and concepts)

IAM roles can be used by users of the same account and also by users of a different account. For an in-depth tutorial on cross-account role access see this article in the AWS documentation.

The AWS CLI provides mechanisms, that let you automatically assume a role when performing calls. To leverage this mechanism you will have to edit the ~/.aws/config.

New profiles with session credentials via roles

Let’s assume the following:

• the id of your prod-account is 012345678901
• the profile prod-account set up above is not allowed to do anything except assuming roles
• there is a role called NetworkAdminRole that will give you permissions to administer networking (i.e. VPCs, route tables etc.)

We have to tell the CLI it can use this role via a profile. To do this add the profile prod-network-admin to ~/.aws/config:

...
[profile prod-network-admin]
role_arn = arn:aws:iam::012345678901:role/NetworkAdminRole
source_profile = prod-account
region = eu-central-1
output = json

We need to provide the role_arn and source_profile settings which tell the CLI that this profile uses a role that can be accessed through the source profile prod-account (see also this deep-link to some documentation).

Using a role will afford temporary session credentials, that will be cached behind the scenes by the AWS CLI. The time that the session will be valid can be configured via the duration_seconds parameter in the profile config. The default is 3600 (1 hour).

credential_source versus source_profile

Other than providing a source profile inside the profile config you can also specify the option credential_source. credential_source can be Environment, Ec2InstanceMetadata or EcsContainer. Set to Environment the cli will use the current environment’s credentials, for example if you provide the environmental variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY. The last two are for use in EC2 or ECS respectively.

The options credential_source and source_profile are mutually exclusive!

Using different profiles

All (or at least most of) the commands that you call via the AWS CLI can be told to use a specific profile. You do this, by appending the --profile <profile-name> option to the call:

• calling S3 list with the default profile:

  aws s3 ls my-bucket
  

• calling S3 list using the prod-account profile:

  aws s3 ls my-bucket --profile prod-account
  

• calling S3 list using the prod-network-admin profile:

  aws s3 ls my-bucket --profile prod-network-admin
  

Adding a --profile ... parameter to every call can be quite cumbersome. You can just export the environment variable AWS_PROFILE into your terminal session. This will make the cli to use the specified profile every time:

export AWS_PROFILE=prod-network-admin

### all subsequent aws calls will use the profile prod-network-admin

aws s3 ls my-bucket

Using multi-factor authentication (MFA) with profiles and the cli

If you (or the company) have a (role) policy in place that only allows programmatic access if there is a multi-factor device enabled you will not have much luck with what we just set up. You will most likely get an AccessDenied error, when performing any operation.

You need to tell the cli that you have to provide an mfa token. To do that add the mfa_serial option to the profile:

...
[profile prod-network-admin]
role_arn = arn:aws:iam::012345678901:role/NetworkAdminRole
source_profile = prod-account
region = eu-central-1
output = json
mfa_serial = arn:aws:iam::012345678901:mfa/Benjamin.Weigel

You can find the mfa serial (arn) under your security credentials. Got to Username @ Account>My Security Credentials (top right corner in the AWS console).

Once you provide that option the cli will as you to provide an mfa token when you make a request. The session is then cached until it expires:

Wrap-Up

That’s it for today. I hope some of you will find this article interesting. I wish I would have known some of this earlier.

Further Reading

• Credential provider chain

TL;DR

Edit ~/.aws/config (and ~/.aws/credentials) to create profiles to use with the AWS CLI (and SDKs etc.). Use role_arn and source_profile to work via roles and avoid having to juggle multiple secrets. Set mfa_serial, when multi-factor authentication is required. Select which profiles are used by to make calls by adding --profile <profile_name> to cli commands, or set a profile for the current terminal session by specifying the environment variable AWS_PROFILE.