The python based AWS Command Line Interface (CLI) is a unified tool to interact with and manage AWS services programmatically via the terminal. This post will show you a set-up to handling multiple identities scattered across multiple accounts, roles and also sometimes requiring multi-factor authentication (MFA).
You can install the AWS CLI in a myriad of ways. The thing that might work for most set-ups (depending on how your Python interpreter is set up) is just running
pip install awscli. Refer to the installation instructions in the AWS CLI GitHub Repo for more options.
Once installed, getting started is quite easy. Just run
aws configure, put in your AWS Access Key Id and Secret Access Key and you are good to go. This will create two INI-formatted files (
credentials) in the
.aws directory of your home folder:
$ ls ~/.aws/ config credentials
credentials file will contain a
[default] section containing the access key id and secret access key information you put in above:
[default] aws_access_key_id = ABCDEFGHIJKLMNOP0123 aws_secret_access_key = AEyao7CklmK3Ye7hT5+//KlkjKHGt765
config file will also contain a
[default] section, but this contains the default region and output format you chose during
[default] region = eu-central-1 output = json
If all stays the same these are the default credentials the CLI will use every time you use it (e.g. for listing files in an S3 bucket).
Having multiple identities
Being in an enterprise environment there will come a point where you might have to juggle multiple identities (think multiple accounts or roles). There are a few ways to go about this.
New profiles using
This simplest way to set up a new identity is to just set up a new profile and provide credentials (in the form of access key id and secret access key) accordingly.
To set up a new profile named
aws configure --profile prod-account
and enter the credentials for this account (if you don't have credentials and just a role see below ...).
You should now have a new section
~/.aws/credentials and also a section
[profile prod-account] in
~/.aws/config (the prefix
profile is mandatory for all entries except
New profiles by directly modifying
The same as above can be achieved by directly editing
[default] aws_access_key_id = ABCDEFGHIJKLMNOP0123 aws_secret_access_key = AEyao7CklmK3Ye7hT5+//KlkjKHGt765 [prod-account] aws_access_key_id = ... aws_secret_access_key = ...
[default] region = eu-central-1 output = json [profile prod-account] region = eu-central-1 output = json
This is basically the gist of it. You need to add a new identity just have your access key id and secret access key ready for each account...
...unless there is another way, without having to juggle countless account logins and secrets.
Roles for granular access
Roles within IAM are a concept to temporarily delegate permissions to anyone (i.e. users) or anything (i.e. ec2 instances, lambdas etc.) that is allowed to use (assume) this role. (see IAM roles terms and concepts)
IAM roles can be used by users of the same account and also by users of a different account. For an in-depth tutorial on cross-account role access see this article in the AWS documentation.
The AWS CLI provides mechanisms, that let you automatically assume a role when performing calls. To leverage this mechanism you will have to edit the
New profiles with session credentials via roles
Let's assume the following:
- the id of your prod-account is
- the profile
prod-accountset up above is not allowed to do anything except assuming roles
- there is a role called
NetworkAdminRolethat will give you permissions to administer networking (i.e. VPCs, route tables etc.)
We have to tell the CLI it can use this role via a profile. To do this add the profile
... [profile prod-network-admin] role_arn = arn:aws:iam::012345678901:role/NetworkAdminRole source_profile = prod-account region = eu-central-1 output = json
We need to provide the
source_profile settings which tell the CLI that this profile uses a role that can be accessed through the source profile
prod-account (see also this deep-link to some documentation).
Using a role will afford temporary session credentials, that will be cached behind the scenes by the AWS CLI. The time that the session will be valid can be configured via the
duration_seconds parameter in the profile config. The default is 3600 (1 hour).
Other than providing a
source profile inside the profile config you can also specify the option
credential_source can be
EcsContainer. Set to
Environment the cli will use the current environment's credentials, for example if you provide the environmental variables
The last two are for use in EC2 or ECS respectively.
source_profile are mutually exclusive!
Using different profiles
All (or at least most of) the commands that you call via the AWS CLI can be told to use a specific profile. You do this, by appending the
--profile <profile-name> option to the call:
- calling S3 list with the default profile:
aws s3 ls my-bucket
- calling S3 list using the
aws s3 ls my-bucket --profile prod-account
- calling S3 list using the
aws s3 ls my-bucket --profile prod-network-admin
--profile ... parameter to every call can be quite cumbersome. You can just export the environment variable
AWS_PROFILE into your terminal session. This will make the cli to use the specified profile every time:
export AWS_PROFILE=prod-network-admin ### all subsequent aws calls will use the profile prod-network-admin aws s3 ls my-bucket
Using multi-factor authentication (MFA) with profiles and the cli
If you (or the company) have a (role) policy in place that only allows programmatic access if there is a multi-factor device enabled you will not have much luck with what we just set up. You will most likely get an
AccessDenied error, when performing any operation.
You need to tell the cli that you have to provide an mfa token. To do that add the
mfa_serial option to the profile:
... [profile prod-network-admin] role_arn = arn:aws:iam::012345678901:role/NetworkAdminRole source_profile = prod-account region = eu-central-1 output = json mfa_serial = arn:aws:iam::012345678901:mfa/Benjamin.Weigel
You can find the mfa serial (arn) under your security credentials. Got to
Username @ Account>
My Security Credentials (top right corner in the AWS console).
Once you provide that option the cli will as you to provide an mfa token when you make a request. The session is then cached until it expires:
That's it for today. I hope some of you will find this article interesting. I wish I would have known some of this earlier.
~/.aws/credentials) to create profiles to use with the AWS CLI (and SDKs etc.). Use
source_profile to work via roles and avoid having to juggle multiple secrets. Set
mfa_serial, when multi-factor authentication is required.
Select which profiles are used by to make calls by adding
--profile <profile_name> to cli commands, or set a profile for the current terminal session by specifying the environment variable