Securing AWS accounts with IAM and friends

If you've read a few of my posts, you'll know how much I love AWS. I especially love the flexibility of having a very large set of resources at your fingertips, ready to scale up or down at the push of a button, or automatically, as you need.

That can be a disadvantage though, too. Through an accidental check-in to one of our public repositories at work, we recently had leaked some infrastructure credentials with permission to create new EC2 instances. As you probably already know, bad people run scanners looking at checkins to the likes of GitHub, searching for such gems. When found, they immediately and automatically create hundreds of EC2 instances across the world, each doing the modern equivalent of panning for gold: trying to find the next Bitcoin.

The unusual activity was noticed almost immediately by both us and Amazon, and we worked quickly with them to shut down the instances.

It was a good opportunity to review our AWS security, so today I'm going to share what we do, including the things we've changed, or will be, as a result of that review. I'm not going to cover network or host security today, perhaps a topic for another time.

Terminology

Let me first introduce you to the key AWS systems relating to authentication and authorisation, then I'll go into detail about how we're using them:

• Identity and Access Management (IAM): AWS accounts by default have a God-like root account: a single account with access to everything. With IAM you create separate user accounts, each with their own credentials and associated access keys and secrets. Through policies applied at the user level or group level, fine-grained access control can be applied to AWS resources.
• Security Token Service (STS): Grants temporary access tokens.
• CloudTrail: records audit trails of AWS control APIs.

The root account

Don't use it. Well, use it sparingly when you need to set up everything else I talk about in this post. Why not? Well, firstly there's the principle of least privilege, which states that you should only grant privileges for what's needed. For example, a process that uploads files to an S3 bucket should only be allowed to put objects into a specific bucket. There's no need for it to be able to launch EC2 instances, or even delete files from S3. This makes sure that jobs can't get out of their tree and run amok. Since the root account is an all-doing account, there's no way to limit what it can do.

Secondly, it means that you don't need to share passwords. Each person will use their own credentials to authenticate, which makes things a lot easier when someone leaves.

Oh, and for pity's sake, make sure you enable multi-factor authentication (MFA) on the root account.

IAM

Each real person has their own account, which has permissions across a subset of AWS resources. Those permissions are defined by policies attached to groups, to which they belong. We used to also use special-purpose IAM users to grant access to certain resources. Take access to backups or deleting EBS snapshots, for example: this wasn't permitted from user accounts, but from separate, isolated, accounts. We've since changed this, though; see below for how we now use roles instead.

Each user would create their own AWS access key and access key secret pairs, which could be used to authenticate when using the AWS command-line interface, and elsewhere. Here's the rub, though: while access to the console is protected with a MFA token, those generated access keys and secrets had no such protection once created.

So now most of the statements in the policy attached to our main group have the following condition added:

"Condition": {
  "Bool": {
    "aws:MultiFactorAuthPresent": "true"
  }
}

Permissions granted through any statements with this condition attached now require that request to have been authenticated with a MFA token.

That's all well and good except the command-line interface don't support authentication using an MFA token. I'd love to be able to just take my IAM user access key and secret, run aws ec2 describe-instances and be prompted for an MFA token. Alas not (yet, at least). Instead we need to add another step: session tokens.

STS

Security Token Service allows you to create various types of short-lived session tokens. Given we're not using enterprise authentication, we had a couple of options: AssumeRole or GetSessionToken. There are advantages and disadvantages to each:-

We plumped for the AssumeRole approach because, not only are we firm believers in the principle of least privilege, it also let us get rid of our special-purpose IAM users at the same time by creating additional roles with elevated privileges for the likes of administration of backups.

Roles

A role has an associated policy, much like a group. Amazon kindly provide a bunch of managed policies for common cases, so it's easy to create a role with read-only S3 access using the AmazonS3ReadOnlyAccess role, for example. A custom policy can also be attached, so you're able to fine-tune a set of permissions to match each use-case. Using tags in rules lets you only give permissions to a subset of resources.

Trust relationships then allow you to control who is able to assume a role. Unfortunately, this doesn't yet allow you to specify a group so users must be listed individually. So here I'm letting tom and tim assume the role:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "",
      "Effect": "Allow",
      "Principal": {
        "AWS": [
          "arn:aws:iam::448613307115:user/tim",
          "arn:aws:iam::448613307115:user/tom",
        ]
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

It's annoying that groups can't be referenced, but hopefully that'll be fixed soon.

Hosts can also be assigned roles. This is great if, for instance, you've got a process running on an EC2 instance to upload files to S3, you don't need to deploy credentials to it. You just launch an instance with a particular role, then it's able to get session tokens through the instance metadata. We don't use host roles yet, but will be soon.

Assuming roles

As I mentioned, the command-line interface doesn't work with session tokens natively. Instead, the workflow is:

1. Create access key and access key secret using an IAM user
2. Use these credentials to obtain a session token using aws sts assume-role
3. Use created session token in subsequent API calls

This is Quite cumbersome if done manually as it requires a number of hops to look up various resource identifiers and the like. Instead, please give a warm welcome to our handy shell script, assume-aws-role! Let's see it in action:

// First up, we configure it by giving it an IAM user's key and secret
› assume-aws-role -s

Setting up profile iam-user. You should provide an access key
and secret for an IAM user with rights to assume roles.

Access key: ██████████████████████
Access key secret: ████████████████████████████████████████████

Credentials saved

// Let's see what roles we can assume

› assume-aws-role -l
All roles (you may not have access to any or all of these):

user-s3-readonly
user-ebs-snapshot-remove
user-backup-readwrite

› assume-aws-role user-s3-readonly 051859
Session token saved as default set of AWS credentials

› aws s3 ls
2015-04-30 15:37:27 my-first-s3-folder
...

› 

The script works by using multiple profiles; one with the long-lived key for creating session tokens, and then the default profile to house session tokens. You're welcome to the script if you've a use for it. Sold (given away) as seen, of course.

CloudTrail

Configure CloudTrail to audit all API actions, so if the worst happens, you can audit what was done. It's configured per region, so make sure you set it up everywhere.

Good work, Amazon

At the precise moment I was thinking about what I'd say in this closing section about how powerful IAM and friends are, an email landed in my inbox from Amazon shouting about them having been named a "Leader in Cloud Security" in a recent Forrester report. A lot of that was about physical security, but IAM also rightly got a mention.

There are still some wrinkles—not all resource types support per-resource permissions for example—and I really want to see some more things to really encourage best practice; namely native support for MFA tokens using their own command-line interface and a sensible default security configuration on account creation. They're continuing to invest in security, as demonstrated by the plethora of improvements such as the policy simulator and managed policies, so hopefully we'll see these things soon.

Another great example of what you get for free from AWS—factored into your per-hour charges elsewhere, of course—that you'd have to build or integrate from scratch elsewhere.