In this series, I’m going to talk about Amazon SQS, one of the first Amazon services released for public usage. Released in November 2004, it’s one of the oldest services in the AWS catalog. Recently I bumped upon two interesting SQS features, leading me to write this blog. These features are Encryption at rest and SQS Resource Policy.
Encryption at rest with SQS
At this time, AWS SQS offers you two options for encryption at rest:
- Server-side encryption using AWS-owned keys (SSE-SQS)
- Server-side encryption using AWS Key Management Service (SSE-KMS)
The differences between the two in short are:
- SSE-KMS: customer-managed keys, the customer can manage the policies of the keys, enable/disable keys, rotate their cryptographic material, schedule key deletion, add tags and create aliases.
- SSE-SQS: keys owned by AWS. Customers can’t delete keys, rotate, or enable/disable them.
On the other hand, they’re evenly matched. The best choice fully depends on the use case:
- If you need encryption at rest to be enabled without enforcing policies around key usage, then you’re better off using SSE-SQS, because it’ll take away the complexity of managing keys.
- If there’s a need for more strict policies around data encryption -banks fall into this category- and there is a need to manage the encryption key themselves, SSE-KMS is the better choice.
Enough theory 😉 Let’s take a look at some real-life scenarios. Take the following use case, an SQS FIFO queue subscribed to an SNS Topic:
How to enable encryption at rest on the SQS queue? Allow me to show the solution by showing the mistake I made setting this up (second time in my career already). In fact, deploying this, I bricked the communication channel between the SNS topic and the SQS queue 😓
To break the communication link between SQS and SNS, all I did was enable encryption at rest using an SSE-SQS key while overlooking the queue was subscribed to SNS. Soon after, it became clear the queue wasn’t receiving messages from SNS anymore. After troubleshooting and browsing the documentation, I discovered the reason for the disconnect. Out of the box, the SSE-SQS key I added to the queue doesn’t have the policy permission allowing SNS to use the encryption key. The Result? Missing that policy on the key, SNS could not long publish messages on the queue.
The correct way to enable encryption at rest on an SQS queue that is subscribed to SNS is by leveraging a KMS key:
- Create a KMS key
- Create a key policy allowing SNS to use the key:
{
"Sid": "Allow Amazon SNS to use this key",
"Effect": "Allow",
"Principal": {
"Service": "sns.amazonaws.com"
},
"Action": [
"kms:Decrypt",
"kms:GenerateDataKey*"
],
"Resource": "*"
}
- Enable SSE-KMS on the SQS queue using your custom managed key
- If required, use the same key to encrypt the SNS Topic at rest. In this case, also don’t forget to give the publisher permissions to use the KMS key:
{
"Sid": "Allow publisher permission to KMS CMK",
"Effect": "Allow",
"Action": [
"kms:GenerateDataKey",
"kms:Decrypt"
],
"Resource": <your-kms-key-arn>
}
Problem solved 😉 Now let’s look at our second topic: SQS Resource policies.
SQS Resource Policy
Currently, AWS has two policy types:
- Identity-based policy ⇒ policies attached to an IAM user, group, or role. The policy defines what an identity can do.
- Resource-based policy ⇒ policies attached to a resource. These types of policies define who can access a resource and what action can be performed. For the list of AWS services supporting resource-based policies, see List of AWS resources.
One of the resources supporting identity-based and resource-based policies is AWS SQS. Let’s take a look at how you can leverage a resource policy by a simple example. Imagine the following architecture:
We have an SNS topic with an SQS subscription, both in account A. There’s an ECS service running in another account (account B). The ECS service polls the queue for messages. In this example, let’s create an SQS queue resource-based policy to allow SNS to send messages to the SQS queue. Additionally, the ECS service in account B is allowed to poll the queue for messages. As good citizens, this also has to comply with AWS’s least privilege principle.
Let’s see what this looks like. The first section of the resource policy is to allow a specific SNS topic to send messages to the SQS queue:
{
"Sid": "Allow Amazon SNS to send message to SQS",
"Effect": "Allow",
"Principal": {
"Service": "sns.amazonaws.com"
},
"Action": "sqs:SendMessage",
"Resource": <your-sqs-queue-arn>,
"Condition": {
"ArnEquals": {
"aws:sourceArn": <your-sns-topic-arn>
}
}
}
It allows(=Effect) the principal sns.amazonaws.com (=Service) to send messages (=Action) to the queue (=Resource). But, it only allows this if the topic ARN matches the specified ARN (=Condition). By defining the policy this way, only the topic with the specified ARN can send messages to the queue.
There’s also a second part. The section of the SQS resource policy below allows the ECS service in account B to receive messages from the queue:
{
"Sid": "Allow Amazon ECS service to read message from SQS",
"Effect": "Allow",
"Principal": {
"Service": "ecs.amazonaws.com"
},
"Action": [
"sqs:ReceiveMessage",
"sqs:DeleteMessage"
],
"Resource": <your-sqs-queue-arn>,
"Condition": {
"ArnEquals": {
"aws:sourceArn": <your-ecs-service-arn>
}
}
}
It allows (=Effect) the principal ecs.amazonaws.com (=Service) to receive and delete messages (=Action) to the queue (=Resource). It only allows an ECS service if the ARN matches the ARN defined in the policy (=Condition).
In this example, aws:sourceArn is used as the condition key. Note that you can use different condition keys to get the same result. Visit the following link for the condition keys available for ECS: Link to ECS Condition keys. Keep in mind that the described policy only enables cross-account resource access (it only allows the ECS service to access the SQS queue). In order to work, also define sqs:ReceiveMessage and sqs:DeleteMessage on the ECS task role.
Summary
To summarize what we’ve discussed:
- SQS encryption at rest + real-life scenario example
- SQS Resource Policy + real-life scenario example
In a nutshell: don’t enable encryption at rest using SSE-SQS keys when subscribed to an SNS topic and know when to leverage SQS queue resource policy vs. identity-based policy.
If you enjoyed this post, don’t hesitate to share it with your friends and colleagues 😉
Enjoy and until next time!