How to push Docker Images to AWS ECR(Elastic Container registry)?
Docker is all about containerizing your applications. For a developer, running containers locally through docker can be a bit overwhelming as there are multiple dimensions involved, such as ensuring that containers can communicate over a network, provisioning persistent storage, pushing, pulling, and copying images from one environment to another, and managing image registries. All these factors add complexity to the development process and can increase the deployment time of the application.
Table of Content
- What is AWS ECS?
- AWS ECS Serverless computing
- Prerequisites
- Setting up IAM role for ECS?
- What is ECR repository?
- How to Creat an ECR repository?
- How to Login into ECR repository?
- Performing operations on container images
- Managing Elastic container registry
- Creating lifecycle policies for container images
- How to fix - Not authorized to perform: ecr:GetAuthorizationToken on resource: * because no identity-based policy allows the ecr:GetAuthorizationToken
- How to fix -An error occurred (AccessDeniedException) when calling the GetAuthorizationToken operation: User:arn:aws:iam::ACCOUNT_NUMBER:user/MY_USER is not authorized to perform: ecr:GetAuthorizationToken on resource:
- Docker push to AWS ECR hangs immediately and times out 14 .Conclusion
What is AWS ECS?
Amazon Elastic Container Service is a fully managed container orchestration service from Amazon that supports docker containers. The service eases the container deployment process by removing all of the complexity of managing servers, storage, networking, and security. It allows the end users to:
- Build and publish container images in container registries
- Deploy containers on AWS that have been built in your local environment
- Make containers readily accessible from anywhere with no downtime
- Run containers using your own EC2 infrastructure or using a serverless compute engine such as AWS Fargate
- Integrate with AWS Identity and Management to assign permissions
- Secure containers using the built-in security features of AWS
The AWS ECS contains multiple components such as container agents, reg tasks, and services. A detailed explanation of these components can be found here.
AWS ECS Serverless computing
The most important concept tied with AWS ECS is serverless computing. This is achieved through AWS Fargate. Fargate is a compute engine for deploying containers without needing to provision and configure any servers for your container hosts. This makes the process of deploying containerized applications fairly simple as all the infrastructure layers are abstracted from the end user.
We’re going to cover AWS Fargate in the next tutorial. In this tutorial, we’re going to focus solely on Elastic container service and registries.
Prerequisites
The following items are required for this tutorial:
- AWS free tier account
- An IAM user with the access to perform relevant functions
- AWS CLI configured on your local system (See this link for details)
- Any Linux distribution with docker installation
Setting up IAM role for ECS?
It is recommended to use a non-root user for the tasks performed in this tutorial. To set up an IAM user with access to ECS, follow these steps.
-
Go to the IAM dashboard in your AWS account
Add AWS IAM user -
Create a new user and check the "Access key - Programmatic access" option.
Add AWS IAM user with programmatic access -
Select the “Attach existing policies directly” option and search for "ECS".
Attach IAM policy to AWS IAM user -
Select the "AmazonECS_FullAccess" in policy and proceed next.
Review AWS IAM user -
Create the user and make sure to save the access key ID and secret access key. They will be used when configuring the CLI.
Access key and secret key
What is ECR repository?
To store container images, AWS provides the Elastic container registry. This is a fully managed docker registry provided by AWS. Each repository is configured with a name and URI, which is unique to the user's AWS account and region. Using ECR for storing your container images offers several benefits, such as:
- integration with other AWS services
- permissions to allow or deny access to container images and operations (such as push, pull, etc)
- authenticating a user’s IAM credentials
- removing orphaned container images by implementing lifecycle policies
How to Create an ECR repository?
In the following example, we’ll see how we can create an ECR repository through the AWS console and AWS CLI. It is recommended to use a non-root user for these tasks.
Example
- On the AWS console, search and select ECS.
- Select Repositories from the menu on the left side, and then click “Create repository”.
- You can set the repository type to either public or private.
To create a repository named “docker-images” from AWS CLI, run the following command:
1aws ecr create-repository --repository-name docker-images
Note down the "repositoryUri". This is the ECR URL and it will be used for pushing and pulling container images
How to Login into ECR repository
To login to your ECR repository through docker on your local system, run the following command. You’ll need to specify -
- AWS region in place of "aws_region"
- Your AWS account ID in place of “account_id.
1aws ecr get-login-password --region aws_region | docker login --username AWS --password-stdin account_id.dkr.ecr.us-east-1.amazonaws.com
Command Output-
Performing operations on container images
Now that our ECR repository is up and running, and we’ve logged into it, we can use it like any other container registry. To push images to the ECR repository, we need to perform the following steps:
- Have the ECR URL (repositoryUri) available with us, this is important as it will be used for all image operations
- Authenticate to the ECR repository through AWS CLI
- Build and tag the container images with the ECR URL
- Push the images to the ECR repository
We’ve already completed the first two steps. In the following examples, we’ll see how we can push, pull and delete container images from the ECR repository.
Example - Pushing images
To push an image to the ECR repository, tag it as follows, make sure to enter the correct repository URI.
1docker tag my-image repositoryURI/repository-name:myimage 2 3docker push repositoryURI/repository-name:myimage
Command Output-
Once the image has been pushed, you can verify it as follows:
1aws ecr list-images --repository-name repository_name
Command Output-
Example - Pulling images from ECR and launching containers
Images in ECR repository can be pulled like any regular container image. We just need to specify the correct repository URI.
1aws@node:~$ docker pull 870087281092.dkr.ecr.us-east-1.amazonaws.com/docker-images:nginx
Similarly, we can launch a local docker container using the images available in ECR.
1aws@node:~$ docker run --name nginx -d 870087281092.dkr.ecr.us-east-1.amazonaws.com/docker-images:nginx 247c8c50549ac1592e02e5ea7fa402d8f98a5533bf611f506641419636e58b2bc
Example - Deleting images from ECR
To delete the container image from ECR repository, run the following:
1aws ecr batch-delete-image --repository-name docker-images --image-ids imageTag=nginx
Command Output-
Managing Elastic container registry
The AWS ECR makes our container images highly available. Although it is fully managed by AWS, it is still important for developers and administrators to understand its key features and concepts. For example, storing unnecessary or orphaned container images can result in additional charges in your AWS bill. The charges for ECR mainly depend on the following two factors:
- The amount of data stored in the ECR repository
- The amount of data transferred to the internet (pushing/pulling images)
To avoid unnecessary charges in your AWS bill, it is important for users to implement lifecycle policies in ECR.
Creating lifecycle policies for container images
Using lifecycle policies with ECR can help with the following tasks:
- Delete old and orphaned images based on a user defined criteria (such as image size, image age etc)
- Manage the number of active images
To delete unwanted images, we can choose from one of the following options:
- Since image pushed X days: This means that once X days have passed since the image was pushed, it will be deleted
- Image count more than X: This option enforces the image count cap. This option will ensure that once the number of images in ECR repository exceed X, the extra images will be deleted
Example - Creating ECR lifecycle policy (based on days) In the following example, we’re going to create a lifecycle policy using the "sinceImagePushed" option. This policy will delete images that are seven days older. If you want to define multiple policies, it is a good idea to assign priority to each policy. Here, we’re giving this policy a “rulePriority” of 1, which is the highest. In case of multiple policies, this policy will be applied first.
Create a .json file and add the following contents
1{ 2 "rules": [ 3 { 4 "rulePriority": 1, 5 "description": "Expire container images older than 7 days", 6 "selection": { 7 "tagStatus": "untagged", 8 "countType": "sinceImagePushed", 9 "countUnit": "days", 10 "countNumber": 7 11 }, 12 "action": { 13 "type": "expire" 14 } 15 } 16 ] 17}
Once you’ve defined the policy in json file, create the policy as follows:
1aws ecr put-lifecycle-policy --repository-name "name_of_ECR_repository" --lifecycle-policy-text 2"file://path/to/policy.json" 3 4aws ecr get-lifecycle-policy --repository-name “name_of_ECR_repository”
Command Output-
1aws@node:~$ aws ecr get-lifecycle-policy --repository-name docker-images
Example - Creating ECR lifecycle policy (based on image count)
In the following example, we’re going to create a lifecycle policy by specifying the “imageCountMoreThan” option. This policy will delete excess images once the total number of images in the ECR repository is greater than seven. Here, we’re assigning it a priority of “2”.
1{ 2 "rules": [ 3 { 4 "rulePriority": 2, 5 "description": "Expire images when image count crosses 7", 6 "selection": { 7 "tagStatus": "untagged", 8 "countType": "imageCountMoreThan", 9 "countNumber": 7 10 }, 11 "action": { 12 "type": "expire" 13 } 14 } 15 ] 16}
How to fix- Not authorized to perform: ecr:GetAuthorizationToken on resource: * because no identity-based policy allows the ecr:GetAuthorizationToken
I faced tthe following error when I was trying to push the docker image to AWS ECR
Not authorized to perform: ecr:GetAuthorizationToken on resource: * because no identity-based policy allows the ecr:GetAuthorizationToken
Here are steps for troubleshooting the error-
- Make sure you have create a correct IAM user.
- If you are using AWS CLI then check the
.aws/credential
file and make sure you have updated the correct -- aws_access_key_id
- aws_secret_access_key
Note - If you still face the issue then look for next troubleshooting instruction
Here is reference of stackoverflow discussion
How to fix An error occurred (AccessDeniedException) when calling the GetAuthorizationToken operation: User: arn:aws:iam::ACCOUNT_NUMBER:user/MY_USER is not authorized to perform: ecr:GetAuthorizationToken on resource:
The main reason you are getting the following error is you did not set the correct permission to the IAM user inside your AWS account.
An error occurred (AccessDeniedException) when calling the GetAuthorizationToken operation: User: arn:aws:iam::ACCOUNT_NUMBER:user/MY_USER is not authorized to perform: ecr:GetAuthorizationToken on resource: *
How to set the correct permission-
-
Set the following permission to the user -
- AmazonEC2ContainerRegistryFullAccess
- AmazonECS_FullAccess
- AmazonEC2ContainerRegistryPowerUser
-
Here in my case I create a user by the name
docker-ecs-user
but initially did not set any permission. But after going through the AWS documentation I set the permission. Refer to the following screenshot -Set the correct IAM permission to AWS User for fixing the error An error occurred (AccessDeniedException) when calling the GetAuthorizationToken operation -
If you have 2FA enabled then use the following command to generate the token -
1aws sts get-session-token --serial-number arn-of-the-mfa-device --token-code code-from-token. arn-of-the-mfa-device
-
Also update the latest AWS credentials AccessKeyId, SecretAccessKey, and SessionToken using the command
aws configure
Here is reference of stackoverflow discussion
How to fix- Docker push to AWS ECR hangs immediately and times out
I had problem after creating AWS Elastic Container registery I was not able to push my docker image. After running
docker push
it hanged and timed out after sometime.
Here is timed out logs -
1The push refers to repository [870087281092.dkr.ecr.us-east-1.amazonaws.com/jhooq-test-docker-images]
271sdf4gddd83: Retrying in 1 second
3d2sdf4gddd068: Retrying in 1 second
430sdf4gd7caf5: Retrying in 1 second
528sdf4gd6682f: Retrying in 1 second
61bsdf4gd842d8: Waiting
7c3sdf4gde0ef3: Waiting
8a4sdf4gd566d3: Waiting
92asdf4gd8029a: Waiting
10EOF
How to fix?
Here is my ECR repository along with repositoryURI -
-
Tag the docker image correctly- The mistake which I did tagging the docker image name with incorrect tag. As my ECR repository name is jhooq-test-docker-images so I should tag my docker image with the same name -
1 docker build -t jhooq-test-docker-images .
-
Tag again with repositoryUri name- The second mistake which I did was I did not tag the docker image with correct AWS ECR repositoryUri. So you need to do one more tagging with repositoryUri
1 docker tag jhooq-test-docker-images:latest 242396018804.dkr.ecr.eu-central-1.amazonaws.com/jhooq-test-docker-images:latest
-
Push the image- After tagging it successfully push the image to AWS ECR docker hub and make sure you have correct repository name along with the docker image name -
1docker push 242396018804.dkr.ecr.eu-central-1.amazonaws.com/jhooq-test-docker-images:latest
Here is reference of stackoverflow discussion
Conclusion
Using Amazon’s Elastic Container service can offer multiple benefits to the end user. It offers an online fully managed Elastic Container registry, which can be either public or private. Developers and system administrators can interact with the registry like any other docker registry and can set lifecycle policies on images as per their requirements.
Posts in this Series
- How to push Docker Images to AWS ECR(Elastic Container registry)?
- How to Copy Docker images from one host to another host?
- What is persistent storage in docker and how to manage it
- Docker Installation on MacOS, Linux and Windows
- Docker - ADD, Update, Export Environment variable
- How to fix-Docker docker failed to compute cache key not found
- How to fix docker driver failed programming external connectivity on endpoint webserver?
- How to fix docker error executable file not found in $PATH?
- How to expose port on live containers?
- How to expose multiple ports with Docker?
- How to restart single docker container within multiple docker container?
- How to edit file within Docker container or edit a file after I shell into a Docker container?
- How to fix Error starting docker service Unit not found?
- How to remove old, unused images of Docker?
- How to fix docker error invalid reference format error?
- How to fix requested access to the resource is denied?
- How to fix Docker error cannot delete docker container conflict unable to remove repository reference?
- How to fix docker error no space left on device?
- How to connect localhost from docker container?
- Docker COPY vs Docker ADD?
- 6 Ways to fix - Got permission denied while trying to connect to the Docker daemon socket?
- 6 Ways to fix – Docker COPY failed: stat no source files were specified