AWS CLI has become a life saver when yous desire to manage your AWS infrastructure efficiently.

This also provides lot of possibilities for Automation and reduce the number of times that you have to login to AWS Management console.

If you are new to AWS CLI. Please consider reading this article on how to setup AWS CLI

With no further ado, Lets get into the objective of this post.

aws s3 cp

In this post nosotros are going to talk most a very specific command of AWS CLI which is AWS S3

While AWS S3 provides complete tool set to manage your S3 bucket. nosotros are going to see one specific characteristic of S3 CLI today. which is re-create

Before we get there. we need to empathize few things nigh AWS S3 CLI

There are two AWS S3 CLI commands available

  • AWS S3
  • AWS S3API

Let u.s. see what they both have to offer.

What is AWS S3 and S3API

The AWS CLI provides two tiers of commands for accessing Amazon S3

The s3 tier consists of high-level commands that simplify performing common tasks, such as creating, manipulating, and deleting objects and buckets.

The s3api tier behaves identically to the aforementioned S3 tier but information technology enables you lot to carry out avant-garde operations that might not be possible with s3 tier.

In this commodity we are going to talk about only the s3 tier and very specifically s3 cp command which helps u.s.a. copying files from and to S3 buckets.

Earlier going any further I want you to know few handy commands which help to list the buckets.

  1. aws s3 aid – To get a list of all of the commands available in high-level commands.
  2. aws s3 ls – To go the list of all buckets.
  3. aws s3 ls s3://bucket-name – Volition list all the objects and folders I that bucket.
  4. aws s3 ls s3://bucket-name/path/ – This command will filter the output to a specific prefix.

Quick Caveats on AWS S3 CP command

  • Copying a file from S3 bucket to local is considered or chosen every bit download
  • Copying a file from Local system to S3 bucket is considered or chosen equally upload
  • Please be warned that failed uploads tin can't be resumed
  • If the multipart upload fails due to a timeout or is manually cancelled past pressing CTRL + C, the AWS CLI cleans up any files created and aborts the upload. This procedure can take several minutes.
  • If the process is interrupted by a kill control or system failure, the in-progress multipart upload remains in Amazon S3 and must be cleaned up manually in the AWS Management Console or with the s3api abort-multipart-upload command.

AWS S3 CP Command examples

Here nosotros accept listed few examples on how to use AWS S3 CP command to copy files.

Copying a local file to S3

Uploading a file to S3, in other words copying a file from your local file organisation to S3, is done with aws s3 cp command

Let'southward suppose that your file proper name is file.txt  and this is how you can upload your file to S3

aws s3 cp file.txt s3://saucepan-proper noun

while executed the output of that control would like something similar this.

aws s3 cp

Copying a local file to S3 with Storage Class

S3 Provides various types of Storage classes to optimize the cost and to manage the deejay efficiency and IO operation during file read and write operations.

  • S3 Standard
  • S3 Intelligent-Tiering
  • S3 Standard-IA
  • S3 One Zone-IA
  • S3 Glacier
  • S3 Glacier Deep Archive

Yous can read more information well-nigh all of them here

aws s3 cp file.txt s3://bucket-proper noun --storage-class course-name

aws s3 cp

Panel Output:

In the prceding snapshot yous can see that the test2.txt file which we have uploaded just now is showing the Standard-IA equally the storage course

Copying an S3 object from one bucket to another

At times we would want to copy the content of one S3 saucepan to some other S3 bucket and this is how it tin be done with AWS S3 CLI.

aws s3 cp s3://source-bucket-name/file.txt s3://destination-bucket-proper noun/

aws s3 cp

How to Recursively upload or download (copy) files with AWS S3 CP command

When passed with the parameter --recursive  the aws s3 cp control recursively copies all objects from source to destination.

It can exist used to download and upload large gear up of files from and to S3.

Here is the AWS CLI S3 command to Download listing of files recursively from S3.  here the dot . at the destination terminate represents the current directory

aws s3 cp s3://bucket-name . --recursive

aws s3 cp

the same command can be used to upload a big set up of files to S3. past but changing the source and destination

aws s3 cp . s3://bucket-name  --recursive

Here we have just changed source to current directory and destination to the bucket and now all the files on the current directory(local) would be uploaded to the bucket.

It includes all the subdirectories and subconscious files

Setting the Access Command List (ACL) while copying an S3 object

For security and compliance reasons, we might desire to restrict the files that we are copying to S3 buckets with set of Access control. Like Read just or Read Write etc.

Here is the control that copies a file from 1 bucket to some other bucket with specific ACL practical on the destination bucket ( afterward copied )

aws s3 cp s3://source-bucket-name/file.txt s3://dest-bucket-name/ --acl public-read-write

At that place are 5 types of ACL permissions available with S3 which are listed here on the post-obit snapshot.

aws s3 cp
Image credits to AWS

Promise this helps.

How to copy files between EC2 and S3

We have then far discussed how to re-create files from local to EC2 and vice versa.

When it comes to EC2 instances it is a little unlike as the authentication can be taken care of by IAM roles assigned to the server.

S3 buckets can besides accept certain bucket policies to allow servers coming from a sure AWS business relationship of VPC endpoint etc.

We take created an commodity that talks about how to easily copy files betwixt EC2 and S3 with a simple 4 step configuration.

Copy files from EC2 to S3 Bucket in 4 steps

Automate S3 tasks with Ansible

We accept an sectional article covering diverse examples of Ansible S3 module usage and examples.

If you are looking for some automation with S3. I would recommend yous to requite information technology a try.

How to use ansible with S3 – Ansible aws_s3 examples | Devops Junction

If y'all accept any feedback or all-time practices. please feel free to comment and let us know.

Cheers
Hanumanth

gritfy