Copy Files to S3 Using the AWS CLI

If you have to transfer large amounts of data into the AWS cloud, you need to make sure that the data somehow gets into S3 when you switch to a cloud service like S3. There are several ways to do this, and one of them is to copy the files using the AWS CLI. Two commands handle the data transfer: copy and sync.

While “aws s3 cp” simply copies files, “aws s3 sync” can, for example, synchronize a local directory with AWS S3. During synchronization, only changed files are transferred, which saves considerable resources with large amounts of data. The basic call for this is:

aws s3 sync . s3://

This copies all data in the current directory (.) to the specified bucket. Before you do this, you have to be connected to the S3 bucket (“aws configure”). Like all commands, “sync” also accepts further options, for example to explicitly include or omit certain files and directories. Here is an example in which only the files and directories that start with “1” are transferred:

aws s3 sync . s3:// —exclude '' —include “1”

Incidentally, all S3 commands of the AWS CLI can be found in the official documentation. For some applications it can also be useful to throttle the maximum transmission speed to S3.