AWS S3 Selectively copy files based on timestamp
by hz36t5 from LinuxQuestions.org on (#5FN8Y)
I have struggled wit this for time time now.
My goal is to do the following:
Copy files that are newer than the indicated date in the CLI script from my aws bucket to my local linux server.
aws s3 ls --recursive s3://my bucket/ | awk '$1 > "2021-03-18 00:00:00" {print $0}' | aws s3 cp --recursive my bucket/{} localfolder
I keep receiving: [Errno 32] Broken pipe
Any ideas?
My goal is to do the following:
Copy files that are newer than the indicated date in the CLI script from my aws bucket to my local linux server.
aws s3 ls --recursive s3://my bucket/ | awk '$1 > "2021-03-18 00:00:00" {print $0}' | aws s3 cp --recursive my bucket/{} localfolder
I keep receiving: [Errno 32] Broken pipe
Any ideas?