开发者

How to delete lot of objects named with common prefix from S3 bucket?

开发者 https://www.devze.com 2023-01-05 22:40 出处:网络
I have files in S3 bucket, and their names have the following format: username#file_id#... 开发者_JAVA百科How to remove all john#doe#* items, without listing them? There are thousands of them, so w

I have files in S3 bucket, and their names have the following format:

username#file_id#...
开发者_JAVA百科

How to remove all john#doe#* items, without listing them? There are thousands of them, so when user request my app to delete all of them, he has to wait.


For anyone who stumbles upon this now, you can create a lifecycle rule to either delete or set expiration of files with a certain prefix.


There's no way to tell S3 to delete all files that meet a specific criteria - you have to delete one key at a time.

Most client libraries offer a way to filter and paginate such that you'd only list the files you need to delete and you can provide a status update. For an example, Boto's bucket listing accepts prefix as one of the parameters.


I have mistakenly create logging files in same bucket and there are like tons of log file in my bucket. Luckily I came across a nodejs util node-s3-utils and it save my day!

example of delete files with foo/ prefix, having extension .txt $ s3utils files delete -c ./.s3-credentials.json -p foo/ -r 'foo\/(\w)+\.txt'

0

精彩评论

暂无评论...
验证码 换一张
取 消

关注公众号