Delete files, directories and buckets in amazon s3 java

I'm wondering how to do this. I looked at the sdk documentation and have some examples, but am confused how the syntax generally goes.

If I want to delete a file, I assume I use deleteObje开发者_开发知识库ct(path, key). However, what is the "key"?

Also how do you delete a directory? I can't seem to find a method for doing that.

---

This snippet of code works for me. folderPath is something like "topDir/secondDir/"

void deleteObjectsInFolder(String bucketName, String folderPath) {
   for (S3ObjectSummary file : s3.listObjects(bucketName, folderPath).getObjectSummaries()){
      s3.deleteObject(bucketName, file.getKey());
    }
}

---

A "key" in S3 is similar to a file path:

http://bucket.s3.amazonaws.com/some/path/to/use

... is in a bucket named bucket and has a key of some/path/to/use.

It's not actually a path though, because there are no folders. The S3 key is just the file name for a file in one big directory (the entire bucket). S3 keys can contain /, but it has no special meaning unless you set the delimiter argument with listing a bucket.

In other words, having an object named some/object doesn't tell you anything about the object some (it might or might not exist -- the two objects are not related).

However, you can request keys with a specific prefix, so I could say "give me all keys starting with some/path/to/ and it will return some/path/to/use. It looks like "listing a directory", but it's really just asking for files that start with a specific string of characters.

I could just as easily name things like this:

somepathtousea
somepathtouseb

And say "give me everything starting with somepathtouse" (and it would say somepathtousea and somepathtouseb).

Note: S3 URL's come in several forms:

http://s3.amazonaws.com/bucket/key
http://bucket.s3.amazonaws.com/key
http://bucket/key (where bucket is a DNS CNAME record pointing to bucket.s3.amazonaws.com)

EDIT:

I looked at the JavaDocs and this is the function signature I see (for AmazonS3Client):

public void deleteObject(java.lang.String bucketName,
                         java.lang.String key)
                  throws AmazonClientException,
                         AmazonServiceException

EDIT again:

Folders do kind-of exist now, as zero-length objects with a content-type of application/x-directory and a key ending in /:

$ AWS_PROFILE=prod aws s3api head-object --bucket example-bucket --key example-directory/
{
    "AcceptRanges": "bytes",
    "LastModified": "Mon, 29 Apr 2019 14:59:36 GMT",
    "ContentLength": 0,
    "ETag": "\"d41d8cd98f00b204e9800998ecf8427e\"",
    "ContentType": "application/x-directory",
    "ServerSideEncryption": "AES256",
    "Metadata": {}
}

This is still just convention and there's nothing stopping you from having files ending / or files inside of "folders" that don't exist.

---

You might want to take a look at this example for a quick reference on how you can delete objects from S3.

The syntax for delete is actually deleteObject( bucketName, key ) where bucketName is the bucket in which you have placed your files and key is name of the file you want to delete within the bucket.

Think of a bucket as your hard disk drive like C:\ , D:\ etc. And key as the absolute pathname of a file you want to delete.

---

/*Here is solution that works for me. Here Bucket_Name is my bucket name on S3, and key is the path under Bucket_Name. So, if absolute path on S3 is:

s3://my_bucket/Path/to/my/folder

then, the code below should work. */

---

    String Bucket_Name = "my_bucket";
    String key = "Path/to/my/folder";   
    ObjectListing objects = s3Client.listObjects(BUCKET_NAME, key);
        for (S3ObjectSummary objectSummary : objects.getObjectSummaries()) 
            {
            s3Client.deleteObject(BUCKET_NAME, objectSummary.getKey());
            }           

---

As question is asking about Deleting files, directories and buckets in amazon S3 java, I would like to offer code for deleting a non-empty S3 bucket (AWS Reference):

 public void deleteBucket(final String bucketName) {

        final AmazonS3 s3 = AmazonS3ClientBuilder.defaultClient();

        try {
            ObjectListing objectListing = s3.listObjects(bucketName);
            while (true) {
                for (Iterator<?> iterator = objectListing.getObjectSummaries().iterator(); iterator.hasNext(); ) {
                    S3ObjectSummary summary = (S3ObjectSummary) iterator.next();
                    s3.deleteObject(bucketName, summary.getKey());
                }

                if (objectListing.isTruncated()) {
                    objectListing = s3.listNextBatchOfObjects(objectListing);
                } else {
                    break;
                }
            }

            VersionListing versionListing = s3.listVersions(new ListVersionsRequest().withBucketName(bucketName));
            while (true) {
                for (Iterator<?> iterator = versionListing.getVersionSummaries().iterator(); iterator.hasNext(); ) {
                    S3VersionSummary vs = (S3VersionSummary) iterator.next();
                    s3.deleteVersion(bucketName, vs.getKey(), vs.getVersionId());
                }

                if (versionListing.isTruncated()) {
                    versionListing = s3.listNextBatchOfVersions(versionListing);
                } else {
                    break;
                }
            }

            s3.deleteBucket(bucketName);
        } catch (AmazonServiceException e) {
            System.err.println(e.getErrorMessage());
        }
    }

---

Works for me, beware of truncation!

    long start = System.currentTimeMillis();
    long totalSize = 0;
    int totalItems = 0;

    String key ="path/to/folder/"
    String bucket = "my-bucket"

    final ListObjectsRequest listObjectsRequest = new ListObjectsRequest().withBucketName(bucketName).withPrefix(key);

    ObjectListing objects = s3.listObjects(listObjectsRequest);
    do {
        for (S3ObjectSummary objectSummary : objects.getObjectSummaries()) {
            totalSize += objectSummary.getSize();
            totalItems++;
            s3.deleteObject(bucketName, objectSummary.getKey());
        }
        objects = s3.listNextBatchOfObjects(objects);
    } while (objects.isTruncated());

    long stop = System.currentTimeMillis();

    LOG.trace("User {} had {} items with {} Kb, took {} ms to delete", user.getName(), totalItems, totalSize / 1024, stop
            - start);

---

The ListObjectsV2Result worked for me. Try once.

private void deleteObjectsInFolder() {
    try {
        ListObjectsV2Result result;
        do {
            String folderPath =   " ";



            result = s3.listObjectsV2(Constants.BUCKET_NAME, folderPath);

            Log.e("count:", result.getKeyCount() + "");

            if (result.getKeyCount() != 0) {

                for (S3ObjectSummary objectSummary :
                        result.getObjectSummaries()) {
                    s3.deleteObject(Constants.BUCKET_NAME, objectSummary.getKey());
                }

            }


            System.out.println("Next Continuation Token : " + result.getNextContinuationToken());
        } while (result.isTruncated() == true);

    } catch (AmazonServiceException ase) {
        System.out.println("Caught an AmazonServiceException, " +
                "which means your request made it " +
                "to Amazon S3, but was rejected with an error response " +
                "for some reason.");
        System.out.println("Error Message:    " + ase.getMessage());
        System.out.println("HTTP Status Code: " + ase.getStatusCode());
        System.out.println("AWS Error Code:   " + ase.getErrorCode());
        System.out.println("Error Type:       " + ase.getErrorType());
        System.out.println("Request ID:       " + ase.getRequestId());
    } catch (AmazonClientException ace) {
        System.out.println("Caught an AmazonClientException, " +
                "which means the client encountered " +
                "an internal error while trying to communicate" +
                " with S3, " +
                "such as not being able to access the network.");
        System.out.println("Error Message: " + ace.getMessage());
    }
}

---

Deleting a list of objects from S3 bucket by bulks:

public void deleteObjects(String bucketName, List<String> keys) {

    List<KeyVersion> bulk = new ArrayList<>();
    for (int i = 0; i < keys.size(); i++) {
        bulk.add(new KeyVersion(keys.get(i)));
        if (i % 100 == 0) {
            try {
                s3Client.deleteObjects(new DeleteObjectsRequest(bucketName).withKeys(bulk));
            } catch (Exception e) {
                System.err.println(e.getErrorMessage());
            }
            bulk.clear();
        }
    }
    if (bulk.size() > 0) {
        try {
            s3Client.deleteObjects(new DeleteObjectsRequest(bucketName).withKeys(bulk));
        } catch (Exception e) {
            System.err.println(e.getErrorMessage());
        }
    }
}

Source: http://codeflex.co/delete-objects-from-amazon-s3-bucket-using-aws-sdk-for-java/

---

This line of code works in my case where the keyName is the file name:

s3Client.deleteObject(new DeleteObjectRequest(bucketName, keyName));

---

kotlin

class S3(
    var bucketName: String? = null,
    var key: String? = null,
    val accessKey: String? = null,
    val secretKey: String? = null,
    val region: String? = null
)

fun delete(
    s3: S3,
    keyword: String = "",
) {
    with(s3) {
        val client = client(accessKey, secretKey, region)

        var objects = client.listObjects(bucketName, key)
        while (true) {
            for (i in objects.objectSummaries) {
                if (!i.key.contains(keyword)) {
                    continue
                }
                client.deleteObject(bucketName, i.key)
            }

            if (objects.isTruncated) {
                objects = client.listNextBatchOfObjects(objects)
            } else {
                break
            }
        }

        var versions = client.listVersions(bucketName, key)
        while (true) {
            for (i in versions.versionSummaries) {
                if (!i.key.contains(keyword)) {
                    continue
                }
                client.deleteVersion(bucketName, i.key, i.versionId)
            }

            if (versions.isTruncated) {
                versions = client.listNextBatchOfVersions(versions)
            } else {
                break
            }
        }
    }
}

---

Here's my answer which worked for me, I used the answer from Danger

This takes time but eventually it deletes all the files and folder.

public static void deleteObjects(AmazonS3 s3Client, String bucketName, String folderPath)
    {
    while(getFilesListInBucket(s3Client, bucketName, folderPath).size()>0) {
        for (S3ObjectSummary file : s3Client.listObjects(bucketName, folderPath).getObjectSummaries()) {
            s3Client.deleteObject(bucketName, file.getKey());
        }
    }
}