Beginner Tutorial for Amazon S3 Service with Kotlin

Storage

Amazon S3 stands for Amazon Simple Storage Service. It is a storage service on the internet for developers to provide highly scalable, reliable and fast data storage infrastructure.

Amazon s3 provides easier storage and retrieval of large data over the web. So it becomes easy to share the application data to clients located in different locations. To know it better you may need to look at core concepts like buckets and objects.

Amazon is designed as a simple key-value store. You can store as many objects (documents, photos, videos) in it as you want. To store these objects you need to first create buckets. These buckets are AWS region-specific. A bucket can hold any number of objects in it. So Bucket and Object both are resources of S3. As it is meant for specifically developers, you must provide APIs to manage resources using it.

The default endpoint to Amazon s3 client is s3.amazonaws.com. But to access buckets (region-specific) client endpoint can be defined as:
s3.<region>.amazonaws.com

Amazon provides AWS SDK support for APIs for different languages. Here for Kotlin, we are going to use SDK for java. It comes with two API versions v1.11 and v2.0.

In this tutorial, we are going to use the AWS SDK v1.11. Let's get started with coding...

Create a new Gradle project with Kotlin. And to access S3 service in project add below dependency in the project.

compile("com.amazonaws", "aws-java-sdk-s3", "1.11.7")

Once the dependency is resolved, First we would create Amazon S3 Clientmanage the resources. To create S3 client, it requires basic AWS credentials to access the service. Additionally, it needs an endpoint to identify the service over the web.

fun initAmazonS3Client(
    endpoint: String,
    accessKey: String,
    secretKey: String
) =
    AmazonS3Client(
        BasicAWSCredentials(accessKey, secretKey)
    ).apply {
        setEndpoint(endpoint).apply {
            println("S3 endpoint is ${endpoint}")
        }
        setS3ClientOptions(
            S3ClientOptions.builder()
                .setPathStyleAccess(true).build()
        )
    }

One more thing you might have noticed till now here we are configuring the client with the option setPathStyleAccess. As per AWS documentation,

Amazon S3 supports virtual-hosted-style and path-style access in all Regions. The path-style syntax, however, requires that you use the region-specific endpoint when attempting to access a bucket.

Now we have out s3 client ready to use. Let's say you have a “first_upload_doc_s3.txt” text file stored on the S3 storage in bucket “test” earlier. Now you want to download the same file in your application to use. By using S3 client and bucket, we can simply download it using our below-defined function -

fun downloadFileFromS3(
    s3Client: AmazonS3Client,
    bucketName: String,
    targetFilename: String,
    pathToDownload: String
) =
    s3Client
        .getObject(GetObjectRequest(bucketName, targetFilename))
        .objectContent
        .use { inStream ->
            File("$pathToDownload/$targetFilename")
                .outputStream().buffered().use { outStream ->
                    inStream.buffered().transferTo(outStream)
                }
        }.also {
            println("Downloading file ${targetFilename} from S3")
        }

Here, our download function takes 4 parameters,

1. s3 client instance
2. bucketName — s3 bucket name which holds the target file.
3. targetFilename — file to download.
4. downloadPath — where the target file gets downloaded locally.

To actually download the file “first_upload_doc_s3.txt”, we have to call these defined functions from application. For testing, we are calling directly from the main function as below -

fun main() {

    val s3Client = initAmazonS3Client("s3.amazonaws.com",
        "XXXX", "XXXX")
    downloadFileFromS3(s3Client, "test",
        "first_upload_doc_s3.txt", "home/user1/Downloads")

}

In the above code, pass the <access_key> and <secret_key> for credentials provided to you by AWS. This code upon execution will download the target file to path home/user1/Downloads/first_upload_doc_s3.txt.

So using AWS SDK, we leaned to download the object from S3 storage. Likewise, you can explore the other APIs to store the objects, create and manage the buckets and many more.

Here is the links for detailed amazon s3 documentation: https://docs.aws.amazon.com/AmazonS3/latest/dev/Welcome.html

https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/welcome.html

I hope you found this tutorial helpful!

Happy Diwali :)