How do I store variables in S3 and access them from a KS test script?

I had a bit of a challenge presented to me. I need to store variables to be used in KS test scripts (such as e-mail addresses and passwords for logins) in S3 (for security reasons) and populate them into fields as the test script runs. Has anyone done this before? If so, I have no idea where to begin implementing something like this and could use some help. Storing them in the KS project itself is not an option (even with the built in password encryption), per the higher ups.

You can mount a S3 Bucket as a Windows Drive using some tools installed on your PC. For example, quick search showed this:
https://tntdrive.com/mount-amazon-s3-bucket.aspx

Once it is mounted, it is just a Windows drive (or a folder). You should be familiar with reading/writing files on it.

1 Like

You can try using AWS SDK for Java in Katalon Studio, though there could be a lot of hurdles for you to overcome. The following post shows one of the hurdles I encountered previously.

Hello, I love this and thanks, but unfortunately I’m not familiar with how to code in Katalon writing/reading files to/from a folder, can you provide me an example of how to do that? I want to take the json file or text that I receive from an API response and write that to a S3 bucket, how do I code this in Katalon? Thanks in advance and I appreciate your help.

A simple example:

import java.nio.file.Files
import java.nio.file.Path
import java.nio.file.Paths

import com.kms.katalon.core.configuration.RunConfiguration

Path projectDir = Paths.get(RunConfiguration.getProjectDir())
Path tmpDir = projectDir.resolve("tmp")
Files.createDirectories(tmpDir)
Path jsonFile = tmpDir.resolve("hero.json")

String jsonText = """{
   "name": "Shohei Ohtani",
   "team": "Los Angels Angels"
}"""

// write a text into file
File f = jsonFile.toFile()
f.text = jsonText

// read a file
String s = f.text
println s

1 Like

If you are going to bother yourself working on AWS SDK, please go through the AWS tutorials first.

Once you have got familiar with AWS SDK for S3, then you want to configure your Katalon Studio project so that it has all required extenal dependencies (jar files) in the Drivers folder. Katalon Studio does NOT support resolving dependencies for S3 out of box. You need to resolve it yourself. The following post explains how to do it.

@Stephanie.LovingJone

Mounting Amazon S3 Bucket to make it accessible as a local directory would be an easier solution for you than using AWS SDK.

Search Google with key “mount S3 windows”, then you will find a lot of proposals. For example

The TntDrive possibly uses AWS SDK internally. It hides all complexities and it maps a directory of local file system to a S3 bucket so that you do not need to worry about the AWS SDK at all.

2 Likes

WOW U ROCK!!! Thanks soooo much for this quick response, I will give your sample a try and provide feedback, this is exciting and I really do appreciate you, having these answers will definitely give me a jump start at completing this task for my team, thank you soooo much.

1 Like

S3 buckets can be accesed also through REST API, see the quick guide:

It is not trivial, you will have to code first your own solution to sign the request.
Katalon does not have support implemented for this.
Start to read here on this matter:

Luckily they provide also a java sample, should work as it is:

Another way is, you can expose the S3 bucket as an NFS share.
You have to setup an NFS Gateway first, see here:

Compared with the Windows Drive driver solution proposed before, mounting an NFS will work also on Linux and Mac systems so you can use that to execute the tests on whatever environment.

Alternate, you can setup a SMB (CIFS) share:

I have ever used the S3FS library. With it, I could let my Katalon Test case read/write to AWS S3 via java.nio.file.Path interface.

I was successful.

I am sure you can use it as well. You do not need to purchace products like TntDrive if you are capable to use the S3FS.

1 Like

This worked perfectly, wooo hooo, thanks again, now on to testing my Katalon script to read and write to my S3 bucket, lots of fun.

Sorry, not sure how to declare the mapped path. Now that my S3 bucket is mapped to windows directory Z:\MyS3Bucket do I set it to a String and then re-declare it as a Path? The example is perfect because the getProjectDir is already declared, so what’s the syntax that I need to use for my Z:\MyS3Bucket? Thanks in advance for your help.

Learn the following article for the Java8 Path API.

Possibly you want to write

Path dirMappedToS3 = Paths.get('Z:\MyS3Bucket')

Awesomeness, that did it, U ROCK AGAIN!!! You make this look so easy :slight_smile: thank you my friend

No, I don’t. AWS SDK does. AWS SDK is so well-designed that it performs perfect underneath the surface.