Create test data file for different environments outside katalon project

[rajeshri.zade]

Hello All,

I have seen solutions to a similar issue which I am facing at the moment. However, I have one specific requirement due to which I am seeking help on the forum.

My requirement is :

For different environments, I have different URLs like customer_name.env_name.com
I am using Global variables along with test execution profiles in KS to achieve the same.

As long as we create same name customer the framework works smoothly. But I do not want to restrict this. I want to pass this customer name from file (or any other option). I can use test data here , but the problem is test data will reside inside the Katalon project and it is hard to change that test data file every time we have new customer ( and few other things which changes per deployment used for API testing). The problem is not just to update the file , test data file check in , tagging it so that it can run through CI CD pipeline creates lot of work and is definitely not recommended.

Has anyone of you faced similar issue?

I saw @kazurayam @jason.tolotta @manpreet.mukkar comments on similar issue. Could you please guide me.

[kazurayam]

@rajeshri.zade

the problem is test data will reside inside the Katalon project and it is hard to change that test data file every time we have new customer

Too little information what you have now. Please describe your problem while sharing your current code, screenshots etc. I had the following questions about your description:

1. Where the test data file resides in your current Katalon project?
2. Why you find it is hard to change? You can edit a text file with your favorite editor. Isn’t it enough? Why?
3. It seems that you want to locate the test data outside the Katalon project. Then what is your favorite location to store the data? Windows shared file server? NFS? Some RDBMS? Aws S3 or Google Drive? Download data from some URL?

[Brandon_Hein]

I agree with @kazurayam, it sounds like this requirement boils down to this:

From where would you like to retrieve the data?

This will drive the solution. Once you’ve figured out a way to reliably source the data, then building the URL from that data is a trivial task.

[rajeshri.zade]

I apologise if the problem was not clear. Here is the problem with my current setup of Katalon project.

data or test data which is different for different environments is basically stored in different execution profiles using Global variables concept.

This data also includes some URI Codes which are specific to the environments (and varies to each deployment on any given environment.

In order to run the test suites after every deployment through CI CD pipeline, I have to change these values (which I only get to know after the deployment) by changing those global variables in all the execution profile , check in the code and tag the code. ( yes I have to tag the code every time I change it as the current CI CD pipeline picks only the latest tagged code - so when I change execution profile data I have to do this process)

Here is an example of fields I pass it dynamically ( w.r.t env and for each deployment)

That’s why I can not keep this data inside Katalon project as I will have to check-in and tag every time.

And hence I am seeking for solution where in I can store this information/data outside of Katalon project. And then pass this information/data to Katalon execute CLI command.

1. Where the test data file resides in your current Katalon project? Currently we are not using any test data. We are using Global Variables. The values of these variables are different for different profiles like SIT, UAT etc. I have tried this using test data file but what I notices there is test data file will reside in your katalon project under ‘Test Data’ folder. So, the challenge I have now with Global variables is with Test Data file too.

2. Why you find it is hard to change? You can edit a text file with your favorite editor. Isn’t it enough? Why?
   Its a problem when I have to run it through CI CD pipeline! I cannot keep changing these things every now and then. So I want to have these things outside of Katalon project and then anyone can change it. (AT the moment only team members who are working on Katalon project can change it no one else can)

3. It seems that you want to locate the test data outside the Katalon project. Then what is your favorite location to store the data? Windows shared file server? NFS? Some RDBMS? Aws S3 or Google Drive? Download data from some URL? Thanks for all these options. I will need to think on these options. I have to keep in mind that basically I have to pass this file to my Katalon project which will read the data stored in that file.

[kazurayam]

Your question seems to be simple. You want to build a URL string interpolating $variables with values read from a file, rather than values from GlobalVariables. You want to know how to. Am I right?

1. Could you share code fragment of your test case script which executes (indirectly?) “GET http://${uri_code_user}.execute-api.${aws_region}.amazonaws.com/${env}/${aws_region}/usermanagement/users/${tenantId}”

2. Could you provide a set of sample values of the variables: uri_code_user, aws_region, env, aws_region, tenantId. The values can be psuedo one, need not be the actual (sensitive) values. I want this sample to make our discussion concrete and easier to understand.

uri_code_user	aws_region	env	aws_region	tenantId
?	?	?	?	?

[rajeshri.zade]

@kazurayam

?

Yes. that’s right. keeping in mind whatever is the solution should work through CI CD pipepline. At the moment , the command through which test suite gets executed is as follows (inside Jenkins file)

katalon-execute.sh -browserType=“Chrome” -retry=0 -testSuitePath=“${testSuitePath}” -apiKey=${API_KEY} -executionProfile=${profile}

@kazurayam

Could you share code fragment of your test case script which executes (indirectly?) “GET http://${uri_code_user}.execute-api.${aws_region}.amazonaws.com/${env}/${aws_region}/usermanagement/users/${tenantId}”

Another example

test case is

API is

Global variables from regression env profile

@kazurayam

Could you provide a set of sample values of the variables: uri_code_user, aws_region, env, aws_region, tenantId. The values can be psuedo one, need not be the actual (sensitive) values. I want this sample to make our discussion concrete and easier to understand.

uri_code_user (different for every env and changes every deployment)	env	aws_region(value from defined set)	tenantId ( different if we create different tenant after every deployment; so these are just samples)
b17f78u2zj	dev	us-west-2	wd4um1uygarr1yi2
4tvidr7wp6	test	ap-southeast-2	ncswtkvqfcidt99d
fvhbagoedi	regr	ap-southeast-2	zywpcak4oz96aon5
f0h27oznl9	uat	us-west-2	amyudwxowgawfegt
c8n818ihie	prod	us-west-2	5y4xq4sk9ylxt665

[kazurayam]

Now I see you have 5 sets of variable values.

Then do you want to iterate over all of these 5 sets in a test case run?

Or do you want to choose only one set out of these per a test case run?

[kazurayam]

Let’s assume you found a way to store the variable values in a CSV or Excel file.

And in the original post, you mentioned:

it is hard to change that test data file every time we have new customer

Then how do you want to change the file when you got a new customer?

Do you want in append a line at the tail of the file?

[rajeshri.zade]

@kazurayam

Then do you want to iterate over all of these 5 sets in a test case run?

Or do you want to choose only one set out of these per a test case run?

Only one set (valid for one env)

[rajeshri.zade]

@kazurayam

And in the original post, you mentioned:

it is hard to change that test data file every time we have new customer

Then how do you want to change the file when you got a new customer?

it is hard to change the file when the file resides inside Katalon project. ( because any change in any of the file inside katalon project needs to be checked in and tagged in order to work through CI CD pipeline - that’s the way it is set up now)

However, if it is outside, anyone can change these fields’ values according to the name of the tenant/customer created for that env and rest of the fields too.

[kazurayam]

@rajeshri.zade

if it is outside

Please be a bit more specific. The file can be located anywhere outside the katalon project. But where do you want to locate it? Is it ok, as an example, to locate the file somewhere in your local folder of the machine where the test runs, of course outside the project?

[rajeshri.zade]

@kazurayam

Is it ok, as an example, to locate the file somewhere in your local folder of the machine where the test runs, of course outside the project?

that’s ok but then how is it going to fetch this file when test suite runs through CI CD pipeline. I should have also mentioned in my earlier comments how exactly it works through CI CD now. That will probably make things clear.

The way it gets run through CI CD is - we get to run our build for different environments which is mapped to different katalon profiles

Jenkins file has (below is very high level logic)

1. Map which stores these katalon profiles (i.e. env names) and tags to be used for these env ( store this in ‘profile’)
2. Create a docker image of katalon studio
3. Run the command katalon-execute.sh -browserType=“Chrome” -retry=0 -testSuitePath=“${testSuitePath}” -apiKey=${API_KEY} -executionProfile=${profile}

Now, if we keep the test data/fields’ value in file which is local on my machine; how can this Jenkins file (or rather that docker image ) fetch the local file?

Hope this is more clear now. I apologize for not mentioning this before.

[anon46315158]

@rajeshri.zade

it is hard to change the file when the file resides inside Katalon project. ( because any change in any of the file inside katalon project needs to be checked in and tagged in order to work through CI CD pipeline - that’s the way it is set up now)

However, if it is outside, anyone can change these fields’ values according to the name of the tenant/customer created for that env and rest of the fields too.

since you need only one set of data per/execution, the easiest way is to simply use global variables (one profile for each case) as you already did. test data won’t help you too much here.

to add more flexibility, since you are using jenkins to execute them, you can create also a dummy profile and substitute the variable values from the running script just before the execution, using sed. in the end, a profile is just an xml file … easy to parse it.
for each variable needed, you can add a build variable in the jenkins job (with default values) so anybody using the job can pass custom values to it.

or, you can use the ‘override global variable’ feature from command line, see the doc:

https://docs.katalon.com/katalon-studio/docs/console-mode-execution.html#general-options

-g_XXX 	

Starting from Katalon Studio version 5.9

Override Execution Profile variables.

Example:

-g_userName="admin"

[kazurayam]

Let me confirm that the content of the “file” you mean is the following table, am I right?

|uri_code_user | env|aws_region |tenantId|
|:—|:—|:—|:—|:—|
|b17f78u2zj|dev|us-west-2|wd4um1uygarr1yi2|
|4tvidr7wp6|test|ap-southeast-2|ncswtkvqfcidt99d|
|fvhbagoedi|regr|ap-southeast-2|zywpcak4oz96aon5|
|f0h27oznl9|uat|us-west-2|amyudwxowgawfegt|
|c8n818ihie|prod|us-west-2|5y4xq4sk9ylxt665|

By the way in the original post, you mentioned :

@rajeshri.zade

The problem is not just to update the file , test data file check in , tagging it so that it can run through CI CD pipeline creates lot of work and is definitely not recommended.

I find this sentence is difficult to understand. To me the following one line looks something mysterious and complexed enouth; well worth being committed into Git.

fvhbagoedi, regr, ap-southeast-2, zywpcak4oz96aon5

Do you and your team want to update this line often and casually without any source code control? I do not see why you want it.

[kazurayam]

You want to get a URL string for example

https://fvhbagoedi.execute-api.ap-sourtheast-2.amazonaws.com/regr/ap-sourtheast-2/usermanagement/users/zywpcak4oz96aon5

I guess this URL is dynamic. “usr_code_user” would be determined by AWS dynamically; you can not make them fixed, right?

If so, it is not a good idea to write the values of “uri_code_user”, “aws_region” and “tenantId” statically regardless as GlobalVariable or external file.

How about making a query to AWS to look up currently-available URL for the “regr” environment?

How to make a query to AWS? — You can do it by developing a bash script which employs AWS CLI or a TestCase script in Groovy which employs AWS SDK for Java.

[rajeshri.zade]

@anon46315158 Thanks. Let me give a try.

[rajeshri.zade]

@kazurayam

Let me confirm that the content of the “file” you mean is the following table, am I right?

yes it can be one file with all this data OR one file with one row (i.e. env) in each ( not ideal/recommended though)

@kazurayam

I find this sentence is difficult to understand. To me the following one line looks something mysterious and complexed enouth; well worth being committed into Git.

The data in this one line will change for every deployment ( which is almost daily especially uri_code_user and tenantID). we definitely do not want to update my global variables default values for every deployment.

[rajeshri.zade]

@kazurayam

I guess this URL is dynamic . “usr_code_user” and “tenatId” would be determined by AWS dynamically; you can not make them fixed, right?

Exactly. that is determined by AWS and not fixed.

@kazurayam

How about making a query to AWS to look up currently-available URL for the “regr” environment?

seems to be a better solution! Let me try.

[kazurayam]

You are using AWS API-Gateway service. API-Gateway service allows you to create a Custom Domain Name, which is fixed, and associate it with the default base url format, which is dynamic. Once you set up a Custom Domain Name, you can test URL with fixed prefix, which is far easier to deal with.

See the following AWS:

https://docs.aws.amazon.com/apigateway/latest/developerguide/how-to-custom-domains.html

---

“tenantId” seems to be out of the control of the AWS API-Gateway service. I guess it is controlled by your Application Under Test. “tenantId” seems to be the only URL component that should be parameterised in your test suites in Katalon Studio.

Let me assume you have created a Custom Domain Name api.example.com, and choose base path such as ‘myservice/regr’. You will have a URL to test: https://api.example.com/myservice/regr/zywpcak4oz96aon5

So, you want to create a set of TestObjects with following URL template

• https://api.example.com/myservice/dev/${tenantId}
• https://api.example.com/myservice/test/${tenantId}
• https://api.example.com/myservice/regr/${tenantId}
• https://api.example.com/myservice/uat/${tenantId}
• https://api.example.com/myservice/prod/${tenantId}

In the Katalo Studio test, the variable ${tenantId} will be interpolated with GlobalVariable.tenantId. And you run your TestSuite passing the value of “tenantId” from the command line.

katalon-execute.sh ... -g_tenantId="zywpcak4oz96aon5"