Loading Execution Profiles in Test Cases by Keyword

I have a GitHub project here:


Problem to solve

I hope Katalon Studio to provide a built-in keyword WebUI.loadExecutionProfile(String profileName) . The keyword should enable me to load the specified Execution Profile dynamically in a Test Case.

Let me show you an example of what I desire.

  1. I will have the default Execution Profile empty.
  2. I will create an Execution Profile named ProductionEnv . It will contain a GlobalVariable named URL with value http://demoaut.katalon.com/ .
  3. I will create another Execution Profile named MimicEnv . It will contain a GlobalVariable named URL with value http://demoaut-mimic.kazurayam.com/ .
  4. I will write a Test Case named mininal . The script code will be as follows:
import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI
import internal.GlobalVariable as GlobalVariable

println "GlobalVariable.URL=" + GlobalVariable.URL

WebUI.loadExecutionProfile("ProductionEnv")
println "GlobalVariable.URL=" + GlobalVariable.URL

WebUI.loadExecutionProfile("MimicEnv")
println "GlobalVariable.URL=" + GlobalVariable.URL

minimal_model

When I execute the script while I select the empty default profile, I hope to see the following output in the Console:

GlobalVariable.URL=null
...
GlobalVariable.URL=http://demoaut.katalon.com
...
GlobalVariable.URL=http://demoaut-mimic.kazurayam.com

Please note that the value of GlobalVariable.URL is dynamically updated by calling the WebUI.loadExecutionProfile(String profileName) Keyword in the Test Case script. You can call that Keyword multiple times in a Test Case. An Execution Profile may contain multiple GlobalVariables (name, value). All of these loaded GlobalVariables should be added/updated by the Keyword call.

I would desire similar keywords WS.loadExecutionProfile(String profileName) and Mobile.loadExecutionProfile(String profileName) as well. These will do the same thing.

Why do I need it? — I will tell it later as it requires a very long description.

However, in fact, The minimal script will not even compile because Katalon Studio does not support the keyword WebUI.loadExecutionProfile() .

Solution

I have developed a set of Groovy classes that enables my Test Case scripts to load Execption Profiles or to add/update GlobalVariables dynamically runtime. A distributable jar is provided. Once you plugin the jar into your Katalon Studio project, you can use Custom Keywords.

Description

Downloading the jar

Visit the Releases page, find the latest version of ExecutionProfilesLoader-x.x.x.jar , download it.

You should place the jar into the Drivers directory of your Katalon Studio project. Stop and restart KS in order to let KS acknowlege the added jar.

How to use it

I have already showed a minimalistic example above.

  1. Make sure you have an Execution Profile named default , which does NOT contain a GlobalVariable named URL .
  2. Create an Exceution Profile named ProductionEnv with a GlobalVariable named URL of String type with value of http://demoaut.katlaon.com/ .
  3. Create a Test Case with any name; e.g, minimal . The script should be:
CustomKeywords."com.kazurayam.ks.globalvariable.ExecutionProfilesLoader.loadProfile"("ProductionEnv")

println "GlobalVariable.URL=" + GlobalVariable.URL
  1. or you can also write
import com.kazurayam.ks.globalvariable.ExecutionProfilesLoader

new ExecutionProfilesLoader().loadProfile("ProductionEnv")

println "GlobalVariable.URL=" + GlobalVariable.URL
  1. Run the “minimal” Test Case while you choose the “default” Execution Profile. You should find the following output in the Console. The value http://demoaut.katalon.com/ is loaded from the ProductionEnv Execution Profile by the Custom keyword.
GlobalVariable.URL=http://demoaut.katalon.com/

GroovyDoc

You can read the API doc at

Long story

Why do I want a Keyword to load a Execution Profile in a Test Case?

This question deserves a long description. I will write it here as a note for me. In fact it took me 3 years to solve this problem.

demo1

A few years ago when I got started with Katalon Studio ver 5.x, I initiated a small project. I wanted to automate a task of taking a lot of screenshots of a web system. Also I learned a Katalon Studio feature Execution Profile. I thought it would be a good idea to store URL strings of the target web pages in a Execution Profile so that I can reuse a set of test script for multiple environments (Production, Development, Staging, etc).

  1. I made 2 Execution Profiles: Profiles/demoProductionEnvironment and Profiles/demoDevelopmentEnvironment . Both defines a GlobalVariable named URL1 . Each had valudes of http://demoaut.katalon.com and http://demoaut-mimic.kazurayam.com . 2ExecutionProfiles
  2. I made a Test Case script Test Cases/demo1/takeScreenshotAndReport. This script does the following processing:
  • initialize a directory for outputs
  • open browser, navigate to GlobalVariable.URL1
  • take screenshot and save it into the output directory
  • compile a html with which you can view the screenshot in browser.
  1. When you run the script demo1/takeScreenshotAndReport while you choose the Execution profile demo1ProductionEnvironment , you will obtain a HTML report which shows the screenshot of http://demoaut.katalon.com/ like this: demo1
  2. When you run the script demo1/takeScreenshotAndReport while you choose the Execution profile demo1DevelopmentEnvrironment , you will obtain a HTML report which shows the screenshot of http://demoaut-mimic.kazurayam.com/ .
  3. The script demo1/takeScreenshotAndReport is unable to take screenshots of 2 URLs and compile a single HTML report which showd 2 screenshots together.

demo2

The system had multiple environments: “Production Environment” and "Development Environement. I wanted to compare screenshots of web pages of these 2 environments to find out any unexpected differences.

Let me show you an implementation here.

  1. I made a Test Suite Collection Test Suites/demo2/TSC . You can run the demo2 by running the TSC . It includes 4 instructions to run child Test Suites.
  • Test Suites/demo2/preProcess
  • Test Suites/demo2/processURL with Execution Profile demoProductionEnvironment specified
  • Test Suites/demo2/processURL with Execution Profile demoDevelopmentEnvrironment specified
  • Test Suites/demo2/postProcess TSC
  1. Test Suites/demo2/preProcess calls Test Cases/demo2/preProcess . The Test Case does initializing the output directory
  2. Test Suites/demo2/processURL calls Test Cases/demo2/processURL . The Test Case opens the URL in browser, takes screeshot, save the image into file. The URL will be retrieved as the value of GlobalVariable.URL1 . Therefore it was necessary to execute Test Suites/demo2/processURL twice while specifying 2 Exceution Profiles demoProductionEnvironment and demoDevelopmentEnvironment for each.
  3. Test Suites/demo2/postProcess calls Test Cases/demo2/postProcess . The Test Case compiles a HTML report, which looks like this: demo2_html

Appreciation of demo2

What do you think of this implementation? I think this is terribly complexed.

A single Test Case script Test Cases/demo1/takeScreenshotAndReport could implement similar processing. On the other hand, the demo2 involves a Test Suite Collection, 3 Test Suite, 3 Test Cases. This lots of components. Why do I need them?

Test Suite Collection is the only way provided by Katalon Studio for me to apply 2 different Execution Profiles ( demoProductionEnvironment and demoDevelopmentEnvironment ) to a single script (for taking screenshots of URLs). This is the reasone why I needed this complication.

demo3

So I have developed a custom keyword com.kazurayam.ks.globalvariable.ExecutionProfilesLoader.loadProfile(String profileName) . This enabled me simplify my cripts that takes screenshots of the 2 environements and compile a single HTML report.

Test Cases/demo3/batch does the following stuff in a single script.

  • initialize the output directory
  • load Execution Profile demoProductionEnvironment to find the value of GlobalVariable.URL1 ; open the URL, take screenshot, save the image into a file.
  • load Exceution Profile demoDevelopmentEnvironement to find out the value of GlobalVariable.URL1 ; open the URL, take screenshot, save the image into a file.
  • compile a HTML report, like this: demo3_html

Appreciaton of demo3

The Test Cases/demo3/batch is short and simple enough. Far easier to understand and maintain than the complicated codes of demo2.

Conclusion

Once I developed and published VisualTestingInKatalonStudio project. I believe it is useful but is too much complicated. It’s code structure is like demo2. It involves multiple Test Suite Collections, many Test Suites, a lot of Test cases. H’m…

Now I have got an idea of simplifying that project using the ExecutionProfilesLoader . Yes, I believe it is possible to accomplish.

1 Like

i’ve been wanting a way to load an execution profile for a while now
i am testing a website on several environments, which react slightly different depending on which environment(url) i’m using
what i wanted is say in the default profile what environment it is on/should be on, and depending on that to load the correct profile with the correct variables(url, support site url, login credentials, etc), right now i am forced to either copy all the variables to the default profile, or select the right profile every time i want to run a scenario on a specific environment

also, when time is short, i want to select a random profile to execute the testcase on, rather than having to execute it on all profiles(or the same profile for every execution using testsuites, unless i change it manually), a way to load a profile after the testcase has started would also solve that need for me

The API doc is here:

Perhaps you may be interested in the com.kazurayam.ks.globalvariable.ExecutionProfileLoader class.

It can load multiple Execution Profiles at a single method call: loadProfiles(List<String> profileNames)

And you do not even need to prepare an Execution Profile by Katalon Studio GUI. The loadEntries(Map<String, Object>) enables you to update the GlobalVariable object with arbitrary instance of Map<String, Object>.

Also it provides clear() method, by which you can clear all added GlobalVariables and updates which have be made by ExecutionProfilesLoader#loadProfile() method. Once cleared out, your test case will see the list of GlobalVariables and values expressed by the Execution Profile you assigned when you start the test.

By the way, com.kazuraym.ks.globalvariable.ExpandoGlobalVariable class provides a set of static methods that enable you to retrieve the current content of the GlobalVariable object including the entries added/modified by ExecutionProfilesLoader. Try Test Cases/demo3/minimal to learn how to use it.

That’s a great post, Kaz. I do something very similar in my core code base. Kudos to you for making it transportable and reusable. :heart: :nerd_face:

One more read about GlobalVariable.

I have made a GitHub repository for this:


[Katalon Studo] How GlobalVariable works

Problem to solve

In a small Katalon Studio project, I have made 3 Execution Profiles. All of GlobalVariables are String type.

" default " Profile

name value
BASE_PARAM foo

" ProductEnv " Profile

name value
URL http://demoaut.katalon.com/
PARAM_PRODUCT ppp

" DevelopEnv " Profile

name value
URL http://demoaut-mimic.kazurayam.com/
PARAM_DEVELOP ddd

3ExecutionProfiles

I have got questions.

I have prepared 3 Execution Profiles in which 4 GlobalVariables are defined. But I have to choose one of the 3 Profiles to run a test case. For example, when I choose the ProductEnv Profile, I expect 2 GlobalVariables in that Profile URL and PARAM_PRODUCT to be present and accessible. But how about the other 2 GlobalVariables BASE_PARAM and PARAM_DEVELOP ? Are they present? If present, what value do I see?

Solution

I have developed a set of custom Groovy classes that provides capabilities to inspect internal.GlobalVariables object.

We can get answers to the questions above by a short script.

Description

I wrote a Test Case Test Cases/TC2 . This script uses com.kazurayam.ks.ExpandoGlobalVariable class to inspect the runtime state of internal.GlobalVariable object. Alos it demonstrates how to use com.kazurayam.ks.ExecutionProfilesLoader class, which enable us to dynamically add a new GlobalVariable runtime withouth preparing a Execution Profile through Katalon GUI.

I ran this script 3 times while applying Execution Profiles default , ProductEnv and DevelopEnv each. The flowing table shows the result.

Test Case output with " default " Profile

Execution Profile applied                       : default
GlobalVariable.BASE_PARAM is present?           : true
GlobalVariable.BASE_PARAM value                 : foo
GlobalVariable.URL is present?                  : true
GlobalVariable.URL value                        : null
GlobalVariable.PARAM_PRODUCT is present?        : true
GlobalVariable.PARAM_PRODUCT value              : null
GlobalVariable.PARAM_DEVELOP is present?        : true
GlobalVariable.PARAM_DEVELOP value              : null

Test Case output with " ProductEnv " Profile

Execution Profile applied                       : ProductEnv
GlobalVariable.BASE_PARAM is present?           : true
GlobalVariable.BASE_PARAM value                 : foo
GlobalVariable.URL is present?                  : true
GlobalVariable.URL value                        : http://demoaut.katalon.com/
GlobalVariable.PARAM_PRODUCT is present?        : true
GlobalVariable.PARAM_PRODUCT value              : ppp
GlobalVariable.PARAM_DEVELOP is present?        : true
GlobalVariable.PARAM_DEVELOP value              : null

Test Case output with " DevelopEnv " Profile

Execution Profile applied                       : DevelopEnv
GlobalVariable.BASE_PARAM is present?           : true
GlobalVariable.BASE_PARAM value                 : foo
GlobalVariable.URL is present?                  : true
GlobalVariable.URL value                        : http://demoaut-mimic.kazurayam.com/
GlobalVariable.PARAM_PRODUCT is present?        : true
GlobalVariable.PARAM_PRODUCT value              : null
GlobalVariable.PARAM_DEVELOP is present?        : true
GlobalVariable.PARAM_DEVELOP value              : ddd

Analysis

Here I will list concrete questions and answeres based on the above data:

Q1. When I ran a Test Case with the " default " Profile applied, is the GlobalVariable.URL present? Is it accessible for the test case?

In that case, GlobalVariable.URL is present. But the value is null.

Q2. When I ran a Test Case with the " default " Profile applied, what value of GlobalVariable.PARAM_PRODUCT can the test case see?

In that case, GlobalVariable.PARAM_PRODUCT is present. But the value is null.

Q3. When I ran a Test Case with the " ProductionEnv " Profile applied, is the GlobalVariable.BASE_PARAM defined in the default present? Is it accessible for the test case? What value of GlobalVariable.BASE_PARAM can the test case see?

In that case, GlobalVariable.BASE_PARAM is present, and the value (" foo ") is available for the test case. This means that the GlobalVariables defined in the default Profile is always visible regardles which Profile is actually chosen.

Q4. When I ran a Test Case with the " ProductionEnv " Profile applied, is the GlobalVariable.PARAM_DEVELOP defined in the DevelopEnv present? Is it accessible for the test case? What value of GlobalVariable.PARAM_DEVELOP can the test case see?

In that case, GlobalVariable.PARAM_DEVELOP is present. But the value is null.

APPENDEX

loading Execution Profiles by Keyword

The Test Case Test Cases/TC3 casts a spell:

CustomKeywords."com.kazurayam.ks.globalvariable.ExecutionProfilesLoader.loadProfile"("ProductEnv")

This is just the same as:

new ExecutionProfilesLoader().loadProfile("ProductEnv")

Also you can load multiple Execution Profiles at a time like this:

new ExecutionProfilesLoader().loadProfiles("ProductEnv", "DevelopEnv")

TC3 emits the following output in the Console:

Execution Profile applied                       : default

GlobalVariable.URL is present before loading?   : true
GlobalVariable.URL value before loading         : null

GlobalVariable.URL is present after loading Product?    : true
GlobalVariable.URL value after loading Product         : http://demoaut.katalon.com/

GlobalVariable.URL is present after loading Develop?      : true
GlobalVariable.URL value after loading Develop            : http://demoaut-mimic.kazurayam.com/

GlobalVariable.PARAM_PRODUCT is present?        : true
GlobalVariable.PARAM_PRODUCT                    : ppp

GlobalVariable.PARAM_DEVELOP is present?        : true
GlobalVariable.PARAM_DEVELOP                    : ddd

GlobalVariable.URL is present after clear?      : true
GlobalVariable.URL value after clear            : http://demoaut-mimic.kazurayam.com/

This proves that, even when I started the Test Case with the Execution Profile " default " selected, I can explicitly load other Execution Profiles " ProductEnv " and " DevelopEnv ". As you can see, the property " URL " of the internal.GlobalVariable object is updated by the values from the loaded Profiles.

APPENDIX: adding a GlobalVariable dynamically

One more Test Case Test Cases/TC4

It casts a spell:

new ExecutionProfilesLoader.loadEntries(["ADDED": "Hello, world!"])

This statement adds the GlobalVariable.ADDED with value Hello, world! . The name ADDED does not appear in the prepared 3 Execution Profiles. This means, using this custom class, you can create any number of GlobalVariables runtime.

Also another spell:

new ExecutionProfilesLoader().clear()

The clear() methods erases all properties added into the internal.GlobalVariable object by the loadEntries call.

TC4 emits the following output in the Console

Execution Profile applied                       : default
GlobalVariable.ADDED is present before loading? : false
GlobalVariable.ADDED is present after loading?  : true
GlobalVariable.ADDED value                      : Hello, world!
GlobalVariable.ADDED is present after clear?    : false
GlobalVariable.ADDED value after clear          : null

Conclusion

The artifact of ExecutionProfilesLoader will make my work on Katalon Studio easier.