How to pass variables between test cases in a suite (the "right" way...)

[Brandon_Hein]

Hey all,

I would just like to share a super simple approach to passing variables between test cases in a suite, as I’ve seen many topics over the years with questions on how to do this efficiently. To be clear, there are already several approaches to this, including, but not limited to:

1.) Creating “empty” GlobalVariables in a Profile, assigning values to those variables at runtime, then consuming them as a global variable in later scripts.

• This approach works fine, but in my opinion is non-ideal, as it contaminates the GlobalVariable namespace with a bunch (potentially 1000’s) of empty variables of global scope.

2.) Using callTestCase() with parameters.

• This also works fine, but is cumbersome both syntactically and in terms of test case organization, “Variables” tab usage, etc. (again, in my opinion).

3.) Writing the variables to data files.

• This can also be quite cumbersome, and of course any data will persist even after a test case/test suite ends, which has the potential to cause a lot of problems.

Instead, we can use static collections to do this.

Create a utility class which just instantiates a few different types of collections, with a static reference:

What this means is that any values put into these collections will be available for the lifetime of any Test Case and/or any Test Suite. For example, if I have two test cases, and put both into a suite, I can pass a variable generated in TC1 into the map, and retrieve that value when the suite runs TC2:

TC1:

TC2:

Test Suite:

Output:

To demonstrate that the lifetime of this variable map is that of the test suite (and no longer), if I then just run TC2 alone, you’ll see that the value is no longer in there:

Disclaimer: I’m aware that this is nothing groundbreaking. All it takes is understanding the nature of static references. I’m just sharing this because I’ve yet to see this solution posted anywhere. Also, I generally write scripts in such a way that there are zero external dependencies (they are “hermetically sealed”), but have some colleagues that had this requirement, so yeah…

Hopefully this is helpful

[Russ_Thomas]

@Brandon_Hein

I’ve yet to see this solution posted anywhere

I’ve posted it but you’re right in the sense that I didn’t make a point of describing the full mechanism. I have a map GLOBALS which is populated from GlobalVariables, JSON data read from disk and a few other bits and pieces.

Yeah, it’s not very neat and could do with a cleanup

Great post though, Brandon. Bookmarked.

[Brandon_Hein]

@Russ_Thomas

I’ve posted it but you’re right in the sense that I didn’t make a point of describing the full mechanism. I have a map GLOBALS which is populated from GlobalVariables, JSON data read from disk and a few other bits and pieces.

Link? Thanks!

[Russ_Thomas]

Here’s one: How to use Global Variables between test suites while executing Test Suite Collection?

Context is different but you get the gist.

[Brandon_Hein]

Like you said, kinda hard to tell what’s happening, but I agree, it’s the same general idea. Nice.

[Russ_Thomas]

Yes, no, kinda…

Yours is far more robust, especially in terms of types. Mine is minimalist by comparison (belies my JS tendencies, relying on coercion etc). But that’s okay, I can call on you when I need something more-better-good

[anon46315158]

Boths solutions are great, used with the right context.
The one proposed by @Brandon_Hein may be faster since everything is stored in memory, but won’t persist across suites in a collection. With the caveat that … with large amount of data can be memory hungry
The one presented by @Russ_Thomas may be a bit slower but will work across suites and will preserve the final data set for debug if needed. And with a proper flush-to-disk routine can be memory friendly.
One may use a combination of those, using properly written hooks
Nice job, guys! Kudos!

L.E. one day i will present my POC using an sqlite db. That may solve both remaining issues. But not yet … everybody knows I am lazy …

[Russ_Thomas]

@anon46315158

one day i will present my POC using an sqlite db.

One too many moving parts for my tastes.

[Brandon_Hein]

@anon46315158

but won’t persist across suites in a collection

This is true. Good point.

@anon46315158

With the caveat that … with large amount of data can be memory hungry

In theory, sure. In my example, I’ve allowed the collections to contain Objects, so if the script-writer added many many large objects, the heap may be exceeded. But:

1.) This could be tempered by only allowing Strings in the collections (i.e. <String, String> instead of <String, Object>, etc.).
2.) Probably 99% of the time, the script-writer would only every use this for storing String vars anyway. You would need several million of these entries before having to worry about memory.

Great observations!

[gdearest07]

@Brandon_Hein, would it work if the test suite is being run in parallel (e.g. in multiple browsers)?

[Brandon_Hein]

No, these collections are not thread safe. You could easily make them so by throwing them into a ThreadLocal though, for example:

public static ThreadLocal<Map<String, Object>> map = new ThreadLocal<Map<String, Object>>();

Usage:

VariableCollections.map.get().get("testVariable");

However, you’d have to ensure that the test cases that were sharing the collection were running on the same thread, which is not really possible to do…

[kazurayam]

@gdearest07

would it work if the test suite is being run in parallel

2 Test Suites running in parallel run in 2 separated OS processes (independent Java VMs). See the following post for detail:

Parallel execution is inconsistent &/or not reliable

@ThanhTo No. This is not correct. I believe that an OS-level Process (=Java VM instance) is forked per Test Suite in a Test Suite Collection. I have got an evidence. I have made a project published on GitHub: In this project, you will find a Test Suite Collection named “TSC_usingTimestampedLogger” which binds 4 Test Suites and execute them in parallel mode. I executed the TSC and monitored Activity Monitor on Mac to see how OS-level processes behave. When I executed the TSC, I could see 4…

Therefore thread-safety of collections would not matter much when you run 2 Test Suites.

Thread-safety matters only when you create threads in Test Cases yourself. I do not think there are many people who does multi-threading in Katalon Studio, but there are some posts regarding multi-threading in Test Cases.

• How to perform mobile actions on an app within a separate thread - #6 by kazurayam
• TestClosure --- executing Groovy Closures in multi-Threads in a Test Case simultaneously

[tomislav.duic]

Hi Brandon.
I’ve just found this while searching exactly for the same problem. I don’t want to use global variables for a specific test suite, I just want test suite scoped variables to use between test cases of that suite.
This is exactly what I want.
But what I’m struggling with at the moment is how to implement this. In which type of file should I put this code (file extension?), and where in folder hierarchy under the project folder, to be able to use it in my test cases?

[kazurayam]

@tomislav.duic

how to implement this. In which type of file should I put this code (file extension?), and where in folder hierarchy under the project folder, to be able to use it in my test cases?

You have 2+α options.

1. Under the Keywords/ folder, you want to create a package, where you would want a Groovy class, like MyVariableCollections.groovy

2. Under the Include/scripts/groovy folder, you want to create a package, where you would want a Groovy class, like MyVariableCollections.groovy

or

Under the Includes/scripts/groovy folder, you want to create a packege, where you would want a Java class, like MyVariableCollections.java

[kazurayam]

I think that you do not need to create a custom class to carry hundreds of variables across Test Cases in a Test Suite. A GlobalVariable can be an instance of java.util.Map, which is enough for this purpose. See the following screenshot as an example.

1. I created GlobalVariable.keyValuePairs of type Map
2. I created Test Cases/TC1, that sets a value into the GlobalVariable with key “singer”.
3. I created Test Cases/TC2, that prints the value of GlobalVariable.keyValuePairs.get("singer")
4. I created Test Suites/TS1, that binds the TC1 and TC2
5. I executed the TS1. I saw the value was passed from TC1 to TC2.

as you see in this example, you can set as many key-value pairs as you want in a single GlobalVariable.

I would argue that the GlobalVariable.keyValuePairs of type Map is equivalent to the base.util.VariableCollections class which @Brandon_Hein proposed years ago.

In fact you can find the Groovy source code generated by Katalon Studio in <projectDir>/Libs/internal/GlobalVariable.groovy, which looks something like:

package internal

import com.kms.katalon.core.configuration.RunConfiguration
import com.kms.katalon.core.main.TestCaseMain


/**
 * This class is generated automatically by Katalon Studio and should not be modified or deleted.
 */
public class GlobalVariable {
     
    /**
     * <p></p>
     */
    public static Object keyValuePairs
     

    static {
        try {
            def selectedVariables = TestCaseMain.getGlobalVariables("default")
			selectedVariables += TestCaseMain.getGlobalVariables(RunConfiguration.getExecutionProfile())
            selectedVariables += TestCaseMain.getParsedValues(RunConfiguration.getOverridingParameters())
    
            keyValuePairs = selectedVariables['keyValuePairs']
            
        } catch (Exception e) {
            TestCaseMain.logGlobalVariableError(e)
        }
    }
}

If you study this code in detail, you would find that the GlobalVariable class is essentially quite similar to the base.util.VariableCollections of @Brandon_Hein.

[tomislav.duic]

Hi there, thanks for your feedback.
Yeah, I suspected the Keywords folder would be a likely candidate, so gave it a try.
But what I found is that Katalon cannot use these variables directly (or I don’t know how), so I had to write get/set methods, and in this way it works just fine. This is from Keywords/util/Variables.groovy file:

So if you have an idea how to use the variables directly without methods, that would be much appreciated. I never played with Include folder, but can give it a try if it will enable me to use the vars directly.

As for your other answer for using the Global variables…I have more than 30 different profiles, mainly for different URLs, logins, API certificates and so on, and going that route would be impractical because I would have to define that global map in all 30 profiles, I wanted to avoid that.
I never used default profile. If I can define it in default profile so others inherit this from it, then that’s another matter.

[kazurayam]

@tomislav.duic

Katalon cannot use these variables directly (or I don’t know how),

If you show your faulty code (both of Keyword and Test Case) that does not work, then I might be able to give you some advice.

I am pretty sure Katalon is not guilty.

Your code must have some mistakes.

[kazurayam]

@tomislav.duic

I would have to define that global map in all 30 profiles, I wanted to avoid that.

Fine. Up to you.

[tomislav.duic]

Ah, I’ve found a way to use them. So if you see the commented code at the bottom, that was my first try, to use conventional keyword method calling syntax, and that didn’t work.
So after writing it like it is now, it works in two different test cases.
Tnx

[kazurayam]

I tried to reproduce your problem.

A Test Case:

import util.Variables

CustomKeywords."util.Variables.str1" = "Hello"
println CustomKeywords."util.Variables.str1"

When I executed this, I got an error.

2022-12-21 17:23:00.122 ERROR c.k.katalon.core.main.TestCaseExecutor   - ❌ Test Cases/TC5 FAILED.
Reason:
groovy.lang.MissingPropertyException: No such property: util.Variables.str1 for class: CustomKeywords
	at TC5.run(TC5:7)
        ...

Obviously, you wrote a wrong code.