TestCaseStack --- a chain of test case IDs by WebUI.callTestCase()

I made a demo project on GitHub:


Problem to solve

In a TestListener with the follwing code,

class MyTestListener {
    @BeforeTestCase
    def beforeTestCase(TestCaseContext testCaseContext) {
        GlobalVariable.CURRENT_TESTCASE_ID = testCaseContext.getTestCaseId()
    }
}

GlobalVariable.CURRENT_TESTCASE_ID will have the name of testcase invoked by Katalon Studio.

That’s fine. But we want to know more. Here we assume we have a chain like

  1. Test Suite TS1 invokes a Test Case Root .
  2. Root calls another Test Case Level1 , which calls Level2 , which calls Level3 with the built-in keyword WebUI.callTestCase()

I ran the “Test Cases/Root” and looked at the log to find the following result.

Please find that the value of GlobalVariable.CURRENT_TESTCASE_ID stayed the same "Test Cases/Root" inside Level1, Level2 and Leve3. This experiment shows a fact that Katalon Studio does NOT invoke the @beforeTestCase-annotated method in a TestListener when the callTestCase() keyword is called by a parent TestCase. This behavior was a bit of my surprise. But this is given. OK, I would accept it.

However, in some situation due to some reason, I would like to know more:

  1. How the code of "Test Cases/Level1" can find the name of itself is "Test Cases/Level1" ?

  2. How the code of "Test Cases/Level2" can find the name of itself is "Test Cases/Level2" ?

  3. How the code of "Test Cases/Level3" can find the name of itself is "Test Cases/Level3" ?

  4. How the code of "Test Cases/Level1" can find the name of parent: Test Cases/Root ?

  5. How the code of "Test Cases/Level2" can find the name of parent: Test Cases/Level1 ?

  6. How the code of "Test Cases/Level3" can find the name of parent: Test Cases/Level2 ?

Solution

Katalon Studio does not provide any built-in solution for the above question.

If you really want the solution, you need to change the source of the built-in keyword WebUI.callTestCase() .

But how the change would be?

I can show you how to change the source of the built-in keyword WebUI.callTestCase() with a running example by changing the implementation runtime using Groovy’s metaprogramming.

Description

Running the demo

  1. Get the zip file of the demo project from the Releases page, unzip it, open it with your Katalon Studio.
  2. Open the test suite TS1 , and manually run it.
  3. TS1 invokes a parent test case Root .
  4. Root calls other test case Level1 , which calls Level2 , which calls Level3 . The builtin keyword WebUI.callTestCase() is used.
  5. In the Console tab, you can see the following output:
----------------------------------------------------
top-level TestCaseId : Test Cases/Root
----------------------------------------------------
top-level TestCaseId : Test Cases/Root
callee TestCaseId    : Test Cases/Level1
parent caller was    : Test Cases/Root
----------------------------------------------------
top-level TestCaseId : Test Cases/Root
callee TestCaseId    : Test Cases/Level2
parent caller was    : Test Cases/Level1
----------------------------------------------------
top-level TestCaseId : Test Cases/Root
callee TestCaseId    : Test Cases/Level3
parent caller was    : Test Cases/Level2

Notable points

In the Console message, you can find that:

  1. GlobalVariable.CURRENT_TESTCASE_ID stays unchanged during the course of testcase chain. This proves that the TestListener#beforeTestCase() is NOT invoked for the test cases Level1 , Level2 and Level3 which are invoked by WebUI.callTestCase() by the caller test cases.
  2. By my trick, you can see the callee TestCaseIds are printed.

My trick

My trick is here in the source code of [ Test Listeners/MyTestListener ](Test Listeners/MyTestListener.groovy).

In the method annotated with @BeforeTestCase , I do

@BeforeTestCase
def beforeTestCase(TestCaseContext testCaseContext) {
    // save the name of top-level test case
    GlobalVariable.CURRENT_TESTCASE_ID = testCaseContext.getTestCaseId()

    // initialize TESTCASE_STACK with java.util.Stack object
    if (GlobalVariable.TESTCASES_STACK == null) {
        GlobalVariable.TESTCASES_STACK = new Stack<String>()
    }

    // modify WebUI.callTestCase() implementation
    // see for the original https://github.com/katalon-studio/katalon-studio-testing-framework/blob/master/Include/scripts/groovy/com/kms/katalon/core/keyword/BuiltinKeywords.groovy
    BuiltinKeywords.metaClass.'static'.callTestCase = { TestCase calledTestCase, Map binding, FailureHandling flowControl ->
        ((Stack)GlobalVariable.TESTCASES_STACK).push(calledTestCase.getTestCaseId())
        Object result = (Object)KeywordExecutor.executeKeywordForPlatform(KeywordExecutor.PLATFORM_BUILT_IN, "callTestCase", calledTestCase, binding, flowControl)
        ((Stack)GlobalVariable.TESTCASES_STACK).pop()
        return result
    }
    BuiltinKeywords.metaClass.'static'.callTestCase = { TestCase calledTestCase, Map binding ->
        ((Stack)GlobalVariable.TESTCASES_STACK).push(calledTestCase.getTestCaseId())
        Object result = (Object)KeywordExecutor.executeKeywordForPlatform(KeywordExecutor.PLATFORM_BUILT_IN, "callTestCase", calledTestCase, binding)
        ((Stack)GlobalVariable.TESTCASES_STACK).pop()
        return result
    }
}

Here I used Groovy’s metaprogromming feature. See ExpandoMetaClass for technical detail. I would not talk about it here.

Please note that the GlobalVariable.TESTCASE_STACK must be declared with type Null in order to initialize it with a Stack object.

6 Likes

Beyond cool, Kaz.

Sourcing relevant and helpful information (beyond dry and typically terse official documentation) for Groovy’s metaprogramming toolset is not easy. You just did that in a single post.

Excellent. Thank you.

Do not even know why this is not out-of-the box. Really useful when using the test case Id as the column ID for No-SQL databases.

@ThanhTo
@devalex88

How about promoting this idea as a built-in feature?