There appears to be no capability to support organizing Test Listeners in sub-folders. Forcing by creating sub-folders outside of the IDE causes the folder and listener contained within it to disappear from Katalon Studio.
Additionally, new Errors appear in the Problems tab: The declared package "" does not match the expected package "TestSuiteStatus" TestSuiteStatus.groovy
It appears there is some implicit package declaration that is not explicitly defined in the listener script. Supplying a package declaration that matches the error message does resolve the error and allows complication and execution. However, some functionality is no longer working.
Specifically, it seems that the @BeforeTestCase annotation fires, but the @AfterTestCase annotation does not fire. A more complicated listener had a lot more errors to debug including not recognizing other imported packages in the include path, but this simple example illustrates a clearer problem.
Steps:
- Create a New Test Listener: right-click “Test Listeners” folder > Click New → New Test Listener
- Create a sub-folder within the Test Listeners folder
- Move New Listener into sub-folder
- sub-folder and its contents are not visible from Katalon Studio IDE
- Add
package TestSuiteStatusto listener file in sub-folder to resolve compilation error. - Execute test.
Desired result:
- Able to create sub-folders to organize listeners and view them from the IDE.
Actual Result:
- Compilation errors.
- Vanishing folders and files in the IDE.
- Fixing errors with package declaration causes some annotations to be ignored.
Listener TestSuiteStatus.groovy:
package TestSuiteStatus
import com.kms.katalon.core.annotation.AfterTestSuite
import com.kms.katalon.core.annotation.BeforeTestCase
import com.kms.katalon.core.annotation.BeforeTestSuite
import com.kms.katalon.core.context.TestCaseContext
import com.kms.katalon.core.context.TestSuiteContext
import com.kms.katalon.core.util.KeywordUtil
import com.qa.katalon.GlobalVariableHelpers
import internal.GlobalVariable
class TestSuiteStatus {
GlobalVariableHelpers gvars = new GlobalVariableHelpers()
TestSuiteStatus() {
}
/**
* Executes before every test suite starts.
* @param testSuiteContext: related information of the executed test suite.
*/
@BeforeTestSuite
def sampleBeforeTestSuite(TestSuiteContext testSuiteContext) {
if ( ! GlobalVariable.exists('testCasesRun') ) {
GlobalVariable.set('testCasesRun', 0)
} else {
GlobalVariable.testCasesRun = 0
}
println "test1";
}
/**
* Executes after every test suite ends.
* @param testSuiteContext: related information of the executed test suite.
*/
@AfterTestSuite
def verifyTestCaseExecuted(TestSuiteContext testSuiteContext) {
if( GlobalVariable.testCasesRun <= 0 ) {
KeywordUtil.markErrorAndStop("No tests cases ran during execution.");
} else {
KeywordUtil.logInfo("Number of tests cases run: ${GlobalVariable.testCasesRun}")
}
println "test3";
}
/**
* If we get into test
* @param context: TestCaseContext
* @return void
*/
@BeforeTestCase
def incrementTestCaseRunCount(TestCaseContext context) {
GlobalVariable.testCasesRun += 1
println "test2";
}
}
Execution log:
2022-06-30 09:08:04.716 INFO c.k.katalon.core.main.TestCaseExecutor - --------------------
2022-06-30 09:08:04.720 INFO c.k.katalon.core.main.TestCaseExecutor - START Test Cases/Unit Tests/Passing
2022-06-30 09:08:05.011 DEBUG c.k.katalon.core.main.TestCaseExecutor - Evaluating variables for test case
2022-06-30 09:08:05.428 DEBUG c.k.k.core.context.internal.TestHooker - STEP incrementTestCaseRunCount
2022-06-30 09:08:05.429 DEBUG c.k.k.core.context.internal.TestHooker - Starting invoke 'com.kms.katalon.core.annotation.BeforeTestCase' method: 'TestSuiteStatus.TestSuiteStatus.incrementTestCaseRunCount(...)'
test2
2022-06-30 09:08:05.456 DEBUG c.k.k.core.context.internal.TestHooker - Invoke 'com.kms.katalon.core.annotation.BeforeTestCase' method: 'TestSuiteStatus.TestSuiteStatus.incrementTestCaseRunCount(...)' completed.
2022-06-30 09:08:05.457 TRACE c.k.k.core.context.internal.TestHooker - END null: println("test2")
2022-06-30 09:08:05.457 TRACE c.k.k.core.context.internal.TestHooker - END null: incrementTestCaseRunCount
2022-06-30 09:08:05.521 DEBUG testcase.Passing - 1: logger.logInfoHighlight("Very important message.")
2022-06-30 09:08:05.537 INFO com.kms.katalon.core.util.KeywordUtil -
2022-06-30 09:08:05.538 INFO com.kms.katalon.core.util.KeywordUtil - ╭─────────────────────────────╮
2022-06-30 09:08:05.539 INFO com.kms.katalon.core.util.KeywordUtil - │ │
2022-06-30 09:08:05.540 INFO com.kms.katalon.core.util.KeywordUtil - │ Very important message. │
2022-06-30 09:08:05.540 INFO com.kms.katalon.core.util.KeywordUtil - │ │
2022-06-30 09:08:05.541 INFO com.kms.katalon.core.util.KeywordUtil - ╰─────────────────────────────╯
2022-06-30 09:08:05.542 INFO com.kms.katalon.core.util.KeywordUtil -
2022-06-30 09:08:05.543 DEBUG testcase.Passing - 2: logInfo("Just going to pass this test case.")
2022-06-30 09:08:05.545 INFO com.kms.katalon.core.util.KeywordUtil - Just going to pass this test case.
2022-06-30 09:08:05.546 DEBUG testcase.Passing - 3: markPassed("Marking that this is passed.")
2022-06-30 09:08:05.548 DEBUG com.kms.katalon.core.util.KeywordUtil - ✓ Marking that this is passed.
2022-06-30 09:08:05.567 DEBUG testcase.Passing - 4: logInfo("The test continues to execute.")
2022-06-30 09:08:05.569 INFO com.kms.katalon.core.util.KeywordUtil - The test continues to execute.
2022-06-30 09:08:05.569 TRACE c.k.katalon.core.main.TestCaseExecutor - END null: logInfo("The test continues to execute.")
2022-06-30 09:08:05.570 DEBUG c.k.katalon.core.main.TestCaseExecutor - ✓ Test Cases/Unit Tests/Passing
2022-06-30 09:08:05.582 INFO c.k.katalon.core.main.TestCaseExecutor - END Test Cases/Unit Tests/Passing