Help me with a Custom listener (has to run only when all test cases are passed)

Product Forums Katalon Studio
,
1

Hi guys, I wonder if I can execute a custom keyword only when all the test cases have passed after running the Suit. I tried to use the following listener to get the content of the suit. But my problem was that “COMPLETE” state is when all cases have been executed (even if they were passed or failed)

	@AfterTestSuite
	def AfterTestSuite(TestSuiteContext testSuiteContext) {
		if(testSuiteContext.getStatus()=='COMPLETE'){
			CustomKeywords.'lemonbrew.GlobalVariableUpdater.increaseEmailVariable'("Agent", "RealtorEmail")
		}
	}

So I tried to perform the above Listener receiving the context of test cases but I’m not sure If this is the correct way to do it

@AfterTestSuite
	def IncreaseGlobalVariableEmail(TestCaseContext testCaseContext) {
		String suitStatus
		List testcases= testCaseContext.testCaseStatus
		for(TestCaseContext testcase in testcases) {
			if(testcase =="FAILED")
				suitStatus= "FAILED"
		}
		if (suitStatus!="FAILED"){
			CustomKeywords.'lemonbrew.GlobalVariableUpdater.increaseEmailVariable'("Agent", "RealtorEmail")
		}
	}

Results of a suit with only 1 test failed:

Log viewer

image
image1411×939 61.6 KB

Console

2019-11-28 13:29:37.220 INFO c.k.katalon.core.main.TestSuiteExecutor - START Test Suites/SUIT Create an Agent
2019-11-28 13:29:37.264 INFO c.k.katalon.core.main.TestSuiteExecutor - hostName = rodri - DESKTOP-GVDDC6C
2019-11-28 13:29:37.266 INFO c.k.katalon.core.main.TestSuiteExecutor - os = Windows 10 64bit
2019-11-28 13:29:37.266 INFO c.k.katalon.core.main.TestSuiteExecutor - hostAddress = 192.168.1.13
2019-11-28 13:29:37.268 INFO c.k.katalon.core.main.TestSuiteExecutor - katalonVersion = 6.3.3.11
2019-11-28 13:29:37.293 INFO com.kms.katalon.core.util.KeywordUtil - This Katalon Studio version has been deprecated. Please upgrade Katalon Studio to the latest version.
2019-11-28 13:29:37.513 INFO c.k.katalon.core.main.TestCaseExecutor - --------------------
2019-11-28 13:29:37.513 INFO c.k.katalon.core.main.TestCaseExecutor - START Test Cases/Create an Agent flow/3. Sign in
2019-11-28 13:29:37.796 DEBUG testcase.3. Sign in - 1: openBrowser(“”)
2019-11-28 13:29:38.012 INFO c.k.k.core.webui.driver.DriverFactory - Starting ‘Chrome’ driver
nov 28, 2019 1:29:38 PM org.openqa.selenium.remote.DesiredCapabilities chrome
INFORMACIÓN: Using new ChromeOptions() is preferred to DesiredCapabilities.chrome()
2019-11-28 13:29:38.065 INFO c.k.k.core.webui.driver.DriverFactory - Action delay is set to 0 seconds
Starting ChromeDriver 78.0.3904.70 (edb9c9f3de0247fd912a77b7f6cae7447f6d3ad5-refs/branch-heads/3904@{#800}) on port 38043
Only local connections are allowed.
Please protect ports used by ChromeDriver and related test frameworks to prevent access by malicious code.
nov 28, 2019 1:29:41 PM org.openqa.selenium.remote.ProtocolHandshake createSession
INFORMACIÓN: Detected dialect: W3C
2019-11-28 13:29:41.174 INFO c.k.k.core.webui.driver.DriverFactory - sessionId = bacd561267f03f7e5de56d5224781a24
2019-11-28 13:29:41.190 INFO c.k.k.core.webui.driver.DriverFactory - browser = Chrome 78.0.3904.108
2019-11-28 13:29:41.191 INFO c.k.k.core.webui.driver.DriverFactory - platform = Windows 10
2019-11-28 13:29:41.191 INFO c.k.k.core.webui.driver.DriverFactory - seleniumVersion = 3.141.59
2019-11-28 13:29:41.192 INFO c.k.k.core.webui.driver.DriverFactory - proxyInformation = ProxyInformation{proxyOption=NO_PROXY, proxyServerType=HTTP, password=, proxyServerAddress=, proxyServerPort=0}
2019-11-28 13:29:41.193 DEBUG testcase.3. Sign in - 2: navigateToUrl(“https://dev.lemonbrew.com/”)
2019-11-28 13:29:44.959 DEBUG testcase.3. Sign in - 3: setText(findTestObject(“Page_LemonBrew HomePage/input_Username”), RealtorEmail)
2019-11-28 13:29:46.570 DEBUG testcase.3. Sign in - 4: setText(findTestObject(“Page_LemonBrew HomePage/input_password”), Password)
2019-11-28 13:29:46.785 DEBUG testcase.3. Sign in - 5: click(findTestObject(“Page_LemonBrew HomePage/button_LOG IN”))
2019-11-28 13:29:46.971 DEBUG testcase.3. Sign in - 6: waitForPageLoad(5)
2019-11-28 13:29:47.080 DEBUG testcase.3. Sign in - 7: verifyElementPresent(findTestObject(“Object Repository/Page_DocuSign/div_Profile realtor”), 4, STOP_ON_FAILURE)
2019-11-28 13:29:51.252 INFO c.k.k.c.webui.common.WebUiCommonHelper - Unable to find the element located by ‘By.cssSelector: div [class=“container realtor-profile-tabbed”]’. Please recheck the objects properties to make sure the desired element is located.
2019-11-28 13:29:51.589 ERROR c.k.k.core.keyword.internal.KeywordMain - :x: com.kms.katalon.core.webui.exception.WebElementNotFoundException: Web element with id: ‘Object Repository/Page_DocuSign/div_Profile realtor’ located by ‘By.cssSelector: div [class=“container realtor-profile-tabbed”]’ not found (Root cause: com.kms.katalon.core.exception.StepFailedException: com.kms.katalon.core.webui.exception.WebElementNotFoundException: Web element with id: ‘Object Repository/Page_DocuSign/div_Profile realtor’ located by ‘By.cssSelector: div [class=“container realtor-profile-tabbed”]’ not found
at com.kms.katalon.core.webui.keyword.internal.WebUIKeywordMain.stepFailed(WebUIKeywordMain.groovy:64)
at com.kms.katalon.core.webui.keyword.builtin.VerifyElementPresentKeyword$_verifyElementPresent_closure1.doCall(VerifyElementPresentKeyword.groovy:85)
at com.kms.katalon.core.webui.keyword.builtin.VerifyElementPresentKeyword$_verifyElementPresent_closure1.call(VerifyElementPresentKeyword.groovy)
at com.kms.katalon.core.webui.keyword.internal.WebUIKeywordMain.runKeyword(WebUIKeywordMain.groovy:20)
at com.kms.katalon.core.webui.keyword.builtin.VerifyElementPresentKeyword.verifyElementPresent(VerifyElementPresentKeyword.groovy:92)
at com.kms.katalon.core.webui.keyword.builtin.VerifyElementPresentKeyword.execute(VerifyElementPresentKeyword.groovy:68)
at com.kms.katalon.core.keyword.internal.KeywordExecutor.executeKeywordForPlatform(KeywordExecutor.groovy:56)
at com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords.verifyElementPresent(WebUiBuiltInKeywords.groovy:1418)
at com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords$verifyElementPresent$4.call(Unknown Source)
at 3. Sign in.run(3. Sign in:31)
at com.kms.katalon.core.main.ScriptEngine.run(ScriptEngine.java:194)
at com.kms.katalon.core.main.ScriptEngine.runScriptAsRawText(ScriptEngine.java:119)
at com.kms.katalon.core.main.TestCaseExecutor.runScript(TestCaseExecutor.java:337)
at com.kms.katalon.core.main.TestCaseExecutor.doExecute(TestCaseExecutor.java:328)
at com.kms.katalon.core.main.TestCaseExecutor.processExecutionPhase(TestCaseExecutor.java:307)
at com.kms.katalon.core.main.TestCaseExecutor.accessMainPhase(TestCaseExecutor.java:299)
at com.kms.katalon.core.main.TestCaseExecutor.execute(TestCaseExecutor.java:233)
at com.kms.katalon.core.main.TestSuiteExecutor.accessTestCaseMainPhase(TestSuiteExecutor.java:169)
at com.kms.katalon.core.main.TestSuiteExecutor.accessTestSuiteMainPhase(TestSuiteExecutor.java:142)
at com.kms.katalon.core.main.TestSuiteExecutor.execute(TestSuiteExecutor.java:91)
at com.kms.katalon.core.main.TestCaseMain.startTestSuite(TestCaseMain.java:157)
at com.kms.katalon.core.main.TestCaseMain$startTestSuite$0.call(Unknown Source)
at TempTestSuite1574958571198.run(TempTestSuite1574958571198.groovy:35)
)
2019-11-28 13:29:51.894 ERROR c.k.k.core.keyword.internal.KeywordMain - :x: Unable to verify object ‘Object Repository/Page_DocuSign/div_Profile realtor’ is present (Root cause: com.kms.katalon.core.exception.StepFailedException: Unable to verify object ‘Object Repository/Page_DocuSign/div_Profile realtor’ is present
at com.kms.katalon.core.webui.keyword.internal.WebUIKeywordMain.stepFailed(WebUIKeywordMain.groovy:64)
at com.kms.katalon.core.webui.keyword.internal.WebUIKeywordMain.runKeyword(WebUIKeywordMain.groovy:26)
at com.kms.katalon.core.webui.keyword.builtin.VerifyElementPresentKeyword.verifyElementPresent(VerifyElementPresentKeyword.groovy:92)
at com.kms.katalon.core.webui.keyword.builtin.VerifyElementPresentKeyword.execute(VerifyElementPresentKeyword.groovy:68)
at com.kms.katalon.core.keyword.internal.KeywordExecutor.executeKeywordForPlatform(KeywordExecutor.groovy:56)
at com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords.verifyElementPresent(WebUiBuiltInKeywords.groovy:1418)
at com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords$verifyElementPresent$4.call(Unknown Source)
at 3. Sign in.run(3. Sign in:31)
at com.kms.katalon.core.main.ScriptEngine.run(ScriptEngine.java:194)
at com.kms.katalon.core.main.ScriptEngine.runScriptAsRawText(ScriptEngine.java:119)
at com.kms.katalon.core.main.TestCaseExecutor.runScript(TestCaseExecutor.java:337)
at com.kms.katalon.core.main.TestCaseExecutor.doExecute(TestCaseExecutor.java:328)
at com.kms.katalon.core.main.TestCaseExecutor.processExecutionPhase(TestCaseExecutor.java:307)
at com.kms.katalon.core.main.TestCaseExecutor.accessMainPhase(TestCaseExecutor.java:299)
at com.kms.katalon.core.main.TestCaseExecutor.execute(TestCaseExecutor.java:233)
at com.kms.katalon.core.main.TestSuiteExecutor.accessTestCaseMainPhase(TestSuiteExecutor.java:169)
at com.kms.katalon.core.main.TestSuiteExecutor.accessTestSuiteMainPhase(TestSuiteExecutor.java:142)
at com.kms.katalon.core.main.TestSuiteExecutor.execute(TestSuiteExecutor.java:91)
at com.kms.katalon.core.main.TestCaseMain.startTestSuite(TestCaseMain.java:157)
at com.kms.katalon.core.main.TestCaseMain$startTestSuite$0.call(Unknown Source)
at TempTestSuite1574958571198.run(TempTestSuite1574958571198.groovy:35)
Caused by: com.kms.katalon.core.exception.StepFailedException: com.kms.katalon.core.webui.exception.WebElementNotFoundException: Web element with id: ‘Object Repository/Page_DocuSign/div_Profile realtor’ located by ‘By.cssSelector: div [class=“container realtor-profile-tabbed”]’ not found
at com.kms.katalon.core.webui.keyword.internal.WebUIKeywordMain.stepFailed(WebUIKeywordMain.groovy:64)
at com.kms.katalon.core.webui.keyword.builtin.VerifyElementPresentKeyword$_verifyElementPresent_closure1.doCall(VerifyElementPresentKeyword.groovy:85)
at com.kms.katalon.core.webui.keyword.builtin.VerifyElementPresentKeyword$_verifyElementPresent_closure1.call(VerifyElementPresentKeyword.groovy)
at com.kms.katalon.core.webui.keyword.internal.WebUIKeywordMain.runKeyword(WebUIKeywordMain.groovy:20)
… 19 more
)
2019-11-28 13:29:51.896 ERROR c.k.katalon.core.main.TestCaseExecutor - :x: Test Cases/Create an Agent flow/3. Sign in FAILED.
Reason:
com.kms.katalon.core.exception.StepFailedException: Unable to verify object ‘Object Repository/Page_DocuSign/div_Profile realtor’ is present
at com.kms.katalon.core.webui.keyword.internal.WebUIKeywordMain.stepFailed(WebUIKeywordMain.groovy:64)
at com.kms.katalon.core.webui.keyword.internal.WebUIKeywordMain.runKeyword(WebUIKeywordMain.groovy:26)
at com.kms.katalon.core.webui.keyword.builtin.VerifyElementPresentKeyword.verifyElementPresent(VerifyElementPresentKeyword.groovy:92)
at com.kms.katalon.core.webui.keyword.builtin.VerifyElementPresentKeyword.execute(VerifyElementPresentKeyword.groovy:68)
at com.kms.katalon.core.keyword.internal.KeywordExecutor.executeKeywordForPlatform(KeywordExecutor.groovy:56)
at com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords.verifyElementPresent(WebUiBuiltInKeywords.groovy:1418)
at com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords$verifyElementPresent$4.call(Unknown Source)
at 3. Sign in.run(3. Sign in:31)
at com.kms.katalon.core.main.ScriptEngine.run(ScriptEngine.java:194)
at com.kms.katalon.core.main.ScriptEngine.runScriptAsRawText(ScriptEngine.java:119)
at com.kms.katalon.core.main.TestCaseExecutor.runScript(TestCaseExecutor.java:337)
at com.kms.katalon.core.main.TestCaseExecutor.doExecute(TestCaseExecutor.java:328)
at com.kms.katalon.core.main.TestCaseExecutor.processExecutionPhase(TestCaseExecutor.java:307)
at com.kms.katalon.core.main.TestCaseExecutor.accessMainPhase(TestCaseExecutor.java:299)
at com.kms.katalon.core.main.TestCaseExecutor.execute(TestCaseExecutor.java:233)
at com.kms.katalon.core.main.TestSuiteExecutor.accessTestCaseMainPhase(TestSuiteExecutor.java:169)
at com.kms.katalon.core.main.TestSuiteExecutor.accessTestSuiteMainPhase(TestSuiteExecutor.java:142)
at com.kms.katalon.core.main.TestSuiteExecutor.execute(TestSuiteExecutor.java:91)
at com.kms.katalon.core.main.TestCaseMain.startTestSuite(TestCaseMain.java:157)
at com.kms.katalon.core.main.TestCaseMain$startTestSuite$0.call(Unknown Source)
at TempTestSuite1574958571198.run(TempTestSuite1574958571198.groovy:35)
Caused by: com.kms.katalon.core.exception.StepFailedException: com.kms.katalon.core.webui.exception.WebElementNotFoundException: Web element with id: ‘Object Repository/Page_DocuSign/div_Profile realtor’ located by ‘By.cssSelector: div [class=“container realtor-profile-tabbed”]’ not found
at com.kms.katalon.core.webui.keyword.internal.WebUIKeywordMain.stepFailed(WebUIKeywordMain.groovy:64)
at com.kms.katalon.core.webui.keyword.builtin.VerifyElementPresentKeyword$_verifyElementPresent_closure1.doCall(VerifyElementPresentKeyword.groovy:85)
at com.kms.katalon.core.webui.keyword.builtin.VerifyElementPresentKeyword$_verifyElementPresent_closure1.call(VerifyElementPresentKeyword.groovy)
at com.kms.katalon.core.webui.keyword.internal.WebUIKeywordMain.runKeyword(WebUIKeywordMain.groovy:20)
… 19 more

2019-11-28 13:29:51.899 INFO c.k.katalon.core.main.TestCaseExecutor - END Test Cases/Create an Agent flow/3. Sign in
2019-11-28 13:29:51.906 ERROR c.k.k.core.context.internal.TestHooker - :x: java.lang.NullPointerException: Cannot get property ‘testCaseStatus’ on null object
2019-11-28 13:29:51.907 INFO com.kms.katalon.core.util.KeywordUtil - This Katalon Studio version has been deprecated. Please upgrade Katalon Studio to the latest version.
2019-11-28 13:29:52.023 INFO com.kms.katalon.core.util.KeywordUtil - Start generating HTML report folder at: C:\Users\rodri\Documents\Lemonbrew-QA\Reports\20191128_132931\SUIT Create an Agent\20191128_132931…
2019-11-28 13:29:52.430 INFO com.kms.katalon.core.util.KeywordUtil - HTML report generated
2019-11-28 13:29:52.430 INFO com.kms.katalon.core.util.KeywordUtil - Start generating CSV report folder at: C:\Users\rodri\Documents\Lemonbrew-QA\Reports\20191128_132931\SUIT Create an Agent\20191128_132931…
2019-11-28 13:29:52.448 INFO com.kms.katalon.core.util.KeywordUtil - CSV report generated
2019-11-28 13:29:52.448 INFO com.kms.katalon.core.util.KeywordUtil - Start generating JUnit report folder at: C:\Users\rodri\Documents\Lemonbrew-QA\Reports\20191128_132931\SUIT Create an Agent\20191128_132931…
2019-11-28 13:29:52.556 INFO com.kms.katalon.core.util.KeywordUtil - JUnit report generated
2019-11-28 13:29:53.252 INFO c.k.katalon.core.main.TestSuiteExecutor - --------------------
2019-11-28 13:29:53.252 INFO c.k.katalon.core.main.TestSuiteExecutor - END Test Suites/SUIT Create an Agent
2019-11-28 13:29:53.252 INFO c.k.katalon.core.main.TestSuiteExecutor - ====================

Any ideas of how I can code my listener? I don’t have to many experiences with development so I’m not sure if that is the correct way to bring and store the status from all test cases inside my suit

2

@rodrigocalabretta

I think the idea is that just remember if any of the test fails into a global variable and read it in the test suite listener.

import com.kms.katalon.core.annotation.AfterTestCase
import com.kms.katalon.core.annotation.AfterTestSuite
import com.kms.katalon.core.context.TestCaseContext
import com.kms.katalon.core.context.TestSuiteContext

import internal.GlobalVariable as GlobalVariable

class NewTestListener {
	@AfterTestCase
	def sampleAfterTestCase(TestCaseContext testCaseContext) {
		println testCaseContext.getTestCaseStatus()
		if(testCaseContext.getTestCaseStatus() != 'PASSED') {
			GlobalVariable.allTcPassed = false;
		}
	}

	@AfterTestSuite
	def sampleAfterTestSuite(TestSuiteContext testSuiteContext) {
		if(GlobalVariable.allTcPassed) {
			println "all has passed"
		} else {
			println "a test has failed"
		}
	}
}

I tested on my end and it seems to work.

1 Like
3

Yes, I saw it in your previous comment and I don’t understand why I was complicating the logic of what it was.

I used that global variable within my listener class instead of using it as a global variable in my execution profile and it works. But that’s okay?

class LemonBrewTestListener {

	boolean suitPass=true;

	@AfterTestCase
	def AfterTestCase(TestCaseContext testCaseContext) {
		if(testCaseContext.getTestCaseStatus()=="FAILED"){
				suitPass=false;
		}
	}

	@AfterTestSuite
	def AfterTestSuite() {
		if (suitPass==true){
			CustomKeywords.'lemonbrew.GlobalVariableUpdater.increaseEmailVariable'("Agent", "RealtorEmail")
			println "Global Variable Email increased"
		}
		else{
		        println "Test Suit Fails"
		} 
}

Thanks for all the help mate in these last days, I owe you a beer :laughing: :raised_hands:

1 Like
4

If it runs as expected then it’s fine. Personally I’d rather use a global variable, but hey ! If it works, it works.

1 Like
5

Has this problem been resolved ? If it has I will mark it as ‘solved’ so that our children (and other people) can take advantage of the accepted solution.

6

Yeap @ThanhTo I would like to add an improvement to the previous solution because if I use the AfterTestSuite listener I would execute that listener on every suit that I run (My keyword updates my execution profile, so I don’t want to execute that on each suit). That’s why I decided to move that listener inside my selected suit and set that custom keyword call inside his TeardDown() function

The code is the following:

Inside the test suit 
@TearDown(skipped = false) // Please change skipped to be false to activate this method.
def tearDown() {
	//Put your code here.
	if (GlobalVariable.EmailIncrease==true){
		CustomKeywords.'lemonbrew.GlobalVariableUpdater.increaseEmailVariable'("Agent", "RealtorEmail")
		println "Global Variable Email increased"
	}
	else{
		println "Test Suit Fails"
	}
}
Inside the listener 
	@AfterTestCase
	def AfterTestCase(TestCaseContext testCaseContext) {
		if(testCaseContext.getTestCaseStatus()=="FAILED"){
			GlobalVariable.EmailIncrease=false;
		}
	}

I would prefer to move my listener inside my suit too to control everything from there (not executing the following listener on every test case I run) but i coulnd’t be able to do it because as you say it in another topic:

“Object testCaseContext is available only in Test Listeners”

That’s the reason I couldn’t be able to use the solution in the teardDownTestCase() annotation

So this code solves my problem :white_check_mark:

PD: By the way, for this to work, I had to create a global variable as you recommend, it also works to enable or disable the whole method of my keyword increasing or not its value (two birds in a shot :slight_smile:)

1 Like