The function "testCaseContext.message" only returns one error for a test case

osamazaza200 · March 6, 2024, 7:00am

Hello Dears,
Have a good day.

I have discovered an issue in katalon, in which test listener @AfterTestCase parameter (testCaseContext) only returns a single error when using the function testCaseContext.message.
When I print a message only one exception error is displayed in the console:

Message:::Test Cases/x FAILED.
Reason:
com.kms.katalon.core.exception.StepFailedException: Object Object Repository/HomePage/NavigationBar/moreButton not found
	at com.kms.katalon.core.keyword.internal.KeywordMain.stepFailed(KeywordMain.groovy:39)
	at com.kms.katalon.core.mobile.keyword.internal.MobileKeywordMain.stepFailed(MobileKeywordMain.groovy:40)
	at com.kms.katalon.core.mobile.keyword.builtin.TapKeyword$_tap_closure1.doCall(TapKeyword.groovy:79)
	at com.kms.katalon.core.mobile.keyword.builtin.TapKeyword$_tap_closure1.call(TapKeyword.groovy)
	at com.kms.katalon.core.mobile.keyword.internal.MobileKeywordMain.runKeyword(MobileKeywordMain.groovy:21)
	at com.kms.katalon.core.mobile.keyword.builtin.TapKeyword.tap(TapKeyword.groovy:84)
	at com.kms.katalon.core.mobile.keyword.builtin.TapKeyword.execute(TapKeyword.groovy:68)
	at com.kms.katalon.core.keyword.internal.KeywordExecutor.executeKeywordForPlatform(KeywordExecutor.groovy:74)
	at com.kms.katalon.core.mobile.keyword.MobileBuiltInKeywords.tap(MobileBuiltInKeywords.groovy:1476)
	at com.kms.katalon.core.mobile.keyword.MobileBuiltInKeywords$tap$0.call(Unknown Source)
	at x.run(x:40)
	at com.kms.katalon.core.main.ScriptEngine.run(ScriptEngine.java:194)
	at com.kms.katalon.core.main.ScriptEngine.runScriptAsRawText(ScriptEngine.java:119)
	at com.kms.katalon.core.main.TestCaseExecutor.runScript(TestCaseExecutor.java:448)
	at com.kms.katalon.core.main.TestCaseExecutor.doExecute(TestCaseExecutor.java:439)
	at com.kms.katalon.core.main.TestCaseExecutor.processExecutionPhase(TestCaseExecutor.java:418)
	at com.kms.katalon.core.main.TestCaseExecutor.accessMainPhase(TestCaseExecutor.java:410)
	at com.kms.katalon.core.main.TestCaseExecutor.execute(TestCaseExecutor.java:285)
	at com.kms.katalon.core.common.CommonExecutor.accessTestCaseMainPhase(CommonExecutor.java:65)
	at com.kms.katalon.core.main.TestSuiteExecutor.accessTestSuiteMainPhase(TestSuiteExecutor.java:148)
	at com.kms.katalon.core.main.TestSuiteExecutor.execute(TestSuiteExecutor.java:106)
	at com.kms.katalon.core.main.TestCaseMain.startTestSuite(TestCaseMain.java:187)
	at com.kms.katalon.core.main.TestCaseMain$startTestSuite$0.call(Unknown Source)
	at TempTestSuite1709660672431.run(TempTestSuite1709660672431.groovy:36)
 end of error message

Where the console shows all errors:

2024-03-05 20:45:39.815 ERROR c.k.k.core.keyword.internal.KeywordMain  - ❌ Object Object Repository/WMV/MoreOptionPage/settingsButton not found
com.kms.katalon.core.exception.StepFailedException: Object Object Repository/HomePage/NavigationBar/moreButton not found
	at com.kms.katalon.core.keyword.internal.KeywordMain.stepFailed(KeywordMain.groovy:39)
	at com.kms.katalon.core.mobile.keyword.internal.MobileKeywordMain.stepFailed(MobileKeywordMain.groovy:40)
	at com.kms.katalon.core.mobile.keyword.builtin.TapKeyword$_tap_closure1.doCall(TapKeyword.groovy:79)
	at com.kms.katalon.core.mobile.keyword.builtin.TapKeyword$_tap_closure1.call(TapKeyword.groovy)
	at com.kms.katalon.core.mobile.keyword.internal.MobileKeywordMain.runKeyword(MobileKeywordMain.groovy:21)
	at com.kms.katalon.core.mobile.keyword.builtin.TapKeyword.tap(TapKeyword.groovy:84)
	at com.kms.katalon.core.mobile.keyword.builtin.TapKeyword.execute(TapKeyword.groovy:68)
	at com.kms.katalon.core.keyword.internal.KeywordExecutor.executeKeywordForPlatform(KeywordExecutor.groovy:74)
	at com.kms.katalon.core.mobile.keyword.MobileBuiltInKeywords.tap(MobileBuiltInKeywords.groovy:1476)
	at com.kms.katalon.core.mobile.keyword.MobileBuiltInKeywords$tap$0.call(Unknown Source)
	at x.run(x:40)
	at com.kms.katalon.core.main.ScriptEngine.run(ScriptEngine.java:194)
	at com.kms.katalon.core.main.ScriptEngine.runScriptAsRawText(ScriptEngine.java:119)
	at com.kms.katalon.core.main.TestCaseExecutor.runScript(TestCaseExecutor.java:448)
	at com.kms.katalon.core.main.TestCaseExecutor.doExecute(TestCaseExecutor.java:439)
	at com.kms.katalon.core.main.TestCaseExecutor.processExecutionPhase(TestCaseExecutor.java:418)
	at com.kms.katalon.core.main.TestCaseExecutor.accessMainPhase(TestCaseExecutor.java:410)
	at com.kms.katalon.core.main.TestCaseExecutor.execute(TestCaseExecutor.java:285)
	at com.kms.katalon.core.common.CommonExecutor.accessTestCaseMainPhase(CommonExecutor.java:65)
	at com.kms.katalon.core.main.TestSuiteExecutor.accessTestSuiteMainPhase(TestSuiteExecutor.java:148)
	at com.kms.katalon.core.main.TestSuiteExecutor.execute(TestSuiteExecutor.java:106)
	at com.kms.katalon.core.main.TestCaseMain.startTestSuite(TestCaseMain.java:187)
	at com.kms.katalon.core.main.TestCaseMain$startTestSuite$0.call(Unknown Source)
	at TempTestSuite1709660672431.run(TempTestSuite1709660672431.groovy:36)

Test Case code
Mobile.startExistingApplication('com.xr.consumer.sit')
String moreButtonSource = "Object Repository/HomePage/NavigationBar/moreButton";
TestObject moreButton = findTestObject(moreButtonSource)
Mobile.tap(moreButton, 5, FailureHandling.CONTINUE_ON_FAILURE)

String settingOptionSource = "Object Repository/WMV/MoreOptionPage/settingsButton";
TestObject settingOption = findTestObject(settingOptionSource)
Mobile.tap(settingOption, 5, FailureHandling.CONTINUE_ON_FAILURE)

Tes Listener AfterTestCase code
@AfterTestCase

def sampleAfterTestCase(TestCaseContext testCaseContext) {
		

    String msg = testCaseContext.getMessage()
    System.out.println("Message:::" + msg + " end of error message");

	 }

kazurayam · March 6, 2024, 9:32am

??? I can find only one error in the text you attached as “the console”. Your explanation does not make sense to me.

osamazaza200 · March 11, 2024, 9:28am

Sorry kazurayam, i didn’t add it by mistake.
These are the exceptions that can be displayed to the console when not able to click on item since it’s not existed:

com.kms.katalon.core.exception.StepFailedException: Object Object Repository/HomePage/NavigationBar/moreButton not found
	at com.kms.katalon.core.keyword.internal.KeywordMain.stepFailed(KeywordMain.groovy:39)
	at com.kms.katalon.core.mobile.keyword.internal.MobileKeywordMain.stepFailed(MobileKeywordMain.groovy:40)
	at com.kms.katalon.core.mobile.keyword.builtin.TapKeyword$_tap_closure1.doCall(TapKeyword.groovy:79)
	at com.kms.katalon.core.mobile.keyword.builtin.TapKeyword$_tap_closure1.call(TapKeyword.groovy)
	at com.kms.katalon.core.mobile.keyword.internal.MobileKeywordMain.runKeyword(MobileKeywordMain.groovy:21)
	at com.kms.katalon.core.mobile.keyword.builtin.TapKeyword.tap(TapKeyword.groovy:84)
	at com.kms.katalon.core.mobile.keyword.builtin.TapKeyword.execute(TapKeyword.groovy:68)
	at com.kms.katalon.core.keyword.internal.KeywordExecutor.executeKeywordForPlatform(KeywordExecutor.groovy:74)
	at com.kms.katalon.core.mobile.keyword.MobileBuiltInKeywords.tap(MobileBuiltInKeywords.groovy:1476)
	at com.kms.katalon.core.mobile.keyword.MobileBuiltInKeywords$tap$0.call(Unknown Source)
	at x.run(x:31)
	at com.kms.katalon.core.main.ScriptEngine.run(ScriptEngine.java:194)
	at com.kms.katalon.core.main.ScriptEngine.runScriptAsRawText(ScriptEngine.java:119)
	at com.kms.katalon.core.main.TestCaseExecutor.runScript(TestCaseExecutor.java:448)
	at com.kms.katalon.core.main.TestCaseExecutor.doExecute(TestCaseExecutor.java:439)
	at com.kms.katalon.core.main.TestCaseExecutor.processExecutionPhase(TestCaseExecutor.java:418)
	at com.kms.katalon.core.main.TestCaseExecutor.accessMainPhase(TestCaseExecutor.java:410)
	at com.kms.katalon.core.main.TestCaseExecutor.execute(TestCaseExecutor.java:285)
	at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:144)
	at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:135)
	at com.kms.katalon.core.main.TestCaseMain$runTestCase$0.call(Unknown Source)
	at TempTestCase1710145272702.run(TempTestCase1710145272702.groovy:25)

2024-03-11 11:21:44.227 ERROR c.k.k.core.keyword.internal.KeywordMain  - ❌ Object Object Repository/WMV/MoreOptionPage/SettingsPage/logoutButton not found
2024-03-11 11:21:44.230 ERROR c.k.katalon.core.main.TestCaseExecutor   - ❌ Test Cases/x FAILED.

And this is the error message I print it using testcaseContext.message, in which only return one error not both errors mentioned above(just first issue founded by test case):

com.kms.katalon.core.exception.StepFailedException: Object Object Repository/HomePage/NavigationBar/moreButton not found
	at com.kms.katalon.core.keyword.internal.KeywordMain.stepFailed(KeywordMain.groovy:39)
	at com.kms.katalon.core.mobile.keyword.internal.MobileKeywordMain.stepFailed(MobileKeywordMain.groovy:40)
	at com.kms.katalon.core.mobile.keyword.builtin.TapKeyword$_tap_closure1.doCall(TapKeyword.groovy:79)
	at com.kms.katalon.core.mobile.keyword.builtin.TapKeyword$_tap_closure1.call(TapKeyword.groovy)
	at com.kms.katalon.core.mobile.keyword.internal.MobileKeywordMain.runKeyword(MobileKeywordMain.groovy:21)
	at com.kms.katalon.core.mobile.keyword.builtin.TapKeyword.tap(TapKeyword.groovy:84)
	at com.kms.katalon.core.mobile.keyword.builtin.TapKeyword.execute(TapKeyword.groovy:68)
	at com.kms.katalon.core.keyword.internal.KeywordExecutor.executeKeywordForPlatform(KeywordExecutor.groovy:74)
	at com.kms.katalon.core.mobile.keyword.MobileBuiltInKeywords.tap(MobileBuiltInKeywords.groovy:1476)
	at com.kms.katalon.core.mobile.keyword.MobileBuiltInKeywords$tap$0.call(Unknown Source)
	at x.run(x:31)
	at com.kms.katalon.core.main.ScriptEngine.run(ScriptEngine.java:194)
	at com.kms.katalon.core.main.ScriptEngine.runScriptAsRawText(ScriptEngine.java:119)
	at com.kms.katalon.core.main.TestCaseExecutor.runScript(TestCaseExecutor.java:448)
	at com.kms.katalon.core.main.TestCaseExecutor.doExecute(TestCaseExecutor.java:439)
	at com.kms.katalon.core.main.TestCaseExecutor.processExecutionPhase(TestCaseExecutor.java:418)
	at com.kms.katalon.core.main.TestCaseExecutor.accessMainPhase(TestCaseExecutor.java:410)
	at com.kms.katalon.core.main.TestCaseExecutor.execute(TestCaseExecutor.java:285)
	at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:144)
	at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:135)
	at com.kms.katalon.core.main.TestCaseMain$runTestCase$0.call(Unknown Source)
	at TempTestCase1710145272702.run(TempTestCase1710145272702.groovy:25)

kazurayam · March 11, 2024, 10:27pm

@osamazaza200

Thank you for providing more information.

I tried to reproduce your issue on my side, and YES I could see a similar incident. The string value returned by testCaseContext.message() contains only 1 StackTrace even if the problem TestCase threw 2 Exceptions during a run. The string returned by testCaseContext.message() did not contain the 2nd StackTraces. Yes, you found a fact.

It seems that the TestCase runner skipped the 2nd StackTrace. I have no idea why. I just guess that Katalon Studio is implemented as such.

And, to be honest, I would not mind this behavior much. To have 0 failure or 1+, that is the question for me. I would not need to see 2 or more failures reported by a buggy TestCase at once, as I would only be able to tackle one-by-one at a time.

If I see 2 or more StackTrace messages appended together, I would rather be bothered with a bulky set of lines; I would have to start with trying to find the boundaries and splitting the lines into a series of Exceptions. It would be tiresome. I wouldn’t like it.

osamazaza200 · March 12, 2024, 8:45am

Thank your kazurayam for your comment and support,
I see that it’s a defect because most of test cases can include multiple UI items[buttons, labels] and etc, and any deviation for any item will be reported an exception in console, and thus, in our single test case we can have more than one issues.

kazurayam · March 12, 2024, 9:28am

If you are a paying Enterprise user, then you should raise an official support request to let Katalon acknowledge the defect.

They would not notice your voice burried in this user forum.

→ @Elly_Tran

Elly_Tran · March 13, 2024, 2:08am

Hi,

Thank you so much for letting us know about this issue. Can you please provide us with more details about your version, your device, expected outcome, … I will create a ticket for the team to investigate more and I will update you soon if anything new. Thank you!

osamazaza200 · March 13, 2024, 9:38am

Thank you for reaching us !
Device: Lenovo thinkpad
Expected Ressult: as I am tester, i am expected to see all issues when using parameter “testCaseContext.message” for function listener @AfterTestCase

Actual Ressult: when a test case fails, there maybe one one issue existed like button x, y not clickable or founded, and message only show the first error message of test case.

osamazaza200 · April 1, 2024, 1:30pm

Hello @Elly_Tran,
Is there any progress about the ticket raised ?
Best Regards,

Elly_Tran · April 2, 2024, 1:54am

Hi,

Which OS version are you using as well as the Katalon version? Sorry for the late response.

osamazaza200 · April 4, 2024, 1:36pm

No issues,
Windows 11
Katalon Version: 8.6.8

Elly_Tran · April 5, 2024, 2:29am

Hi,

Can you please help update Katalon to a newer version? Thank you

osamazaza200 · June 3, 2024, 12:04pm

Dear @Elly_Tran & @kazurayam,
are there any update for the defect raised, note that it’s also existed in the new version of katalon
9.5.0 on windows.

system · June 3, 2024, 12:04pm

Hi there,

Thank you very much for your topic. Please note that it may take a little while before a member of our community or from Katalon team responds to you.

Thanks!

Elly_Tran · June 5, 2024, 1:45am

Hi,

Sorry for our late response. We are still investigating on this

osamazaza200 · June 19, 2024, 6:52am

Dear @Elly_Tran,
have your reached a fix for this issue ?

osamazaza200 · July 11, 2024, 4:37pm

We did not get a resolve from @Elly_Tran or any supportive team. this product is paid and need to provide it’s quality value for it’s customer.

Elly_Tran · July 12, 2024, 2:46am

Hi,

Sorry for my missing and your bad experience. If you are an Enterprise user, please help create a ticket over here for better support: https://katalon-inc.my.site.com/katalonhelpcenter/s/. We would love to helping you

osamazaza200 · July 24, 2024, 12:02pm

Dear @Elly_Tran
I am sorry to say that this is a bug, It’s not a feature that requires me to upgrade to ask for. Appreciate to raise a ticket for this critical bug.

Elly_Tran · July 29, 2024, 8:58am

Yes @osamazaza200,

I have created a ticket for this case and we are investigating. Back to you soon if we need more information

Topic		Replies	Views
How to run Re-run failed testcases if the testcase has been divided into more than one file? API Testing start-page , katalon-studio , api-testing	21	3767	March 24, 2022
TestListener: execute before and after test method twice Web Testing katalon-studio	23	2539	October 29, 2019
How to forward Root Cause Message for Test Step Failure to Test Case Level? Web Testing katalon-studio	11	3559	January 25, 2019
Report FAILED tests only; no PASSED tests, please Bug Report katalon-studio , web-testing , bugs-report	30	3136	October 9, 2023
If a step fails within a test case that is being called by another test case, it display an "Unable to call test case" error. Web Testing web-testing	19	962	March 14, 2023

The function "testCaseContext.message" only returns one error for a test case

Related topics