I tried to reproduce this on my side.
Here is my skeltal example:
Keyword: my.ResultLogger
package my
import com.kms.katalon.core.util.KeywordUtil
public class ResultLogger {
int numberOfFailures = 0
def getNumberOfFailres() {
return numberOfFailures
}
def resultlog(type, expected, actual) {
switch(type) {
case "R02":
KeywordUtil.markFailed("failed")
numberOfFailures += 1
break
}
}
}
WebUI Test Case:
import com.kms.katalon.core.util.KeywordUtil
import my.ResultLogger
ResultLogger logger = new ResultLogger()
logger.resultlog("R02", "expected", "actual")
logger.resultlog("R02", "expected", "actual")
logger.resultlog("R02", "expected", "actual")
/*
if (logger.getNumberOfFailures() > 0) {
KeywordUtil.markFailedAndStop("one or more failures occured")
}
*/
When I executed the Test Case, I got output in the console:
2021-02-18 15:42:08.849 INFO c.k.katalon.core.main.TestCaseExecutor - START Test Cases/TC3
2021-02-18 15:42:10.144 DEBUG testcase.TC3 - 1: logger = new my.ResultLogger()
2021-02-18 15:42:10.163 DEBUG testcase.TC3 - 2: logger.resultlog("R02", "expected", "actual")
2021-02-18 15:42:10.203 ERROR com.kms.katalon.core.util.KeywordUtil - ❌ failed
2021-02-18 15:42:10.260 DEBUG testcase.TC3 - 3: logger.resultlog("R02", "expected", "actual")
2021-02-18 15:42:10.269 ERROR com.kms.katalon.core.util.KeywordUtil - ❌ failed
2021-02-18 15:42:10.284 DEBUG testcase.TC3 - 4: logger.resultlog("R02", "expected", "actual")
2021-02-18 15:42:10.291 ERROR com.kms.katalon.core.util.KeywordUtil - ❌ failed
2021-02-18 15:42:10.319 ERROR c.k.katalon.core.main.TestCaseExecutor - ❌ Test Cases/TC3 FAILED.
Reason:
com.kms.katalon.core.exception.StepFailedException: failed
at com.kms.katalon.core.util.KeywordUtil.markFailed(KeywordUtil.java:19)
at com.kms.katalon.core.util.KeywordUtil$markFailed.call(Unknown Source)
at my.ResultLogger.resultlog(ResultLogger.groovy:15)
at my.ResultLogger$resultlog.call(Unknown Source)
at TC3.run(TC3:7)
at com.kms.katalon.core.main.ScriptEngine.run(ScriptEngine.java:194)
at com.kms.katalon.core.main.ScriptEngine.runScriptAsRawText(ScriptEngine.java:119)
at com.kms.katalon.core.main.TestCaseExecutor.runScript(TestCaseExecutor.java:398)
at com.kms.katalon.core.main.TestCaseExecutor.doExecute(TestCaseExecutor.java:389)
at com.kms.katalon.core.main.TestCaseExecutor.processExecutionPhase(TestCaseExecutor.java:368)
at com.kms.katalon.core.main.TestCaseExecutor.accessMainPhase(TestCaseExecutor.java:360)
at com.kms.katalon.core.main.TestCaseExecutor.execute(TestCaseExecutor.java:255)
at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:114)
at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:105)
at com.kms.katalon.core.main.TestCaseMain$runTestCase$0.call(Unknown Source)
at TempTestCase1613630524859.run(TempTestCase1613630524859.groovy:25)
2021-02-18 15:42:10.371 INFO c.k.katalon.core.main.TestCaseExecutor - END Test Cases/TC3
As you see, the Test Case was marked FAILED when it called KeywordUtil.markFailed(). On the other hand, you wrote that your Test Case is marked PASSED.
I do not see the reason why we see this discrepancy.
