# Test Case level is set to passed even if one of the steps are mark failed (using KeywordUtil.markFailed)

**URL:** <https://forum.katalon.com/t/test-case-level-is-set-to-passed-even-if-one-of-the-steps-are-mark-failed-using-keywordutil-markfailed/52210>\
**Category:** Web Testing\
**Tags:** katalon-studio\
**Created:** [February 17, 2021, 3:02pm UTC](https://forum.katalon.com/t/test-case-level-is-set-to-passed-even-if-one-of-the-steps-are-mark-failed-using-keywordutil-markfailed/52210 "2021-02-17T15:02:16Z")\
**Posts on this page:** 16\
**Page:** 1

<div class="post-metadata">

**Author:** ![Murphy\_Memije](https://dub1.discourse-cdn.com/katalon/user_avatar/forum.katalon.com/murphy_memije/32/16104_2.png) [@Murphy\_Memije](https://forum.katalon.com/u/Murphy_Memije)\
**Post date:** [February 17, 2021, 3:02pm UTC](https://forum.katalon.com/t/test-case-level-is-set-to-passed-even-if-one-of-the-steps-are-mark-failed-using-keywordutil-markfailed/52210/1 "2021-02-17T15:02:16Z")

</div>

Hi,

I created a custom keyword to verify if two values are matched utilizing the built in keyword for WS.

Snippet:

```
@Keyword
def verifyValuesMatch(String actual, String expected) {
	if (WS.verifyMatch(actual, expected, false)) {
		log.resultLog("PASSED", expected, actual);
	} else {
		log.resultLog("FAILED", expected, actual);
	}
}

```

I then call another function to mark the step failed which uses the built-in KeywordUtil.markFailed

 ![image](https://europe1.discourse-cdn.com/katalon/original/3X/f/a/fa533d31cde01c7e18198632f28cb616ef751c47.png)

All is working fine and the steps are successfully mark failed however I just noticed that even though one of the steps are mark failed, the Test Case level is still set to PASSED which gives me a wrong impression that the actual test has no issues. Is it intended that way? Here’s a sample report in TestOps.

 ![image](https://europe1.discourse-cdn.com/katalon/original/3X/c/5/c5beeeb204d43f394804d14cf05250e08046c0db.png)

 ![image](https://europe1.discourse-cdn.com/katalon/original/3X/0/6/062a689189bf33e574306f245dc8e1fc0ff784e6.png)

I wanted to set the test case to failed if it has any steps that are marked as failed. I know we can use other keywords such as _Assertions.assertThat_ but I want my test to continue on failure since I’m testing a web service which involves verification of multiple attributes.

Is there any workaround that I can do? Thanks in advance for the inputs and suggestions!

---

<div class="post-metadata">

**Author:** ![kazurayam](https://dub1.discourse-cdn.com/katalon/user_avatar/forum.katalon.com/kazurayam/32/333_2.png) [@kazurayam](https://forum.katalon.com/u/kazurayam)\
**Post date:** [February 18, 2021, 3:12am UTC](https://forum.katalon.com/t/test-case-level-is-set-to-passed-even-if-one-of-the-steps-are-mark-failed-using-keywordutil-markfailed/52210/2 "2021-02-18T03:12:37Z")

</div>

> [@Murphy\_Memije](#):
>
> Is it intended that way?

~~I think, yes, it is intended way.~~

~~I can find the javadoc:~~

[https://docs.katalon.com/javadoc/com/kms/katalon/core/util/KeywordUtil.html](https://docs.katalon.com/javadoc/com/kms/katalon/core/util/KeywordUtil.html)

~~Please find 2 methods available~~

~~ 
1. `KeywordUtil.markFailed()`
2. `KeywordUtil.makrFailedAndStop()`

 

> public static void **markFailed** ([String](http://docs.oracle.com/javase/8/docs/api/java/lang/String.html) message)  
> Mark **a keyword to be failed and continue** execution

 ~~

~~The `markFailed` method is intended to mark A KEYWORD to be failed, no Exception will be thrown, no affect to the TEST CASE level.~~

 

~~You should look at another method:~~

~~ 

> public static void **markFailedAndStop** ([String](http://docs.oracle.com/javase/8/docs/api/java/lang/String.html) message)  
> Mark a keyword to be failed and stop execution

 ~~

~~This will affect to the test case level. By this method, the test case will be mark FAILED and an Exception will be thrown.  
 ~~

kazurayam edited: May be I am wrong.

---

<div class="post-metadata">

**Author:** ![Murphy\_Memije](https://dub1.discourse-cdn.com/katalon/user_avatar/forum.katalon.com/murphy_memije/32/16104_2.png) [@Murphy\_Memije](https://forum.katalon.com/u/Murphy_Memije)\
**Post date:** [February 18, 2021, 5:58am UTC](https://forum.katalon.com/t/test-case-level-is-set-to-passed-even-if-one-of-the-steps-are-mark-failed-using-keywordutil-markfailed/52210/5 "2021-02-18T05:58:40Z")

</div>

Hi @kazurayam,

It’s an instance of **com.kms.katalon.util.KeywordUtil**

 ![image](https://europe1.discourse-cdn.com/katalon/original/3X/4/9/49c346ef804f92ebae0a99378e3762d7912723f9.png)

---

<div class="post-metadata">

**Author:** ![Murphy\_Memije](https://dub1.discourse-cdn.com/katalon/user_avatar/forum.katalon.com/murphy_memije/32/16104_2.png) [@Murphy\_Memije](https://forum.katalon.com/u/Murphy_Memije)\
**Post date:** [February 18, 2021, 5:58am UTC](https://forum.katalon.com/t/test-case-level-is-set-to-passed-even-if-one-of-the-steps-are-mark-failed-using-keywordutil-markfailed/52210/6 "2021-02-18T05:58:41Z")

</div>

Thanks @kazurayam, I’ll have a check on this and get back if this will satisfy my use case! Thanks!

---

<div class="post-metadata">

**Author:** ![Murphy\_Memije](https://dub1.discourse-cdn.com/katalon/user_avatar/forum.katalon.com/murphy_memije/32/16104_2.png) [@Murphy\_Memije](https://forum.katalon.com/u/Murphy_Memije)\
**Post date:** [February 18, 2021, 5:58am UTC](https://forum.katalon.com/t/test-case-level-is-set-to-passed-even-if-one-of-the-steps-are-mark-failed-using-keywordutil-markfailed/52210/7 "2021-02-18T05:58:42Z")

</div>

Hi @kazurayam ,

Thanks for the reply and suggestion!

That’s right, I wanted to continue and proceed on the proceeding validations after previous steps were marked failed. Using the **markFailedAndStop** will force stop the execution on the first instance of failure.

To give some examples with my project also, I’m using a BDD approach and I have multiple test case in a single feature file.

e.g.

GIVEN I do this…  
WHEN I do this…  
THEN I verify this… \<— (1st validation)  
AND I verify this… \<----(2nd validation)

I wanted my test to proceed on the 2nd validation without force exiting the execution on the 1st validation.

---

<div class="post-metadata">

**Author:** ![kazurayam](https://dub1.discourse-cdn.com/katalon/user_avatar/forum.katalon.com/kazurayam/32/333_2.png) [@kazurayam](https://forum.katalon.com/u/kazurayam)\
**Post date:** [February 18, 2021, 6:38am UTC](https://forum.katalon.com/t/test-case-level-is-set-to-passed-even-if-one-of-the-steps-are-mark-failed-using-keywordutil-markfailed/52210/8 "2021-02-18T06:38:16Z")

</div>

> [@Murphy\_Memije](#):
>
> I just noticed that even though one of the steps are mark failed, the Test Case level is still set to PASSED

I tried to reproduce this on my side.

Here is my skeltal example:

Keyword: `my.ResultLogger`

```auto
package my

import com.kms.katalon.core.util.KeywordUtil

public class ResultLogger {
	int numberOfFailures = 0

	def getNumberOfFailres() {
		return numberOfFailures
	}

	def resultlog(type, expected, actual) {
		switch(type) {
			case "R02":
				KeywordUtil.markFailed("failed")
				numberOfFailures += 1
				break
		}
	}
}

```

WebUI Test Case:

```auto
import com.kms.katalon.core.util.KeywordUtil

import my.ResultLogger

ResultLogger logger = new ResultLogger()

logger.resultlog("R02", "expected", "actual")
logger.resultlog("R02", "expected", "actual")
logger.resultlog("R02", "expected", "actual")

/*
if (logger.getNumberOfFailures() > 0) {
	KeywordUtil.markFailedAndStop("one or more failures occured")
}
*/

```

When I executed the Test Case, I got output in the console:

```auto
2021-02-18 15:42:08.849 INFO c.k.katalon.core.main.TestCaseExecutor - START Test Cases/TC3
2021-02-18 15:42:10.144 DEBUG testcase.TC3 - 1: logger = new my.ResultLogger()
2021-02-18 15:42:10.163 DEBUG testcase.TC3 - 2: logger.resultlog("R02", "expected", "actual")
2021-02-18 15:42:10.203 ERROR com.kms.katalon.core.util.KeywordUtil - ❌ failed
2021-02-18 15:42:10.260 DEBUG testcase.TC3 - 3: logger.resultlog("R02", "expected", "actual")
2021-02-18 15:42:10.269 ERROR com.kms.katalon.core.util.KeywordUtil - ❌ failed
2021-02-18 15:42:10.284 DEBUG testcase.TC3 - 4: logger.resultlog("R02", "expected", "actual")
2021-02-18 15:42:10.291 ERROR com.kms.katalon.core.util.KeywordUtil - ❌ failed
2021-02-18 15:42:10.319 ERROR c.k.katalon.core.main.TestCaseExecutor - ❌ Test Cases/TC3 FAILED.
Reason:
com.kms.katalon.core.exception.StepFailedException: failed
	at com.kms.katalon.core.util.KeywordUtil.markFailed(KeywordUtil.java:19)
	at com.kms.katalon.core.util.KeywordUtil$markFailed.call(Unknown Source)
	at my.ResultLogger.resultlog(ResultLogger.groovy:15)
	at my.ResultLogger$resultlog.call(Unknown Source)
	at TC3.run(TC3:7)
	at com.kms.katalon.core.main.ScriptEngine.run(ScriptEngine.java:194)
	at com.kms.katalon.core.main.ScriptEngine.runScriptAsRawText(ScriptEngine.java:119)
	at com.kms.katalon.core.main.TestCaseExecutor.runScript(TestCaseExecutor.java:398)
	at com.kms.katalon.core.main.TestCaseExecutor.doExecute(TestCaseExecutor.java:389)
	at com.kms.katalon.core.main.TestCaseExecutor.processExecutionPhase(TestCaseExecutor.java:368)
	at com.kms.katalon.core.main.TestCaseExecutor.accessMainPhase(TestCaseExecutor.java:360)
	at com.kms.katalon.core.main.TestCaseExecutor.execute(TestCaseExecutor.java:255)
	at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:114)
	at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:105)
	at com.kms.katalon.core.main.TestCaseMain$runTestCase$0.call(Unknown Source)
	at TempTestCase1613630524859.run(TempTestCase1613630524859.groovy:25)

2021-02-18 15:42:10.371 INFO c.k.katalon.core.main.TestCaseExecutor - END Test Cases/TC3

```

As you see, the Test Case was marked FAILED when it called `KeywordUtil.markFailed()`. On the other hand, you wrote that your Test Case is marked PASSED.

I do not see the reason why we see this discrepancy.

---

<div class="post-metadata">

**Author:** ![Murphy\_Memije](https://dub1.discourse-cdn.com/katalon/user_avatar/forum.katalon.com/murphy_memije/32/16104_2.png) [@Murphy\_Memije](https://forum.katalon.com/u/Murphy_Memije)\
**Post date:** [February 18, 2021, 7:34am UTC](https://forum.katalon.com/t/test-case-level-is-set-to-passed-even-if-one-of-the-steps-are-mark-failed-using-keywordutil-markfailed/52210/9 "2021-02-18T07:34:11Z")

</div>

> [@kazurayam](#):
>
> As you see, the Test Case was marked FAILED when it called `KeywordUtil.markFailed()` . On the other hand, you wrote that your Test Case is marked PASSED.

Thanks for this detailed example @kazurayam really helps. After further checking with my logs, it is indeed failing my test case. Sample log below:

```
2021-02-18 15:57:19.780 INFO c.k.katalon.core.main.TestCaseExecutor - --------------------
2021-02-18 15:57:19.782 INFO c.k.katalon.core.main.TestCaseExecutor - START Test Cases/xcore/orders/order-creation
2021-02-18 15:57:21.727 DEBUG testcase.order-creation - 1: runFeatureFile("Include/features/xcore/orders/test.feature")
2021-02-18 15:57:21.782 INFO c.k.k.c.c.k.CucumberBuiltinKeywords - Starting run keyword runFeatureFile: 'Include/features/xcore/orders/test.feature' and extract report to folder: 'C:\Users\Murphy\AppData\Local\Temp\Katalon\Test Cases\xcore\orders\order-creation\20210218_155717/cucumber_report/1613635041772'...
log4j:WARN No appenders could be found for logger (org.apache.http.client.protocol.RequestAddCookies).
log4j:WARN Please initialize the log4j system properly.
2021-02-18 15:57:24.048 INFO c.k.k.core.webservice.common.HarLogger - HAR: C:\Users\Murphy\AppData\Local\Temp\Katalon\Test Cases\xcore\orders\order-creation\20210218_155717\requests\main\0.har
2021-02-18 15:57:24.254 DEBUG com.kms.katalon.core.util.KeywordUtil - ✓ [PASSED]: Expected = 200 Actual = 200
2021-02-18 15:57:24.279 INFO com.kms.katalon.core.util.KeywordUtil - [INFO]: Shopee order created with tracking number 0642-4178-CBRQ
2021-02-18 15:57:24.904 INFO c.k.k.core.webservice.common.HarLogger - HAR: C:\Users\Murphy\AppData\Local\Temp\Katalon\Test Cases\xcore\orders\order-creation\20210218_155717\requests\main\1.har
2021-02-18 15:57:24.909 DEBUG com.kms.katalon.core.util.KeywordUtil - ✓ [PASSED]: Expected = 200 Actual = 200
2021-02-18 15:57:24.935 DEBUG com.kms.katalon.core.util.KeywordUtil - ✓ [PASSED]: Expected = Shopee Actual = Shopee
2021-02-18 15:57:25.380 INFO c.k.k.core.webservice.common.HarLogger - HAR: C:\Users\Murphy\AppData\Local\Temp\Katalon\Test Cases\xcore\orders\order-creation\20210218_155717\requests\main\2.har
2021-02-18 15:57:25.383 DEBUG com.kms.katalon.core.util.KeywordUtil - ✓ [PASSED]: Expected = 200 Actual = 200
2021-02-18 15:57:25.392 ERROR c.k.k.core.keyword.internal.KeywordMain - ❌ Unable to verify match between actual text 'MMB' and expected text 'LUZ' (Root cause: com.kms.katalon.core.exception.StepFailedException: Actual text 'MMB' and expected text 'LUZ' are not matched
	at com.kms.katalon.core.keyword.builtin.VerifyMatchKeyword$_verifyMatch_closure1.doCall(VerifyMatchKeyword.groovy:57)
	at com.kms.katalon.core.keyword.builtin.VerifyMatchKeyword$_verifyMatch_closure1.call(VerifyMatchKeyword.groovy)
	at com.kms.katalon.core.keyword.internal.KeywordMain.runKeyword(KeywordMain.groovy:68)
	at com.kms.katalon.core.keyword.builtin.VerifyMatchKeyword.verifyMatch(VerifyMatchKeyword.groovy:60)
	at com.kms.katalon.core.keyword.builtin.VerifyMatchKeyword.execute(VerifyMatchKeyword.groovy:45)
	at com.kms.katalon.core.keyword.internal.KeywordExecutor.executeKeywordForPlatform(KeywordExecutor.groovy:73)
	at com.kms.katalon.core.keyword.BuiltinKeywords.verifyMatch(BuiltinKeywords.groovy:73)
	at utils.Validator.verifyValuesMatch(Validator.groovy:62)
	at utils.Validator$verifyValuesMatch$0.call(Unknown Source)
	at step_definitions.TestFlowOrders.the_order_attribute_should_be_in_the_GET_order_response(TestFlowOrders.groovy:152)
	at cucumber.runtime.Utils$1.call(Utils.java:26)
	at cucumber.runtime.Timeout.timeout(Timeout.java:16)
	at cucumber.runtime.Utils.invoke(Utils.java:20)
	at cucumber.runtime.java.JavaStepDefinition.execute(JavaStepDefinition.java:48)
	at cucumber.runtime.PickleStepDefinitionMatch.runStep(PickleStepDefinitionMatch.java:50)
	at cucumber.runner.TestStep.executeStep(TestStep.java:55)
	at cucumber.runner.TestStep.run(TestStep.java:42)
	at cucumber.runner.PickleStepTestStep.run(PickleStepTestStep.java:53)
	at cucumber.runner.TestCase.run(TestCase.java:47)
	at cucumber.runner.Runner.runPickle(Runner.java:44)
	at cucumber.runtime.Runtime.runFeature(Runtime.java:120)
	at cucumber.runtime.Runtime.run(Runtime.java:106)
	at cucumber.api.cli.Main.run(Main.java:35)
	at cucumber.api.cli.Main$run.call(Unknown Source)
	at com.kms.katalon.core.cucumber.keyword.CucumberBuiltinKeywords$_runFeatureFile_closure1.doCall(CucumberBuiltinKeywords.groovy:106)
	at com.kms.katalon.core.cucumber.keyword.CucumberBuiltinKeywords$_runFeatureFile_closure1.doCall(CucumberBuiltinKeywords.groovy)
	at com.kms.katalon.core.keyword.internal.KeywordMain.runKeyword(KeywordMain.groovy:68)
	at com.kms.katalon.core.keyword.internal.KeywordMain$runKeyword.call(Unknown Source)
	at com.kms.katalon.core.cucumber.keyword.CucumberBuiltinKeywords.runFeatureFile(CucumberBuiltinKeywords.groovy:73)
	at com.kms.katalon.core.cucumber.keyword.CucumberBuiltinKeywords$runFeatureFile$0.callStatic(Unknown Source)
	at com.kms.katalon.core.cucumber.keyword.CucumberBuiltinKeywords.runFeatureFile(CucumberBuiltinKeywords.groovy:246)
	at com.kms.katalon.core.cucumber.keyword.CucumberBuiltinKeywords$runFeatureFile.call(Unknown Source)
	at order-creation.run(order-creation:19)
	at com.kms.katalon.core.main.ScriptEngine.run(ScriptEngine.java:194)
	at com.kms.katalon.core.main.ScriptEngine.runScriptAsRawText(ScriptEngine.java:119)
	at com.kms.katalon.core.main.TestCaseExecutor.runScript(TestCaseExecutor.java:398)
	at com.kms.katalon.core.main.TestCaseExecutor.doExecute(TestCaseExecutor.java:389)
	at com.kms.katalon.core.main.TestCaseExecutor.processExecutionPhase(TestCaseExecutor.java:368)
	at com.kms.katalon.core.main.TestCaseExecutor.accessMainPhase(TestCaseExecutor.java:360)
	at com.kms.katalon.core.main.TestCaseExecutor.execute(TestCaseExecutor.java:255)
	at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:114)
	at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:105)
	at com.kms.katalon.core.main.TestCaseMain$runTestCase$0.call(Unknown Source)
	at TempTestCase1613635037152.run(TempTestCase1613635037152.groovy:25)
)
2021-02-18 15:57:25.396 ERROR com.kms.katalon.core.util.KeywordUtil - ❌ [FAILED]: Expected = LUZ Actual = MMB
2021-02-18 15:57:25.783 INFO c.k.k.core.webservice.common.HarLogger - HAR: C:\Users\Murphy\AppData\Local\Temp\Katalon\Test Cases\xcore\orders\order-creation\20210218_155717\requests\main\3.har
2021-02-18 15:57:25.786 DEBUG com.kms.katalon.core.util.KeywordUtil - ✓ [PASSED]: Expected = 200 Actual = 200
2021-02-18 15:57:25.790 DEBUG com.kms.katalon.core.util.KeywordUtil - ✓ [PASSED]: Expected = cod Actual = cod
2021-02-18 15:57:26.235 INFO c.k.k.core.webservice.common.HarLogger - HAR: C:\Users\Murphy\AppData\Local\Temp\Katalon\Test Cases\xcore\orders\order-creation\20210218_155717\requests\main\4.har
2021-02-18 15:57:26.237 DEBUG com.kms.katalon.core.util.KeywordUtil - ✓ [PASSED]: Expected = 200 Actual = 200
2021-02-18 15:57:26.240 ERROR c.k.k.core.keyword.internal.KeywordMain - ❌ Unable to verify match between actual text 'next_day' and expected text 'same_day' (Root cause: com.kms.katalon.core.exception.StepFailedException: Actual text 'next_day' and expected text 'same_day' are not matched
	at com.kms.katalon.core.keyword.builtin.VerifyMatchKeyword$_verifyMatch_closure1.doCall(VerifyMatchKeyword.groovy:57)
	at com.kms.katalon.core.keyword.builtin.VerifyMatchKeyword$_verifyMatch_closure1.call(VerifyMatchKeyword.groovy)
	at com.kms.katalon.core.keyword.internal.KeywordMain.runKeyword(KeywordMain.groovy:68)
	at com.kms.katalon.core.keyword.builtin.VerifyMatchKeyword.verifyMatch(VerifyMatchKeyword.groovy:60)
	at com.kms.katalon.core.keyword.builtin.VerifyMatchKeyword.execute(VerifyMatchKeyword.groovy:45)
	at com.kms.katalon.core.keyword.internal.KeywordExecutor.executeKeywordForPlatform(KeywordExecutor.groovy:73)
	at com.kms.katalon.core.keyword.BuiltinKeywords.verifyMatch(BuiltinKeywords.groovy:73)
	at utils.Validator.verifyValuesMatch(Validator.groovy:62)
	at utils.Validator$verifyValuesMatch$0.call(Unknown Source)
	at step_definitions.TestFlowOrders.the_order_attribute_should_be_in_the_GET_order_response(TestFlowOrders.groovy:152)
	at cucumber.runtime.Utils$1.call(Utils.java:26)
	at cucumber.runtime.Timeout.timeout(Timeout.java:16)
	at cucumber.runtime.Utils.invoke(Utils.java:20)
	at cucumber.runtime.java.JavaStepDefinition.execute(JavaStepDefinition.java:48)
	at cucumber.runtime.PickleStepDefinitionMatch.runStep(PickleStepDefinitionMatch.java:50)
	at cucumber.runner.TestStep.executeStep(TestStep.java:55)
	at cucumber.runner.TestStep.run(TestStep.java:42)
	at cucumber.runner.PickleStepTestStep.run(PickleStepTestStep.java:53)
	at cucumber.runner.TestCase.run(TestCase.java:47)
	at cucumber.runner.Runner.runPickle(Runner.java:44)
	at cucumber.runtime.Runtime.runFeature(Runtime.java:120)
	at cucumber.runtime.Runtime.run(Runtime.java:106)
	at cucumber.api.cli.Main.run(Main.java:35)
	at cucumber.api.cli.Main$run.call(Unknown Source)
	at com.kms.katalon.core.cucumber.keyword.CucumberBuiltinKeywords$_runFeatureFile_closure1.doCall(CucumberBuiltinKeywords.groovy:106)
	at com.kms.katalon.core.cucumber.keyword.CucumberBuiltinKeywords$_runFeatureFile_closure1.doCall(CucumberBuiltinKeywords.groovy)
	at com.kms.katalon.core.keyword.internal.KeywordMain.runKeyword(KeywordMain.groovy:68)
	at com.kms.katalon.core.keyword.internal.KeywordMain$runKeyword.call(Unknown Source)
	at com.kms.katalon.core.cucumber.keyword.CucumberBuiltinKeywords.runFeatureFile(CucumberBuiltinKeywords.groovy:73)
	at com.kms.katalon.core.cucumber.keyword.CucumberBuiltinKeywords$runFeatureFile$0.callStatic(Unknown Source)
	at com.kms.katalon.core.cucumber.keyword.CucumberBuiltinKeywords.runFeatureFile(CucumberBuiltinKeywords.groovy:246)
	at com.kms.katalon.core.cucumber.keyword.CucumberBuiltinKeywords$runFeatureFile.call(Unknown Source)
	at order-creation.run(order-creation:19)
	at com.kms.katalon.core.main.ScriptEngine.run(ScriptEngine.java:194)
	at com.kms.katalon.core.main.ScriptEngine.runScriptAsRawText(ScriptEngine.java:119)
	at com.kms.katalon.core.main.TestCaseExecutor.runScript(TestCaseExecutor.java:398)
	at com.kms.katalon.core.main.TestCaseExecutor.doExecute(TestCaseExecutor.java:389)
	at com.kms.katalon.core.main.TestCaseExecutor.processExecutionPhase(TestCaseExecutor.java:368)
	at com.kms.katalon.core.main.TestCaseExecutor.accessMainPhase(TestCaseExecutor.java:360)
	at com.kms.katalon.core.main.TestCaseExecutor.execute(TestCaseExecutor.java:255)
	at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:114)
	at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:105)
	at com.kms.katalon.core.main.TestCaseMain$runTestCase$0.call(Unknown Source)
	at TempTestCase1613635037152.run(TempTestCase1613635037152.groovy:25)
)
2021-02-18 15:57:26.241 ERROR com.kms.katalon.core.util.KeywordUtil - ❌ [FAILED]: Expected = same_day Actual = next_day

1 Scenarios (e[32m1 passede[0m)
8 Steps (e[32m8 passede[0m)
0m3.698s

2021-02-18 15:57:26.293 ERROR c.k.katalon.core.main.TestCaseExecutor - ❌ Test Cases/xcore/orders/order-creation FAILED.

```

However I’m not sure how this is being interpreted by TestOps because clearly it’ showing that the test case/ test suite passed where there are actually failed assertions (as for with my original post.)

 ![image](https://europe1.discourse-cdn.com/katalon/original/3X/7/5/75344ea527005fbd40ab4a043566c6921f1a432f.png)

 ![image](https://europe1.discourse-cdn.com/katalon/original/3X/a/3/a3763fb8152d92ca42a5866e2c399e6f07420f34.png)

 ![image](https://europe1.discourse-cdn.com/katalon/original/3X/4/b/4b50b553424ce32a4d245aa625a2e42df97e7bf1.png)

---

<div class="post-metadata">

**Author:** ![kazurayam](https://dub1.discourse-cdn.com/katalon/user_avatar/forum.katalon.com/kazurayam/32/333_2.png) [@kazurayam](https://forum.katalon.com/u/kazurayam)\
**Post date:** [February 18, 2021, 8:47am UTC](https://forum.katalon.com/t/test-case-level-is-set-to-passed-even-if-one-of-the-steps-are-mark-failed-using-keywordutil-markfailed/52210/10 "2021-02-18T08:47:36Z")

</div>

I found that `com.kms.katalon.core.main.TestCaseExecutor` and `com.kms.katalon.core.main.WSVerificationExecutor` — these 2 classes execute test case scripts differently.

- TestCaseExecutor throws Exception and report that the Test Case has FAILED when the test case script calls `com.kms.katalon.util.KeywordUtil.markFailed()` once or more times.

- **WSVerificationExcutor does NOT** throw any Exception, does NOT report Test Case failed when the test case script calls `com.kms.katalon.util.KeywordUtil.markFailed()` once or more times

I could read the source codes and got sure of this.

* * *

---

<div class="post-metadata">

**Author:** ![Murphy\_Memije](https://dub1.discourse-cdn.com/katalon/user_avatar/forum.katalon.com/murphy_memije/32/16104_2.png) [@Murphy\_Memije](https://forum.katalon.com/u/Murphy_Memije)\
**Post date:** [February 18, 2021, 9:14am UTC](https://forum.katalon.com/t/test-case-level-is-set-to-passed-even-if-one-of-the-steps-are-mark-failed-using-keywordutil-markfailed/52210/11 "2021-02-18T09:14:04Z")

</div>

> [@kazurayam](#):
>
> - TestCaseExecutor throws Exception and report that the Test Case has FAILED when the test case script calls `com.kms.katalon.util.KeywordUtil.markFailed()` once or more times.
> - **WSVerificationExcutor does NOT** throw any Exception, does NOT report Test Case failed when the test case script calls `com.kms.katalon.util.KeywordUtil.markFailed()` once or more times

Good to know how these two execute scripts. So TestCaseExecutor is the way for me. But then I’m not sure why TestOps reports behave differently (previous screenshots - Test Case/Suite passed but with failed assertions). Should I raise in the TestOps category?

Appreciate your time taking a deeper look into this @kazurayam very helpful thanks!

---

<div class="post-metadata">

**Author:** ![kazurayam](https://dub1.discourse-cdn.com/katalon/user_avatar/forum.katalon.com/kazurayam/32/333_2.png) [@kazurayam](https://forum.katalon.com/u/kazurayam)\
**Post date:** [February 18, 2021, 9:15am UTC](https://forum.katalon.com/t/test-case-level-is-set-to-passed-even-if-one-of-the-steps-are-mark-failed-using-keywordutil-markfailed/52210/12 "2021-02-18T09:15:59Z")

</div>

> [@Murphy\_Memije](#):
>
> TestOps

I know nothing about TestOps.

---

<div class="post-metadata">

**Author:** ![kazurayam](https://dub1.discourse-cdn.com/katalon/user_avatar/forum.katalon.com/kazurayam/32/333_2.png) [@kazurayam](https://forum.katalon.com/u/kazurayam)\
**Post date:** [February 19, 2021, 3:09am UTC](https://forum.katalon.com/t/test-case-level-is-set-to-passed-even-if-one-of-the-steps-are-mark-failed-using-keywordutil-markfailed/52210/13 "2021-02-19T03:09:04Z")

</div>

> [@Murphy\_Memije](#):
>
> However I’m not sure how this is being interpreted by TestOps because clearly it’ showing that the test case/ test suite passed where there are actually failed assertions (as for with my original post.)

I guess, TestOps is not interpreting the result of test case/test suites. TestOps just displays what was reported by the class which executed your test.

With which executor class you executed your test — TestCaseExecutor or WSVerificationExecutor or anything else?

---

<div class="post-metadata">

**Author:** ![Murphy\_Memije](https://dub1.discourse-cdn.com/katalon/user_avatar/forum.katalon.com/murphy_memije/32/16104_2.png) [@Murphy\_Memije](https://forum.katalon.com/u/Murphy_Memije)\
**Post date:** [February 19, 2021, 4:57am UTC](https://forum.katalon.com/t/test-case-level-is-set-to-passed-even-if-one-of-the-steps-are-mark-failed-using-keywordutil-markfailed/52210/14 "2021-02-19T04:57:10Z")

</div>

> [@kazurayam](#):
>
> With which executor class you executed your test — TestCaseExecutor or WSVerificationExecutor or anything else?

I have tried both, here’s an example of the html report generated if this will help.  
Edit: Actually in order to publish the report to TestOps you must run it via Test Suite level which always uses TestCaseExecutor class I believe.

Using TestExecutorClass:  
[20210219\_112254.html](https://forum.katalon.com/uploads/short-url/ffoEOpbw66RtDFXRJDzrE52iPZ.html) (222.2 KB)

In TestOps:

 ![image](https://europe1.discourse-cdn.com/katalon/original/3X/a/9/a99c8144bcad55ab5a1917feee6225a55ac1f81f.png)

---

<div class="post-metadata">

**Author:** ![kazurayam](https://dub1.discourse-cdn.com/katalon/user_avatar/forum.katalon.com/kazurayam/32/333_2.png) [@kazurayam](https://forum.katalon.com/u/kazurayam)\
**Post date:** [February 19, 2021, 5:21am UTC](https://forum.katalon.com/t/test-case-level-is-set-to-passed-even-if-one-of-the-steps-are-mark-failed-using-keywordutil-markfailed/52210/15 "2021-02-19T05:21:37Z")

</div>

Thank you for sharing the screenshot of TestOps.

I have never used TestOps. I know nothing about the report.

I would like to pass this issue Katalon teaｍ

@ThanhTo  
@duyluong

---

<div class="post-metadata">

**Author:** ![Murphy\_Memije](https://dub1.discourse-cdn.com/katalon/user_avatar/forum.katalon.com/murphy_memije/32/16104_2.png) [@Murphy\_Memije](https://forum.katalon.com/u/Murphy_Memije)\
**Post date:** [February 19, 2021, 5:26am UTC](https://forum.katalon.com/t/test-case-level-is-set-to-passed-even-if-one-of-the-steps-are-mark-failed-using-keywordutil-markfailed/52210/16 "2021-02-19T05:26:44Z")

</div>

Thanks @kazurayam ! Appreciate it.

Looking forward to hear from the Katalon Team. Thanks!

---

<div class="post-metadata">

**Author:** ![kate.mallinson](https://dub1.discourse-cdn.com/katalon/user_avatar/forum.katalon.com/kate.mallinson/32/16104_2.png) [@kate.mallinson](https://forum.katalon.com/u/kate.mallinson)\
**Post date:** [December 8, 2022, 8:27pm UTC](https://forum.katalon.com/t/test-case-level-is-set-to-passed-even-if-one-of-the-steps-are-mark-failed-using-keywordutil-markfailed/52210/17 "2022-12-08T20:27:06Z")

</div>

@here I’d be really interested to know if there is a recommended way forward, we also have Studio and TestOps and are seeing what looks like different results in both?

---

<div class="post-metadata">

**Author:** ![kevin.jackey](https://dub1.discourse-cdn.com/katalon/user_avatar/forum.katalon.com/kevin.jackey/32/22420_2.png) [@kevin.jackey](https://forum.katalon.com/u/kevin.jackey)\
**Post date:** [February 22, 2023, 1:15pm UTC](https://forum.katalon.com/t/test-case-level-is-set-to-passed-even-if-one-of-the-steps-are-mark-failed-using-keywordutil-markfailed/52210/18 "2023-02-22T13:15:17Z")

</div>

> [@kazurayam](#):
>
> I found that `com.kms.katalon.core.main.TestCaseExecutor` and `com.kms.katalon.core.main.WSVerificationExecutor` — these 2 classes execute test case scripts differently.
> 
> - TestCaseExecutor throws Exception and report that the Test Case has FAILED when the test case script calls `com.kms.katalon.util.KeywordUtil.markFailed()` once or more times.
> - **WSVerificationExcutor does NOT** throw any Exception, does NOT report Test Case failed when the test case script calls `com.kms.katalon.util.KeywordUtil.markFailed()` once or more times

Having the same issue here as well