Failed Test case is shown as Warning in Test Report

Feedback & Reviews Bugs Report
1

Hello,

I found a similar topic, but unfortunately it didn’t provide any helpful answers. Every time a test case fails due to a failed test step, the entire case is marked as a warning , and the test suite is shown as passed . This behavior is not very helpful — how can I change this so that the test case and test suite are properly marked as failed ? My failure handling is set to Stop on failure.

Best regards,
Tom

1 Like
2

To ensure failed test cases and suites are correctly marked as failed (not warnings) in Katalon Studio, follow these steps:

1. Configure Failure Handling Properly

  • Project-Level Settings:
    • Go to Project Settings > Execution > Failure Handling.
    • Select “Stop on Failure” for Default Failure Handling.
    • Uncheck Continue executing test case even when one test step fails.
  • Test Case/Suite-Level Override:
    • In your test case’s Script tab, add:

groovy

import com.kms.katalon.core.configuration.RunConfiguration
RunConfiguration.setStopOnFailure(true)

2. Avoid “Mark as Warning” in Scripts

  • Ensure no manual KeywordUtil.markWarning() is used for failures. Replace with:

groovy

if (actual != expected) {
  KeywordUtil.markFailedAndStop("Test failed: Actual ≠ Expected")
}

3. Fix Scripts with Proper Assertions

  • Use built-in verification keywords that auto-fail tests on failure:

groovy

WebUI.verifyEqual(actual, expected) // Fails test if assertion fails

instead of:

groovy

if (actual != expected) {
  KeywordUtil.logInfo("Warning: Values differ")
}

4. Update Test Case Postcondition

  • Add a @TearDown method to explicitly mark failed tests:

groovy

@TearDown
def tearDown() {
  if (TestCaseContext.getTestCaseStatus() == 'FAILED') {
    KeywordUtil.markFailed("Test case failed")
  }
}

5. Check Test Suite Config

  • In your Test Suite settings:
    • Uncheck Continue on failure.
    • Set Failure behavior to “Fail all subsequent test cases”.

6. Update Katalon Studio

  • Upgrade to the latest version to avoid bugs in older versions that misreport statuses.

Example Fix

Before (causes warnings):

groovy

try {
  WebUI.click(findTestObject('LoginButton'))
} catch (Exception e) {
  KeywordUtil.markWarning("Button not found")
}

After (proper failure handling):

groovy

WebUI.click(findTestObject('LoginButton')) // Auto-fails if element not found

7. Verify Execution Logs

  • Check the Log Viewer for entries like Test Case FAILED instead of WARNING.
  • Look for explicit failure reasons (e.g., Step failed: Element not found).

Result

Test cases/suites will now correctly show:

  • Failed :red_circle: when any critical step fails.
  • Passed :white_check_mark: only if all steps pass.
3

Hi @t.haasler,

Thank you for reporting this issue. I will check with my team and back here soon

5

Hi @t.haasler,

Thank you for letting us know your issue. This problem has been acknowledged by our team and we plan to fix it soon in next release: 10.2.1 in the middle of June. For now, please try the suggestion from @dineshh . Thank you

6

I am on 10.2.1 and the issue is still happening.

7

I’m having this same issue as well. It’s a big problem as it looks like my test suites are passing, when in fact there are failures.

8

Hi all,

Sorry for my late response. Due to some changes, the fix for this issue has not yet been implemented and we expect to deliver it as soon as possible. I will keep you guys updated with anything new. Thank you

1 Like