My test passed... but the screen shows an error?

Web Testing
1

Hi everyone, I’m really confused about how Katalon decides if a test passes or fails. I’m running a test to check if a form submits correctly. When I look at the Log Viewer at the end of the run, every single step has a green checkmark and the final status is PASSED.

But when I open the Screenshots folder to see what happened, the final screenshot shows a big red “Invalid Data” error message on the website! If the error message appeared, why didn’t the test fail?

Here is the code I’m using for that part:

WebUI.setText(findTestObject('Object Repository/Input_Email'), 'wrong-email-format')

WebUI.click(findTestObject('Object Repository/Btn_Submit'))

// I thought this would check for success...

WebUI.verifyElementPresent(findTestObject('Object Repository/Success_Message'), 5, FailureHandling.OPTIONAL)

It seems like Katalon just ignores the fact that the “Success” message never showed up and the “Error” message did. How do I make the test actually fail when the UI shows an error?

2

@wazir

The issue is caused by this line:

WebUI.verifyElementPresent(findTestObject('Object Repository/Success_Message'), 5, FailureHandling.OPTIONAL)

When you use:

FailureHandling.OPTIONAL

Katalon will NOT fail the test even if the verification fails

So in your case:

  • “Success_Message” is not found

  • But Katalon treats it as optional → continues execution

  • No step throws a failure → test is marked PASSED

Solution

Remove OPTIONAL

Just change to below verifyElementPresent withoptional

WebUI.verifyElementPresent(findTestObject('Object Repository/Success_Message'), 5)

Default behavior is:

FailureHandling.STOP_ON_FAILURE

Now if success message is not found → test will FAIL

3

FailureHandling.OPTIONAL swallows verification failures—test passes despite “Invalid Data” error.

Fix

Change this:

WebUI.verifyElementPresent(Success_Message, 5, FailureHandling.OPTIONAL)  // Ignores missing success

To this (default STOP_ON_FAILURE):

WebUI.verifyElementPresent(Success_Message, 5)  // Fails test if no success

Better Negative Test

Validate ERROR shown (your real expectation):

WebUI.verifyElementPresent(Error_Message, 5)  // Fail if no error after bad email

Full flow:

WebUI.setText(Email, 'wrong-email')
WebUI.click(Submit)
WebUI.verifyElementPresent(Error_Message, 5)  // PASS: error expected
WebUI.verifyElementNotPresent(Success_Message, 5)  // FAIL if success sneaks in

Screenshot proves error—verify it exists, not optional success!

4

Please let us know whether your issue got resolved?

5

After your click command, you might want to insert a “waitFor” command to ensure your desired message has time to be displayed. As an example, you could try something like below:

WebUI.click(findTestObject('Object Repository/Btn_Submit'))

// WebUI.waitForPageLoad(10)

WebUI.waitForElementPresent(findTestObject('Object Repository/Success_Message'),10)

WebUI.verifyElementPresent(findTestObject('Object Repository/Success_Message'), 5)
6

You need to change two things: how you handle the “Success” check and how you monitor for “Error” states.

  1. Stop using OPTIONAL: Change your failure handling to STOP_ON_FAILURE for critical checks.

  2. Add Negative Validation: Explicitly check that an error message does not exist, or that the specific success criteria must be met.

7

This work but i already coded alot how to fix all at once

8

I removed the Failure Optional as well still didnt work

9

image
image1277×843 124 KB

Follow this