FailureHandling Change breaks legacy code (was STOP_ON_FAILURE does not stop a test when verifyElementPresent returns false)

Feedback & Reviews Bugs Report
,
6

@devalex88 @ThanhTo I think this may be a bug. As you’re aware, I don’t run tests like this, so treat this report with due distrust :wink:

Tested in Katalon 7.0.8

@anbanbanb Okay, here’s my Project Setting:

image

Here is my script (wam is just my way of writing comments - safe to ignore).

    wam 'Nav to Katalon "About us" page'
    WebUI.navigateToUrl("https://www.katalon.com/about-us/")
  
    WebUI.waitForPageLoad(10)
  
    wam 'Check some links are accessible'
    WebUI.verifyLinksAccessible(["https://www.katalon.com/resources-center/blog/", "https://www.katalon.com/partners-and-contributors"])
    
    wam 'Create a failing test object using makeTO'
    TestObject to = makeTO("input.russ")
    
    wam 'Try to verify the fake test object - THIS SHOULD FAIL!'
    WebUI.verifyElementPresent(to, 5)
    
    wam 'Test should NOT reach this line!'
    WebUI.click(to)

The test reaches the last line but should not.

This also does not stop the test:

    wam 'Try to verify the fake test object - THIS SHOULD FAIL!'
    WebUI.verifyElementPresent(to, 5, FailureHandling.STOP_ON_FAILURE)

This works but should not be necessary:

    wam 'Try to verify the fake test object - THIS SHOULD FAIL!'
    if(!WebUI.verifyElementPresent(to, 5, FailureHandling.STOP_ON_FAILURE)) {
      KeywordUtil.markFailedAndStop("Element not present")
    }