FailureHandling Change breaks legacy code (was STOP_ON_FAILURE does not stop a test when verifyElementPresent returns false)

[Russ_Thomas]

@ThanhTo

@ThanhTo

While I agree that the expectation regarding the Failure Handling, in the context of this topic, would usually be about handling the case where verifyElementPresent determines that element is not present, it is not a “failure” per se, but more of a valid piece of information to base your test execution further (deciding what to do if element is not present).

I 100% agree. Let me be more clear (hey, I got some sleep! ) The problem was there before and still exists now. Here it is, stated less academically,

FailureHandling is a mickie-mouse way to plugin users’ guidance as to what katalon should do with errors (and/or exceptions). But…

It should never have been there.

I don’t see any real change. You switched one old buggy behavior for another new buggy behavior.

@ThanhTo

Nevertheless, this decision is entirely up to the users.

And always should have been and always should be. My guess: trying to support the Manual view presentation of code somehow invited approaches like FailureHandling.

@ThanhTo

Of course, the behavior switch from the old version to the new version may cause frictions because with more power to make decisions come more responsibility (to add alert/custom keywords), but I think returning a boolean indicating the state of presence/existence for that element is a good balance between the use cases.

Ignoring the fact that the method was considered buggy before and now is considered fixed (or, at least better) that is how the method always worked. I don’t see how that can be used as an argument for “why this happened”. Seriously. It was broken before (it didn’t honor a false return), and now it doesn’t support STOP_ON_FAILURE. Both are bad. Both are broken. -> WebUI2 !!!

Here’s my take, based on what I said before: You removed the tension between the boolean result and FH by effectively ignoring* FH and honoring the boolean result regardless of value. I t could be argued that you are choosing the better bad over a worse bad. I think I could be persuaded of that.

* Certainly in terms of STOP_ON _FAILURE, I did not research/evaluate the other possibilities.

Now, hopefully to get us back on the same page:

1. The documentation is wrong.

It now seems to be offering a poor-man’s entry into the try-catch nature of Katalon’s exception handling unrelated to the purpose of the method - which is a bad way to implement error handling. Not only that, it doesn’t really work. The assert(s) you (Thanh) mentioned are way, way better than this.

2. The project setting, Default FH for Test Step is broken (or has no meaning/purpose in the context of this API at least)

3. Users would be better advised to treat this API as an IF-constuct and ignore FH.

I’d be remiss for not mentioning the impact on Web recorder/Manual view, but I don’t use them so I really can’t comment. But yes, someone needs to ensure the output of the tool reflects well the intentions of the API(s) and the best advice given here on the forum.

---

@Ibus, @oyvind.mo WebUI2, and see Fix Poorly Named/Broken APIs

I don’t suppose @anbanbanb had any idea the can of worms he’d be opening up when he reported his problem but hey, when the subject matter matters, so be it