When loop is used, test case does not fail at the correct place

[Marek_Melocik]

Hello,

very simple situation, but unexpected result (test case setting is stop on failure).

KeywordUtil.logInfo("Before")

for(int i = 0; i < 5; i++) {
	if(i==2) {
		KeywordUtil.markFailed("This iteration failed.")
	} else {
		KeywordUtil.markPassed("This iteration passed.")
	}
}

KeywordUtil.logInfo("After")

When you loop through XY objects, it is really hard to determine which one has failed. There is no indication, markFailed keyword is logged, but at DEBUG level, the same as tons of other messages.

Moreover, when you have multiple failures in your loop, e.g.

if(i==1 || i==2 || i==3) {
		KeywordUtil.markFailed("This iteration failed.")
	}

You get only one failed message and you can spend hours reading your logs to find where the issue is. Please consider this fix as this behavior really annoying. Thanks.

[Russ_Thomas]

Hey Marek

Agreed, it’s as if the code surrounding the TC (i.e. Katalon wrapping code) errs on the side passing whereas it should be erring on the side of failure. Because of that, I tend to write my traps something like this:

    for(int i = 0; i < 5; i++) {
      if(i==2) {
        KeywordUtil.markFailed("This iteration failed: (" + i + ")")
        throw new StepFailedException("This iteration failed: (" + i + ")")
      } else {
        KeywordUtil.markPassed("This iteration passed.")
      }
    }

But really, with a default execution status of STOP ON FAILURE, this shouldn’t be necessary.

[devalex88]

Thank you all. What is the quickest thing we can do to remedy your situation? A proper improvement needs plan but we can make simple tweak in the next version (which will be mostly about Appium and iOS 12).

[Marek_Melocik]

KeywordUtil.markFailedAndStop() works fine. So this is temporary workaround for this.

[Russ_Thomas]

It’s hard to say for sure Alex. As you know, I’ve modified the way a TC is wrapped with my own try-catch block and from there I’m able to generate my own report mechanism.

If Marek’s code is more conventional in terms of how his TCs are constructed, anything I suggest is likely to only benefit me.

That said, it seems obvious that STOP ON FAILURE should do exactly what it says. It also seems as though markFailed should be following the setting for STOP ON FAILURE and if found, throwing a StepFailed exception.

Of course, you have the added worry of backwards compatibility. Legacy code (old tests) may suffer.

Not an easy challenge!

[innayaimran786]

Hi, @Russ_Thomas, @devalex88, @Marek_Melocik,
I want to print the iteration number in the log viewer but KeywordUtil.markPassed(“This iteration passed: (” + i + “)”). it just prints this line as it is, but I want iteration number instead of “i” in the log viewer.

[kazurayam]

Please share your Test Case script fragment that shows the loop containing the line of KeywordUtil.markPassed(“This iteration passed: (” + i + “)”).

Also please use the code formatting syntax of Markdown to represent the program code readable:

[kazurayam]

In Groovy, List has eachWithIndex() method:
See

• http://docs.groovy-lang.org/latest/html/groovy-jdk/java/util/List.html#eachWithIndex(groovy.lang.Closure)

def list = [a, b, c, d, e]
 
// eachWithIndex can be used with closure: first parameter is value, second is index.
result = ''
list.eachWithIndex { value, index -> result += "$index:$value, " }
assert '0:a, 1:b, 2:c, 3:d, 4:e,' == result

[innayaimran786]

Hi, @kazurayam. Thanks for your reply.
here is the screenshot of the script.

[kazurayam]

@innayaimran786

Currently you have:

for (int i = 0; i < 100; i++) {
    Mobile.markPassed("This iteration passed: (" + i + ")");
}

You just want to print the value of i (0, 1, 2, …) in the log view. Then the markPassed keyword is not appropriate for your intention.

Why not you change this to:

for (int i = 0; i < 100; i++) {
    Mobile.comment("This iteration passed: (" + i + ")");
}

[innayaimran786]

I tried " Mobile.comment(“This iteration passed: (” + i + “)”); " but its also not printed the value of i. its showing me the same result as before.

[kazurayam]

How about this?

import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI

for (int i = 0; i < 100; i++) {
    WebUI.comment("This iteration passed: (" + i + ")");
}

Or

import com.kms.katalon.core.util.KeywordUtil

for (int i = 0; i < 100; i++) {
    KeywordUtil.logInfo("This iteration passed: (" + i + ")")
}

[innayaimran786]

I tried both " WebUI.comment " and " KeywordUtil.logInfo " but getting same result as before, not getting the value of i.

[kazurayam]

I can see the value of i as follows.

I do not see why you cannot.