callTestCase allways returns null if it fails, even If exists a return sentence

Hello.

I´ve some shared test cases that return data. Some days ago I´ve realized that, if the test fails, it is returning null instead of the value I give back.

For example:

--log string--

KeywordUtil.logInfo("Received: ${text}")

def size = text.length()

TestNGKW.verifyGreaterThan(size, 1, FailureHandling.CONTINUE_ON_FAILURE)

KeywordUtil.logInfo("size: ${size}")

return "${size}"

--Main test--

def returned = WebUI.callTestCase(findTestCase('Work In Progress/log string'), [('text') : text], FailureHandling.CONTINUE_ON_FAILURE)

KeywordUtil.logInfo("Returned: ${returned}")

-----------

If ‘text’ is ‘abc’ I see ‘Returned: 3’ in the log. But if I use ‘a’ as input text, the callTestCase is returning null instead of ‘1’ as I expected. Of course, inside the ‘log string’ test the size is showing right.

Can anyone help me? Is this a behaviour change of callTestCase return?

Thanks in advance and greetings,

Julian

Hi @sistemas.aquanima

When you supply a as an input, then do you observe in the log size: 1 get printed out ? You mentioned that this could be a behavior change, does this mean on a previous version you didn’t have this problem, if so what version were you using ?

I tried to reproduce your case on my PC.
With text = ‘a’, the callTestCase returned 1 instead of null.
I could not reproduce your case.
StepFailedException was thrown, and the callTestCase returned null.

As @ThanhTo explained below, this is known now behavior.

@sistemas.aquanima

I noticed you posted this to the category “Katalon Recorder(Browser Extensions)”. But you seem to be working on Katalon Stdudio API. Which is right?

Yes. You are right. I´m sorry it´s my first post here

When I supply ‘a’ I see the size logged in the called test case but not in the returned var of the main test.

I have some test that sometimes, due performance requirements, are marked as failed but the functionality works. Those test are failing now. I´ve upgraded to the latest version (7.8.0) but still fails.

It´s failing for me in a Katalon-studio 7.8.0 (Windows version).

I´ve uploaded the log of my execution: null_return_in_CallTestCase.log (34.5 KB)

Maybe I´m doing something wrong. It would be great if you can lend me a hand.

Thanks to all

Summary

This text will be hidden

@sistemas.aquanima

I have reproduced what you’re observing.

Since your child test case fails, Call Test Case keyword will throw a StepFailedException to the test case that calls it. Only when the test case is successfully executed will the value be returned. In this case, please change the failure handling of the verify statement to OPTIONAL.

CONTINUE_ON_FAILURE will indeed carry on the execution, but that test step will be marked as failed, while OPTIONAL will mark that test step as warning. A test case’s status is determined by the status of its final test step, so if the final test step is not failed or error, then the test case will be considered as passed.

Hopefully it helps !

You are right. If I mark the verification as optional, the return sentence works as expected. Of course, the test is marked as pass and I need a fail :cry: (it´s part of the continous development service).

If there is no way to avoid this behaviour, I think I´m going to need to adapt my tests.

Thank you very much for all your quick responses.

Hi @sistemas.aquanima

Unfortunately I think there’s no way to change this behavior without changing a lot of design. If you assign the value to a global variable prior to the verification statement, you can bypass using the value returned from calling the test case, thus making the test fails. Then you can check the value in the calling test case.

Hope that helps.

Yes. I think that is what I’ll have to do.

Thanks

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.