Is there any way to perform a test stop other than calling STOP ON FAILURE?

prosmartfony · September 9, 2023, 6:10pm

Hello, community! I see three options - Stop on failure, Optional and Continue on failure. I need a fourth option, something like - end the test as “completed” or “passed”.

I will describe in detail with an example.

I have 4 pages with clients.
There are 10 clients on each page.
4x10=40 clients on 4 pages.
I suddenly have a 41st client on the fifth page.

So, there are two problems:

My script is written in such a way that it considers 10 pages and 10 customers on each page. In the described situation, I get the fifth page opened and the first client clicked to write him an email, but then the script clicks the second client on the same fifth page to write him an email.

But I have 41 clients, not 42. So the test case ends up failing. But I don’t need the failure because the list of customers is growing in the future. I put STOP ON FAILURE so it doesn’t go on continuing all the steps up to page 10, but that’s not the best solution. I get the result I want, but in the form of an error, which will prevent me from analysing the real errors if there are any.

Next, if I have exactly 40 clients, all of whom were processed by the script, the script opens the fifth page, which does not exist on the site. This site does not have 404 errors, so the link to the fifth page I get to the first page (but link in browser still shows as it’s page=5) and all clients are processed on the second round, until the total of 10 pages will not be processed. How do I deal with this?

I made a script like this:

def myList = []

WebUI.navigateToUrl('...page=1...')

WebUI.click('findTestObject...Client (1)'), FailureHandling.STOP_ON_FAILURE)

myList << WebUI.getUrl()

println(myList)

/// Pages 2...3...4....

WebUI.navigateToUrl('...page=5...')

WebUI.click('findTestObject...Client (1)'), FailureHandling.STOP_ON_FAILURE)

if (WebUI.getUrl() in myList) {
    println (Test Passed) (FailureHandling.STOP_ON_FAILURE)
} else {
    WebUI.callTestCase(findTestCase('Branches/Favourites 2'), [:], FailureHandling.CONTINUE_ON_FAILURE)
}

WebUI.navigateToUrl('...page=5...')

WebUI.click('findTestObject...client (2)'), FailureHandling.STOP_ON_FAILURE)

In this example, the first client on the first page is added to the list. If the script hits the first page a second time (instead of the fifth page, which does not exist) and selects the first client that has already been processed previously, execution stops. (BUT WITH FAILURE).

If there are no 10 clients on the page, the script selects the ones that exist and then stops with an error. (BUT WITH FAILURE)

I don’t know any other way to stop execution. Everything is done successfully, but I don’t need to continue all the steps further if there are no more clients on the pages. And there are more pages on purpose, because profiles are different and one profile may have 20 clients, another 40, another 80 and so on.

Any ideas?

Dave_Evers · September 9, 2023, 10:16pm

Hi @prosmartfony, something like this might help you: KeywordUtil (Katalon Studio API Specification) Search the Katalon Studio forum for useage.

kazurayam · September 9, 2023, 10:54pm

A silly example how to use KeywordUtil

import com.kms.katalon.core.util.KeywordUtil

int v = 4   // try 0, 1, 2, 3, 4, ...

if (v % 4 == 0) {
	KeywordUtil.logInfo("So so")
	KeywordUtil.markPassed("Take it easy")
} else if (v % 4 == 1) {
	KeywordUtil.markWarning("I warned you not to do this")
} else if (v % 4 == 2) {
	KeywordUtil.markFailed("You got a failure")
} else if (v % 4 == 3) {
	KeywordUtil.markFailedAndStop("More serious failure")
} else {
	KeywordUtil.markError("You got an error")
}

Russ_Thomas · September 10, 2023, 2:22am

@prosmartfony I won’t address your actual question, Dave and Kaz have you covered. However, you may want to play around with these, just for fun…

Add this to your utilities class:

  static void FORCESTOP(String msg = "") {
    throw new StepErrorException("FORCESTOP" + (msg ? ": " + msg : ""))
  // Or...
    throw new StepFailedException("FORCESTOP" + (msg ? ": " + msg : ""))
  }

Or, you can drop this line directly in your TC script:

    throw new Exception("I did something really bad")

prosmartfony · September 10, 2023, 10:48pm

@kazurayam @Russ_Thomas @Dave_Evers Thank you so much for your tips ))

Actually this utility doesn’t solve my problem, because I need to end the test case with success and force stop it.

@kazurayam’s solution might work for me if after the test case doesn’t find the third client in the list, it doesn’t look for the fourth, fifth and so on. If it would just be if/else, then yes, I could get a result. But to do that I would have to break my test case into many others and call each one individually, which is impossible because I also have a “MyList” that can’t read data from the called test cases. In this version of the runtime I get the test passed, but then the other steps that are specified on the outside statement are executed and I get the same error as I had again.

The @Russ_Thomas solution stops the test but does so with an error. Essentially the same thing I did, but in a different way. Maybe you just wanted to show me an example and I didn’t fully understand what you meant, but I’ve been sitting for a day and still haven’t figured out how to make this utility turn the end of my test at the right moment into a green bar instead of a red one.

But, today, after taking a coffee in a coffee shop, walking along the streets of the city, stroking some passing cute cats and all the while thinking about my problem, one thought came to my mind - what happens if I just return null if the third client is not on my page? Would my test case be successful?

When I got home, I wrote this little script:

if (WebUI.verifyElementNotPresent(findTestObject('DS Elements/View his Messages button/View his Messages (3)'), 5, FailureHandling.OPTIONAL)) {
	return
}

WebUI.click(findTestObject('DS Elements/View his Messages button/View his Messages (3)'))

I got a message like this:

Eventually after return test case the rest of the steps are not executed and the test ended with success! And if instead of the third client I substitute the first or the second client that is on the page, the action is executed and the clients receive emails as specified in the test case.

Thank you for guiding me with your decision )) you at least nudged me in the right direction for thinking ))

P.S. Although my solution worked, it looks very primitive. It’s strange why Katalon Studio doesn’t have some command to stop a test case with success if one or another element is not on the page and it’s ok if there are two or three clients, but maybe five or ten. It would just add along with STOP ON FAILURE something like STOP ON PASSED. That way people can stop the execution of their test cases whenever they want because the information on their pages is very variable.

Or imagine that here in Katalon Studio you have an Excel file and you upload it to Test Data, and there you have email addresses in each line. If the user simply wants to periodically add addresses to this table, replenishing it, then in the test case he will also have to specify “take row 20,” although yesterday there were still 19 rows. But he could immediately create a test case that pulls out 100 or 200 client addresses and work with the data that he managed to extract. But if there are only 20 such rows in your table, then this will cause an error, although it is quite normal that the user simply specified as many as possible in advance so as not to edit the test case every time.

Perhaps it’s just that Katalon Studio is intended for web testing, and not for production purposes as I use it. After all, I use this to perform routine tasks at my job, and not to develop or test the sites I work with. Perhaps Katalon Studio and I have not yet shared our preferences with each other

Thanks again!

kazurayam · September 10, 2023, 11:19pm

I would recommend you to do this.

kazurayam · September 10, 2023, 11:28pm

Adding another non-local exits will make your code a heap of spaghetti, I am afraid.

Russ_Thomas · September 11, 2023, 2:04am

@prosmartfony I totally agree with @kazurayam’s points. Do NOT put messy stuff like this in your published test code. It’s a good thing to play around with experiments, and all the fun stuff you’ve seen here, but production tests should be as clear and clean and easy to read as possible.

Don’t say we didn’t warn you.