Failed test case reusing window when test suite is run?

I am seeing pretty consistently that when a test case in a suite is run and fails, subsequent test cases appear to be using the same window. This is messing up future test cases. Is there a setting that needs to be defined for this not to occur?

Im under the impression that a test case should always start a new instance of its driver, so you shouldnt be seeing such behaviour. If you are however you could always add

WebUI.closeBrowser()

to the end of each test case.

Hi there! I have the following at the end of each test case:

// Close Browser
WebUI.closeBrowser()

But when of the sub-test cases fails, instead of exiting the browser and reporting failure, it keeps it opened and continues to the next test in the suite.

I also tried changing the setting to exit webdrivers after each test case but am not seeing a difference. :thinking:

I misread your first post, using

WebUI.closeBrowser()

wont make a different becuase if it fails it wont even reach this step. Is it marking the test as a fail? Ive never experienced such behaviour before.

Im going to move this to a bug report. For devs to take a closer look could you provide what version of KS your using please :slight_smile:

It is marking the test case (and subsequent ones) as fail. Exactly - it’s not hitting WebUI.closebrowser in the “parent” test case. I am using version 6.2.2 build 4. Please let me know if there any further information you need!

Tread carefully, Harry. Look at this more closely…

Morgan, are you using callTestCase? If so…

Are you using try/catch?

1 Like

I am using callTestCase but no try/catch here.

Snippet of the test case that is linked in the suite…

// Execute Add to List Test Case and Verify Spends
WebUI.callTestCase(findTestCase('Common/Add to List Increments Spends (divSponsored, btnAdd, btnRemove, listAddState, action, view)'), ['divSponsored': divSponsored, 'btnAdd': btnAddList, 'btnRemove': btnRemoveList, 'listAddState': listAddState, 'action': action, 'view': view], FailureHandling.STOP_ON_FAILURE)

// Execute Remove from List Test Case (note we don't verify spends here)
WebUI.callTestCase(findTestCase('Common/Remove From List (btnAdd, btnRemove)'), ['btnRemove': btnRemoveList, 'btnAdd': btnAddList, 'listAddState': listAddState], FailureHandling.STOP_ON_FAILURE)

// Close Browser
WebUI.closeBrowser()

The test case that failed here is the Remove from List test case - all that’s in that one is…

// Click Remove From List
WebUI.click(btnRemove)
WebUI.waitForElementPresent(btnAdd, 10, FailureHandling.STOP_ON_FAILURE)

The .waitForElemenntPresent fails (legit in this case) but the window stays open and the suite then tries to run the next test case - that DOES start with a try / catch to open the browser window BUT shouldn’t it have closed the window since the test case was “completed”?

… or perhaps the FailureHandling.STOP_ON_FAILURE needs to be FailureHandling.CONTINUE_ON_FAILURE… ? :thinking:

It’s behaving bit different differently than I’d initially been thinking it functioned in terms of test suite execution.

@Morgan put the closeBrowser stuff in a listener :wink:

1 Like

As I recall, listeners aren’t reliable when executing a test case under callTestcase.

I agree. I think we’ll leave this under bug reports for now. I’m not able to set up a reasonable test bed to work this through (time constraints). I’m hoping @devalex88/@ThanhTo can throw some light on it.

Meanwhile…

try {
  callTestcase(...)  
} catch(Exception e) {
  closeBrowser() // perhaps?
  // throw e // perhaps?
}

I’ll also give a shout out to @kazurayam - he’s done some work in this area, I believe (but he’s likely not around for a good few hours)

1 Like

in this case, is time for a feature request :slight_smile:

I think I did that. Details of my issue are sketchy (poor memory) but I distinctly recall finding a notable difference in listener behavior under callTestCase.

This post details my TC structure. I’m wondering if it might help? I’m basically seizing control of the exit from each TC and deciding myself what is to be done with it.

1 Like

Thanks Russ! I hate to have to work around whatever it’s doing (just time, you get it!) but that looks like it might be a good thing to put in place at some point.

Worth noting, I tested this back to 6.1.5 and it was still behaving in the manner described above. :thinking:

@Morgan
that’s the fun part of (de)bugging.
you can always use the groovy+spock+geb aproach, but without bugs there is no joy in comunity :slight_smile:

You mentioned this before, I think. Post a Tips&Tricks article.

won’t be fair for katalon comunity to do it in tips and tricks
but i will clear my mind and … if i have time i will post some other approaches in the lounge section

My previous work about try { WebUI.callTestCase(...) } catch { ... } is here:

… I am awake in St. Peterburg in Russia.

Теперь белые ночь. White night now.

2 Likes

If you need to clean up whenever a test case has passed or failed please use the TearDown method.

import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI

WebUI.openBrowser('https://google.com')

assert 1 == 0

@com.kms.katalon.core.annotation.TearDown
def tearDown() {
	WebUI.closeBrowser()
}