Wait issues when run via command line / Pipeline

marianocesari · October 9, 2023, 1:55pm

Hi community!
During these days I have been setting up a pipeline in order to run our test cases in azure devops. After finish with forementioned activity, I realized that when I run our automated test cases via command line, practically all are failing, but if I do same action via Katalon Studio IDE, all TC run properly. After digging more over this topic, I figured out that via command line TC run more faster than via IDE and in addition to it, apparently smart wait feature is not enabled to run via command line. So, in this context I did the following to overcome these issues:

1- Stablishing a hardcoded delay between actions of 2 secs (Via IDE configurations)
2- Defining waits and delays that must be performed before the actions that are failing. For example:

*WebUI.delay(10)*

*WebUI.waitForElementPresent(findTestObject('Object Repository/Web/Transf/Page_Transferencias en el pas/input_PIN_ctl00MainContentctl03ctl10PinTextBox'), *
*    10)*

*WebUI.waitForElementVisible(findTestObject('Object Repository/Web/Transf/Page_Transferencias en el pas/input_PIN_ctl00MainContentctl03ctl10PinTextBox'), *
*    10)*

*WebUI.waitForElementClickable(findTestObject('Object Repository/Web/Transf/Page_Transferencias en el pas/input_PIN_ctl00MainContentctl03ctl10PinTextBox'), *
*    10)*

After made those changes test cases are running properly via command line, but it takes so much time to finish. So, in this situation it came to my mind following questions:

1- Am I following the best approach to overcome this situation?
2- Is it possible to enable smart wait feature to executions via command line?
3- Is it possible to define this delay between actions directly as a parameter of command line execution?. I am asking this because my idea is to only consider this delay for command line executions and not for IDE runs (Nowadays I have to change this setup manually)

Thank you in advance!!

Mariano,

atul.rai · October 9, 2023, 5:05pm

On which OS , you are running this?

Brandon_Hein · October 9, 2023, 7:23pm

No. Hard waits are a cardinal sin in the test automation world, and your scenario perfectly demonstrates why. Depending on the environment you are executing these in, the scripts either fail completely, or take way too long to execute. Don’t use them for anything other than confirming that you have a timing issue. You’ve essentially done this, but now you need to replace them with appropriate wait conditions. You should ultimately never have a WebUI.delay() call in any script you run.

Just a quick note on this bit of code:

WebUI.waitForElementPresent(findTestObject('Object Repository/Web/Transf/Page_Transferencias en el pas/input_PIN_ctl00MainContentctl03ctl10PinTextBox'), 10)
WebUI.waitForElementVisible(findTestObject('Object Repository/Web/Transf/Page_Transferencias en el pas/input_PIN_ctl00MainContentctl03ctl10PinTextBox'), 10)
WebUI.waitForElementClickable(findTestObject('Object Repository/Web/Transf/Page_Transferencias en el pas/input_PIN_ctl00MainContentctl03ctl10PinTextBox'), 10)

You actually don’t need to do all three of these, as WebUI.waitForElementClickable() logically encompasses the previous two. In other words, for an element to be clickable, it logically has to be visible, and for an element to be visible, it logically needs to be present. You should just be able to call

WebUI.waitForElementClickable(findTestObject('Object Repository/Web/Transf/Page_Transferencias en el pas/input_PIN_ctl00MainContentctl03ctl10PinTextBox'), 10)

… and call it a day. If that single wait condition is not waiting the appropriate amount of time for your page to be ready for further interaction, we need to investigate the scenario further. If you can share more info on what action(s) you are taking before calling this (i.e. are you clicking a button or a link?), as well as how the page is refreshing after this action, we can probably help you implement an appropriate wait condition.

I’m not sure 100% on this, but it looks like you can toggle smart wait on/off using keywords in your script:

WebUI.enableSmartWait()
WebUI.disableSmartWait()

Check out this page for more: https://docs.katalon.com/docs/create-tests/record-and-spy/webui-record-and-spy-utilities/smart-wait-function

As per my answer to question 1, you shouldn’t need to define a hard-coded delay if you have the appropriate wait conditions implemented. You can set a default timeout for these wait conditions (i.e. wait up to a maximum of x number of seconds), which may be helpful for you, but I’d say if your page takes more than 10 seconds to load elements, you’ve got bigger problems that need to be addressed in your AUT or in the environment it’s running in.

Hopefully this helps. Again if you can share details on what you’re trying to do, we can probably help you further. If the AUT is a public website, even better, we could get you an exact answer.

Cheers!

marianocesari · October 10, 2023, 12:31pm

Hi!, Thank you so much for your detailed response!!! It is really helpful for me. Based on your comments I did some changes over the use of waits in my script and I removed hardcoded delay between actions as well. Regardless those changes I still facing, some wait issues but probably as you said main problem is tied to performance problems of the web that I am testing. For example, here:

WebUI.waitForElementClickable(findTestObject('Web/Adelanto en Efectivo/Txt_pin_adelanto'), 20)

WebUI.sendKeys(findTestObject('Web/Adelanto en Efectivo/Txt_pin_adelanto'), '1412')

In this case I am still getting below error:

10/10/2023 09:03:25 -03:00 - [TEST_STEP][FAILED] - sendKeys(findTestObject("Web/Adelanto en Efectivo/Txt_pin_adelanto"), "1412"): Unable to send keys '1412' to object 'Object Repository/Web/Adelanto en Efectivo/Txt_pin_adelanto' (Root cause: com.kms.katalon.core.exception.StepFailedException: Unable to send keys '1412' to object 'Object Repository/Web/Adelanto en Efectivo/Txt_pin_adelanto'

In addition to this, when I check the log of command run, I see that is not taking the number of seconds defined as timeout to interact with the element:

But if I check the screenshot the element is not clickeable yet:

Thanks!!

Mariano,

Brandon_Hein · October 10, 2023, 4:46pm

Thank you for sharing this, it helps clarify some things.

Looking at your screenshot, I think I see why the waitForElementClickable() isn’t waiting properly. In Selenium, which is the underlying technology behind most of the WebUI keywords, there is a certain definition of what “clickable” means. To you, the visual user of the application, it’s clear that you cannot click that element yet, but to Selenium it’s perfectly clickable.

However, I’ve noticed that you have an “overlay” on your screen while the page is loading. I.e. those three red dots, and the rest of the page is “greyed out”:

This is an absolute godsend in the automation world, because it makes waiting for your page to load extremely easy. Once we get this problem fixed for you, make sure to buy your front-end guys a coffee

If you can locate this element in the HTML, create a test object for it, then you can wait for it to not be visible:

WebUI.waitForElementNotVisible(findTestObject('path/to/object'), 10)

Finding the element for this might be tricky, as it will disappear once the page is loaded. Here are a couple tricks to try and find it:
1.) Try clicking on the three dots while the recorder is running. It should give you a working test object.
2.) If #1 doesn’t work, then you can use a trick to “debug” your web page while it loads, allowing you to watch elements render in 1 by 1. You can follow my guide on this here: How to Watch Page Rendering, Step-by-Step

This wait condition is actually very powerful, and you can/should be using it whenever this overlay appears in your webpage after you take actions.

Give this a try!