Katalon suite taking 30 seconds between steps and I can't figure out why

Keep going. I’m now hooked waiting for the next “episode” :nerd_face: :laughing:

@kevin.jackey

Do you have “Smart Wait” enabled?

If yes, try disabling it.

Let me show you a running code that demonstrates how “Smart Wait” could force your test case to take unexpectedly long time.

Here is a test case:

import org.apache.commons.lang3.time.DurationFormatUtils
import org.apache.commons.lang3.time.StopWatch
import com.kms.katalon.core.testobject.ConditionType
import com.kms.katalon.core.testobject.TestObject
import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI

TestObject makeTestObject(String xpathTemplate, Map<String, String> variables) {
	def groovyscript = '"""' + xpathTemplate + '"""'
	def shell = new GroovyShell(new Binding(variables))
	String expr = shell.evaluate(groovyscript)
	TestObject tObj = new TestObject(expr)
	tObj.addProperty("xpath", ConditionType.EQUALS, expr)
	return tObj
}

StopWatch sp = new StopWatch()
sp.start()

WebUI.openBrowser('https://www.katalon.com/')
WebUI.verifyElementPresent(makeTestObject('//a[contains(.,"Start free trial")]', [:]), 10)
WebUI.closeBrowser()

sp.stop()
println DurationFormatUtils.formatDurationHMS(sp.getTime())

If you run it with “Smart Wait” disabled, you will see

2023-11-17 09:11:00.659 INFO  c.k.katalon.core.main.TestCaseExecutor   - --------------------
2023-11-17 09:11:00.660 INFO  c.k.katalon.core.main.TestCaseExecutor   - START Test Cases/visitKatalonDotComWithSmartWait
2023-11-17 09:11:01.032 INFO  
...
00:00:08.883
2023-11-17 09:11:09.807 INFO  c.k.katalon.core.main.TestCaseExecutor   - END Test Cases/visitKatalonDotComWithSmartWait

It took less than 9 seconds. It is normal.

Now you want to try enable Smart Wait and run the same test. Occationally you would see somthing like …

00:00:29.192

But sometimes you may see the test finishes just as nomal. Or, it may take over 1 minutes. The duration varies occationally. I guess the behavior of Smart Wait depends on how the target URL http://katalon.com/ behaves internally (if any AJAX events continue to flow, when it stops); and I have no idea about that URL.

I wondered if @kevin.jackey is aware of the Smart Wait. So I thought I should let him know.

If you want to know more about how Smart Wait is implemented, please refer to Stale element not found, is this relate to using same Object? - #95 by Brandon_Hein

1 Like

Aha! I do! Will disable it and see what happens

UPDATE: Not only did it resolve the issue in this case, but it significantly sped up the execution of all scripts, thank you!!

But now they’re running TOO fast, and not waiting for some pages to load. Is there a way to enable / disable Smart Wait programmatically?

1 Like

The doc says yes

https://docs.katalon.com/docs/create-tests/record-and-spy/webui-record-and-spy-utilities/smart-wait-function#temporarily-turn-off-smart-wait

but, as far as I examined, the WebUI.enableSmartWait() and WebUI.disableSmartWait() does NOT work as I expect.

@vu.tran @Elly_Tran

As you know, you can make sure a page is completely loaded by writing statements like “WebUI.waitForElementPresent(TestObject, int)”. If you are diligent enough, you would not need Smart Wait in most cases.

In some critical cases, a JavaScript-based keyword is effectie: like

True, but I just finished reading the exhaustive thread you linked to and it sounds like the programmatical solutions don’t work nearly as effectively either :slight_smile:

Since it’s just for this one page it sounds like I may just need to grin and bear it.

Have you (or anyone that you are aware of) submitted a bug report for the enable and disable functions not working? If not I’ll give them a try and (if they fail to work as you imply) submit that myself

What I haven’t had time to do yet is figure out why, once triggered, the smart wait causes EVERY subsequent step to have the 30 second wait, one would think that after the initial wait step the remaining items would flow smoothly, but for some reason SmartWait thinks the page is STILL loading, even after dev tools shows all page load activity has ceased.

SmartWait seems very effective, but still appears to have some bugs in the system…

No, I haven’t. And I have never seen anyone.

Once I tried to develop a code that reproduce the situation that WebUI.enableSmartWat and WebUI.disalbeSmartWait DOES NOT WORK. But I failed. I could not make a reproducible code.

Well, I have a different oppinion. I know the implementation of SmarWait (wait.js) works perfect to achieve what it is designed to do.

SmartWait is just another way of wait for the target HTML DOM to be settled and stable. But I would point out, SmartWait assumes some naïve assumptions:

  1. AJAX event streams in the target web page will cease in a reasonablly short period. The stream would not last forever.

  2. You, as a Katalon user, are well aware how the AJAX Event streams in the target web page are designed to flow. Therefore you are able to predict how long SmartWait will wait for a particular URL to load.

I think these assumptions often break.

Are you, @kevin.jackey, familiar with the JavaScript code of the target web page? Are you aware when and how the AJAX Events in the page start and stop?

Most probably, the testers don’t know much about it. In other words, the testers would never be able to predict how Smart Wait will/should work for a particular web page because they don’t know how the JavaScript of the page is coded.

For example, I have never studied the https://katalon.com/ to find out what sort of JavaScript codes it has; so that I have no idea what type of AJAX event streams are flowing when I open the URL in a browser. All I could guess is that https://katalon.com/ has one or more AJAX event streams that last 20 - 30 seconds after the page loading. I could observe it because SmartWait actually took 20 - 30 seconds to finish waiting for the https://katalon.com is loaded.

SmartWait may misfire for a web page and it may outburst for another web page. SmartWait is a unpredictable & unreliable weapon. I think that the use of Smart Wait should be carried out with caution. We should be able to try applying SmartWait ON and OFF to a target URL and examine how it goes. I would argue that it is too careless to have Smart Wait enabled as default.

I agree that the assumptions being made are definitely flaky; however the end result seems to work 99% of the time and (as you’ve stated above and in the link you provided) at present there doesn’t appear to be a more effective method :frowning:

Because SmartWait is unreliable weapon, I would request Katalon to make the “Default Smart Wait” to be set “disabled”.

@vu.tran @Elly_Tran @duyluong

The only problem there is (as I have experienced) turning it off leads to even WORSE problems (at least for me), as then the script attempts to execute every step as fast as possible and leaves a horribly long trail of errors on pages that DO have some slow loading elements, it’s a catch-22.

The only other alternative is to set an extremely long element wait time, which also causes other problems…

@kevin.jackey

Are you a paying customer of Katalon? Are you qualified raising an official support request? If so, would you please raise one for this issue, and let them know. And let them work on making SmartWait better (less troublesome). A little discussion in this forum might not be enough for their attention. They seems to be busy for developping something new and attractive…

See my post below…

Hope the below examples of “wait” statements can help. This is what I use and sometimes double up on them, like “visible” and “clickable” or “pageload” and “visible”…

Depends. Are you waiting on XHR-type stuff?

Perhaps a change in wait-philosophy might help. Generally speaking, there are usually “effects” of data changing on a page after known page updates have occurred. It’s usually better to test for those being present than to wait for specific calls to complete.

or, simplified, for the entire page to be ‘stable’ (a problem smart wait attempts to solve).
which, as already shown, in certain cases can be undetermined

In Stale element not found, is this relate to using same Object? - #113 by kazurayam, I reported that the WebUI.enableSmartWait keyword does not work as I expect

Today, I have found out the reason why.

I had a look at the source of com.kms.katalon.core.webui.driver.KatalonSmartEventListener and found:

    private void doSmartWait() {

        boolean localSmartWaitEnabled = (boolean) Optional
                .ofNullable(RunConfiguration.getExecutionProperties().get(RunConfiguration.LOCAL_SMART_WAIT_MODE))
                .orElse(true);

        boolean globalSmartWaitEnabled = (boolean) Optional
                .ofNullable(RunConfiguration.getExecutionProperties().get(RunConfiguration.GLOBAL_SMART_WAIT_MODE))
                .orElse(false);

        if (globalSmartWaitEnabled && localSmartWaitEnabled) {
            SmartWait.doSmartWait();
        }
    }

According to this logic, the “SmartWait” is enabled only when both of the following conditions are satisfied:

  1. In the Katalon GUI, I select Project > Settings > Execution > Web UI > Default Smart Wait is Enabled.

  2. The test case has NOT called WebUI.disableSmartWait(), OR the test case has called WebUI.enableSmartWait() explicitly,

I expected that, when I select Default Smart Wait is Disabled but the test case called WebUI.enableSmartWait(), the SmartWait WILL BE enabled. I expected in my mind:

        if (globalSmartWaitEnabled || localSmartWaitEnabled) {
            SmartWait.doSmartWait();
        }

But no. In fact, that is not the case. I was surprised.

I checked the Katalon’s documentation for Smart Wait. I found a sentence:

Temporarily turn off Smart Wait

Requirement

The Default Smart Wait function is enabled.

Ah! I misunderstood.


I am not happy with the current implementation. I would prefer to have no globalSmartWaitEnabled. I would rather like to have only localSmartWaitEnabled. In other words, get rid of the “Project > Settings > Execution > Web UI > Default Smart Wait” settings, and users should control Smart Wait ON/OFF only by the keyword calls. I mean, I would prefer the following if statement:

        if (localSmartWaitEnabled) {
            SmartWait.doSmartWait();
        }

@kevin.jackey

Katalon would never change anything about the SmartWait settings, as they are obliged to respect the backward compatibility. So more and more people would get confused by the SmartWait in future, just as you were.

You are supposed to accept the current design of Smart Wait. You already know how to manage it.

If you want to use Smart Wait, you want “Project > Settings > Execution > Web UI > Default Smart Wait” to be “Enabled”. And you want to use WebUI.disableSmartWait() and “WebUI.enableSmartWait()” appropriately so that your test case run fine.

Otherwise, you want to stop relying on Smart Wait completely. You want to implement appropriate “Wait” by “WebUI.waitFor***” and other instruments.

I have got an idea.

I would be able to develop custom keywords that modifies the globalSmartWaitEnabled property and the localSmartWaitEnabled property ON and OFF on the fly to override the "Project > Settings > Execution > Web UI > Default Smart Wait” setting.

That is, I believe, what @kevin.jackey wants.

In the Keyword dir, make a package “com.kazurayam.ks” and a class

package com.kazurayam.ks

import com.kms.katalon.core.configuration.RunConfiguration
import com.kms.katalon.core.exception.StepFailedException
import com.kms.katalon.core.model.FailureHandling
import com.kms.katalon.core.webui.keyword.internal.WebUIKeywordMain

public class ToggleSmartWait {
	
	public static void toggleON() throws StepFailedException {
		WebUIKeywordMain.runKeyword({
			RunConfiguration.getExecutionProperties().put(RunConfiguration.GLOBAL_SMART_WAIT_MODE, true)
			RunConfiguration.getExecutionProperties().put(RunConfiguration.LOCAL_SMART_WAIT_MODE, true)
		}, FailureHandling.CONTINUE_ON_FAILURE, true, "Unable to enable smart wait !")	
	}
	
	public static void toggleOFF() throws StepFailedException {
		WebUIKeywordMain.runKeyword({
			RunConfiguration.getExecutionProperties().put(RunConfiguration.LOCAL_SMART_WAIT_MODE, false);
		}, FailureHandling.CONTINUE_ON_FAILURE, true, "Unable to disable smart wait !");
	}
}

source

That’s all we need.

I tested this keyword with a Test Case that I made at Stale element not found, is this relate to using same Object? - #113 by kazurayam with a bit of ammendments:

import com.kms.katalon.core.testobject.ConditionType
import com.kms.katalon.core.testobject.TestObject
import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI
import internal.GlobalVariable
import com.kazurayam.ks.ToggleSmartWait

/**
 * automate signing-in to Microsoft Azure DevOps with Smart Wait
 */

TestObject makeTestObject(String id, String xpathExpression) {
	TestObject tObj = new TestObject(id)
	tObj.addProperty("xpath", ConditionType.EQUALS, xpathExpression)
	return tObj
}

// enable Smart Wait explicitly
ToggleSmartWait.toggleON()

String url = "https://dev.azure.com/${GlobalVariable.ACCOUNT}"
WebUI.openBrowser(url)
WebUI.setViewPortSize(800, 800)

TestObject loginfmt = makeTestObject("loginfmt", "//input[@name='loginfmt']")
WebUI.sendKeys(loginfmt, GlobalVariable.EMAIL)

TestObject nextButton = makeTestObject("Next", "//input[@id='idSIButton9']")
WebUI.click(nextButton)

TestObject passwd = makeTestObject("Passwd", "//input[@name='passwd']")
WebUI.sendKeys(passwd, GlobalVariable.PASSWD)

TestObject signinButton = makeTestObject("SignIn", "//input[@id='idSIButton9']")
WebUI.click(signinButton)

TestObject yesButton = makeTestObject("Yes", "//input[@id='idSIButton9']")
WebUI.click(yesButton)

WebUI.closeBrowser()

ToggleSmartWait.toggleOFF()

source

This test case automates going throuh the Login procedure into Microsoft Azure DevOps, which is deeply AJAX driven so that definitely requires the SmartWait enabled. If I don’t have SmartWait enabled, the test case will fail raising an exception of “Stale Element reference”.

Intentionally I chose "Project > Settings > Execution > Web UI > Default Smart Wait” to be “Disabled” and ran this test case. It worked fine for me.

This proved that the com.kazurayam.ks.ToggleSmartWait class can toggle on and off the Smart Wait even if the "Project > Settings > Execution > Web UI > Default Smart Wait” is set “Disabled”. This is what I wanted to see.

Now I am glad that I could find this workaround. I would be able to help those who got confused by Smart Wait. I would expect a lot more in future.