Katalon Studio 8.6.0 - waitForElementNotPresent - bug - timeout

petr.brezina · March 20, 2023, 8:24am

Hi there,

since I have updated to the new version of Katalon Studio 8.6.0, all test cases fall, because command waitForElementNotPresent is not working properly. If I run the same test case on Katalon Studio 8.5.0, test case passes successfully.

There is a problem that waitForElementNotPresent is constantly waiting even though the element is no longer displayed on the page. Like I said before - on version 8.5.0 the same test case was fine, so I suppose there is no problem in my test case.

The log says: Unable to find the element located by 'By.xpath: //div[@class='Preloader ‘]’. Please recheck the objects properties to make sure the desired element is located.
(console: 2023-03-20 09:20:39.898 INFO c.k.k.c.webui.common.WebUiCommonHelper - Unable to find the element located by 'By.xpath: //div[@class='Preloader ‘]’. Please recheck the objects properties to make sure the desired element is located. )

= but that’s not the function of this command waitForElementNotPresent. This command should not wait if the element is not presented on my page.

Please could you verify that issue?

Thank you very much
PB

Operating System
Windows 10

Katalon Studio version
Version 8.6.0

Environment (for Web Testing)
Chrome 111.0.5563.65

duc.phan · March 20, 2023, 8:50am

Hi Petr

The keyword waitForElementNotPresent would wait until the given element to not present (disappear) within the given time in the second unit. This mean that the object need to be appear first then this keyword would help you to wait until that element disappear. For this could you confirm that in the case this keywords is execute the object //div[@class='Preloader ‘] still present? You could use the waitforelementpresent to test the object first to make sure that it still present at the time to test that object.

petr.brezina · March 20, 2023, 9:19am

Hi duc.phan,

thanks for your reply.

It’s strange, because until version Katalon Studio 8.5.0 we were used to use this command like this:

We were used to use waitForElementNotPresent in the sense that the command checks to see if the element is on the page and if so then waits for the specified amount of time. If not so, test case continues despite the fact that the element was not displayed before. We use this as a check that if there is any loader on the page, wait until it disappears. It worked like this for over a year.

Even now if i run version 8.5.0 it behaves like we used to, but after version 8.6.0 the behaviour of the this command is changed.

I have read all change log of the the new version 8.6.0 and there is no mention of a change in logic in this command.

I have read the desctiption of this command (here: [WebUI] Wait For Element Not Present | Katalon Docs) and I’m still convinced that it’s a bug, that the command should check if the element is presented and it not so… test case should continue without waiting for the elemenet. There is also example: “You want to wait until ‘Make Appointment’ button is NOT present in 20 seconds.” What I understand is that the command waits until the button disappears and waits a maximum of 20 seconds (but that doesn’t mean it will wait the full 20 seconds if the button disappears earlier)

EDIT:
I also verified the option that the element is first displayed on the screen and then disappears, and despite this the waitForElementNotPresent command displays a timeout.

kazurayam · March 20, 2023, 1:26pm

Here is the source code of waitForElementNotPresent keyword of v8.0.5:

github.com

katalon-studio/katalon-studio-testing-framework/blob/master/Include/scripts/groovy/com/kms/katalon/core/webui/keyword/builtin/WaitForElementNotPresentKeyword.groovy

package com.kms.katalon.core.webui.keyword.builtin

import groovy.transform.CompileStatic

import java.text.MessageFormat
import java.util.concurrent.TimeUnit

import org.apache.commons.io.FileUtils
import org.openqa.selenium.Alert
import org.openqa.selenium.By
import org.openqa.selenium.Dimension
import org.openqa.selenium.JavascriptExecutor
import org.openqa.selenium.NoSuchElementException
import org.openqa.selenium.NoSuchWindowException
import org.openqa.selenium.TimeoutException
import org.openqa.selenium.WebDriver
import org.openqa.selenium.WebDriverException
import org.openqa.selenium.WebElement
import org.openqa.selenium.interactions.Actions
import org.openqa.selenium.support.ui.ExpectedCondition

This file has been truncated. show original

I do not think your sentence is right.

As far as I understand from the source code, the waitForElementNotPresent keyword of v8.0.5 is stateless. It does not remember at all if the HTML elment once was present first and then later disappeared.

@duc.phan

Do you mean that the implemementation of waitForElementNotPresent was changed to be stateful at v8.6.0 ?

kazurayam · March 20, 2023, 1:28pm

@duc.phan and Katalon team

If possible, could you show the source code of katalon-studio-testing-framework/WaitForElementNotPresentKeyword.groovy at master · katalon-studio/katalon-studio-testing-framework · GitHub of v8.6.0.

@petr.brezina would have a doubt that the source has been changed at 8.6.0. The only way to answer to him is to let us see the diff of the source code of v8.6.0 and its previous version.

bionel · March 20, 2023, 1:33pm

Another option is, decompile this class after a run with 8.5.X vs 8.6.0 and diff them, without access to the code changelog

kazurayam · March 20, 2023, 1:35pm

I tried to see the source but I couldnt

bionel · March 20, 2023, 1:36pm

well … so now we wait until a Katalon developer jumps in …

kazurayam · March 20, 2023, 1:48pm

On my macbook machine, I have v8.6.0 installed.
I found the jar of the source code:

/Applications/Katalon Studio.app/Contents/Eclipse/configuration/resources/source/com.kms.katalon.core.webui/com.kms.katalon.core.webui-sources.jar

and I cound find the source code of WebUI.waitForElementNotPresent keyword of v8.6.0.

I compared it with the v8.0.5 and found differences.

v8.0.5 had a fragment like this:

v8.6.0 has no pollingEvery(...). It is removed. There are a few more changed lines.

That difference might has caused what @petr.brezina noticed.

But I have no idea what Katalon team meant to do by the change.
If it is right or wrong, appropriate or inappropriate — I don’t know.

But I can say, the source code has been changed at v8.6.0. So it may behave differently.

petr.brezina · March 20, 2023, 2:15pm

Thanks a lot @kazurayam @bionel for your answers.

Should I wait until a developers from Katalon will join to this topic or should I rather report a issue through the official way via Katalon Help Center for checking the behaviour?

For now we have rolled back to 8.5.0.

bionel · March 20, 2023, 2:17pm

if you have a license, rise a ticket with support.
you may wait a century on the comunity on this matter.

kazurayam · March 21, 2023, 3:41am

You, a paying user, should raise a support request. I suppose that Katalon developers are not aware of this issue. I am afraid that they haven’t tested their product enough. There could be more problems to be found. You should send an alert to them.

→ @vu.tran

kazurayam · March 21, 2023, 3:53am

Here I will attach a zip file which contains

v8.0.5, WaitForElementNotPresent.groovy
v8.6.0, the same groovy file
the diff betwee the two

WaitForElementNotPreset_8.0.5_8.6.0_diff.zip (248 Bytes)

The diff info is as follows:

81,98c81,100
<                     if (locator != null) {
<                         logger.logDebug(MessageFormat.format(StringConstants.KW_LOG_INFO_FINDING_WEB_ELEMENT_W_ID, to.getObjectId(), locator.toString(), timeOut))
<                         elementNotFound = new FluentWait<WebDriver>(DriverFactory.getWebDriver())
<                                 .pollingEvery(500, TimeUnit.MILLISECONDS).withTimeout(timeOut, TimeUnit.SECONDS)
<                                 .until(new Function<WebDriver, Boolean>() {
<                                     @Override
<                                     public Boolean apply(WebDriver webDriver) {
<                                         try {
<                                             webDriver.findElement(locator)
<                                             return false
<                                         } catch (NoSuchElementException e) {
<                                             return true
<                                         }
<                                     }
<                                 })
<                     } else {
<                         throw new IllegalArgumentException(MessageFormat.format(StringConstants.KW_EXC_WEB_ELEMENT_W_ID_DOES_NOT_HAVE_SATISFY_PROP, to.getObjectId()))
<                     }
---
> 					if (locator != null) {
> 						logger.logDebug(MessageFormat.format(StringConstants.KW_LOG_INFO_FINDING_WEB_ELEMENT_W_ID, to.getObjectId(), locator.toString(), timeOut))
> 						elementNotFound = new FluentWait<WebDriver>(DriverFactory.getWebDriver()).withTimeout(timeOut, TimeUnit.SECONDS)
> 								.until(new Function<WebDriver, Boolean>() {
> 									@Override
> 									public Boolean apply(WebDriver webDriver) {
> 										try {
> 											List<WebElement> elementList = WebUiCommonHelper.findElementsByDefault(to, timeOut)
> 											if (elementList.size() == 0) {
> 												return true
> 											}
> 											return false
> 										} catch (WebElementNotFoundException e) {
> 											return true
> 										}
> 									}
> 								})
> 					} else {
> 						throw new IllegalArgumentException(MessageFormat.format(StringConstants.KW_EXC_WEB_ELEMENT_W_ID_DOES_NOT_HAVE_SATISFY_PROP, to.getObjectId()))
> 					}
100c102
<                     // timeOut, do nothing
---
> 					// timeOut, do nothing

I can easily see that v8.6.0 changed its behavior: which type of Exception to expect. v8.0.5 expects NoSuchElementException. v8.6.0 expects WebElementNotFoundExcetion. I don’t know the detail how these 2 exceptions differ. I guess, @petr.brezina was trapped by this change.

petr.brezina · March 21, 2023, 8:13am

@kazurayam Thank you very much for the analysis. I reported this issue to Katalon and they corfimed that there is strange behaviour of this command and they will report it to development.

I added also your diff to their ticket, but as a text, because WinRar and 7-Zip displays error during unziping the file (WaitForElementNotPreset_8.0.5_8.6.0_diff.zip)

Thanks a lot for your help.

As soon as Katalon developers write the result of their analysis, I will write the information here for others as well.

Have a nice day.
PB

bionel · March 21, 2023, 9:48am

No idea how that pollingEvery() method is implemented and if it was really needed.
There is a change also in the locating logic.
I don’t understand this changes:

try {
  webDriver.findElement(locator)
  return false

vs:

try {
  List<WebElement> elementList = WebUiCommonHelper.findElementsByDefault(to, timeOut)
  if (elementList.size() == 0) {
  return true
  }
  return false

the issue may came also from WebUiCommonHelper.findElementsByDefault
I suppose this is the reason the catched exception class was changed, but that keyword may missbehave in certain conditions.
Apparently, that logic now allways returns false, at least with the OP case and it keeps waiting forever due to until:
https://www.selenium.dev/selenium/docs/api/java/org/openqa/selenium/support/ui/FluentWait.html#until(java.util.function.Function)

Repeatedly applies this instance's input value to the given function until one of the following occurs:
the function returns neither null nor false
the function throws an unignored exception
the timeout expires
the current thread is interrupted

interresting, the until timeout never expire … most probably a TimeoutException is never raised since the keyword only raise WebElementNotFoundException

h.kramer · March 21, 2023, 7:52pm

Thanks for your input. I am also experiencing issues with the ‘wait for element’. I have replaced it for a delay, but it is not very useful. Hopefully there is an alternative or the element’s function can be restored.

kazurayam · March 22, 2023, 12:12am

There are several keyword named “waitForElementXXXX”.
Which one are you talking about?

I looked at the souce of “WaitForElementPresent” keyword of v8.4.1 and v8.6.0. There is no difference. That keyword hasn’t been changed at v8.6.0. Then how about other keywords? nobody knows yet.

kazurayam · March 22, 2023, 1:27am

I have made a tool (Gradle build) to compare the source files of Katalon Studio versions.

I compared the following 2 jars:

v8.4.1, /Applications/Katalon Studio.app/Contents/Eclipse/configuration/resources/source/com.kms.katalon.core.webui/com.kms.katalon.core.webui-sources.jar
v8.6.0, /Applications/Katalon Studio.app/Contents/Eclipse/configuration/resources/source/com.kms.katalon.core.webui/com.kms.katalon.core.webui-sources.jar

My tool generated the following file:
8.4.1-8.6.0.json (16.8 KB)

This file includes the following list of files. These files have some difference between the v8.4.1 and v8.6.0. Other files in the jar stayed unchanged:

    "modifiedFiles": [
        "com/kms/katalon/core/webui/common/CssLocatorBuilder.java",
        "com/kms/katalon/core/webui/common/WebUiCommonHelper.java",
        "com/kms/katalon/core/webui/constants/StringConstants.java",
        "com/kms/katalon/core/webui/driver/DriverFactory.java",
        "com/kms/katalon/core/webui/keyword/builtin/VerifyElementNotPresentKeyword.groovy",
        "com/kms/katalon/core/webui/keyword/builtin/WaitForElementNotPresentKeyword.groovy",
        "com/kms/katalon/core/webui/util/WebDriverPropertyUtil.java"
    ]

So

VerifyElementNotPresent
WaitForElementNotPresent

I hope Katalon team to look at these 2 keywords.

@vu.tran

linh.to · March 22, 2023, 4:55am

@petr.brezina
You can try this custom keyword.
CustomWebUI.groovy (2.7 KB)

The result

bionel · March 22, 2023, 8:48am

I will add WebUiCommonHelper to the list since it is used under the hood at least by WaitForElementNotPresent.
Too bussy right now to look into what actually changed into others.

@vu.tran