Reproducing Selenium StaleElementReferenceException in Katalon Studio

kazurayam · December 5, 2024, 2:53pm

I have published a GitHub repository:

Achronym “SERE”

I would use an achronym “SERE” (Stale Element Reference Exception) in this article for short.

Problem to solve

In the Katalon Community, there are a lot of topics about SERE:

Please make a search in the forum to look up more

The posters were eager to find out some way to fix/manage/avoid SERE but they failed. Most of these topics are still open (unresolved) today.

I found a common shortcoming in these posts about SERE. The original posters ask for help for fixing (avoiding) SERE in their own projects, but they do not provide any sample code in Katalon Studio that enable you to reproduce the SERE in your hand. With no codes shared, the discussions stayed ambiguous resulting no idea what to do next.

Reference

In order to understand what Stale Element Reference Exception is, I refered to the article:

Baeldung, Selenium StaleElementReferenceException.

Solution

In this project, I would show you several test scripts in Katalon Studio. With these sample scripts, you can firmly reproduce SERE on your machine. The scripts are short. If you read the codes carefully, you would understand how a StaleElementReferenceException is thrown.

Decription

The target page

At first, I created a HTML file as the target of my tests. The file located at <ProjectDir>/docs/targetPage.html.

You can see the HTML in action at:

https://kazurayam.github.io/StaleElementReferenceExceptionReproduction/targetPage.html

When the page opens, it will look like this:

3 seconds after open, the <button id='myButton'> element is silently removed. And the button is recreated soon. The content text and the style is slightly changed, but the id value remains the same: myButton.

The HTML contains the following JavaScript. This changes the DOM dynamically.

    function modifyPage() {
      // remove the <button @id='myButton'>
      document.getElementById('myButton').remove();
      // recreate the <button @id='byButton'>
      let btn = document.createElement('button');
      btn.id = 'myButton';
      btn.textContent = 'This button once was removed and recreated'
      btn.classList = ['recreated'];
      document.getElementById('main').append(btn);
      console.log('modified the page')
    }
    // will modify the page at 3 seconds after the initial loading
    window.addEventListener("load", (event) => {
      console.log('the page was loaded initially');
      const timeout = window.setTimeout(modifyPage, 3000);
      console.log('timeout was set');
    });
  </script>
</body>

Object Repository/myButton

I created a Test Object named myButton, which contains a simple XPath expression:

//button[@id='myButton']

This XPath expression will select the button in the target page. The <button id='myButton'> keeps the id value unchanged before and after the DOM modification by JavaScript. Therefore the same xpath applies.

Test Cases/TC1

See the source of
Test Cases/TC1.

import static com.kms.katalon.core.testobject.ObjectRepository.findTestObject

import java.nio.file.Path
import java.nio.file.Paths

import org.openqa.selenium.WebElement

import com.kms.katalon.core.configuration.RunConfiguration
import com.kms.katalon.core.model.FailureHandling
import com.kms.katalon.core.testobject.TestObject
import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI

/**
 * TC1
 *
 * This script can reproduce a Selenium StaleElementReferenceException (SERE).
 * The target HTML is dynamically modifiedy by JavaScript inside it.
 * An HTML node will be removed and recreated at 3 seconds after the initial page load.
 * A reference to the HTML node as org.openqa.selenium.WebElement object will
 * get stale by the DOM modification by JavaScript.
 *
 * Referring to the stale WebElement object will cause a SERE.
 *
 * @author kazurayam
 */
// identify the location of target HTML file
Path projectDir = Paths.get(RunConfiguration.getProjectDir())
Path html = projectDir.resolve("docs/targetPage.html")
URL htmlURL = html.toFile().toURI().toURL()
String urlString = htmlURL.toExternalForm()
WebUI.comment("navigating to " + urlString)

// open a browser, navigate to the target page
WebUI.openBrowser('')
WebUI.setViewPortSize(800, 600)
WebUI.navigateToUrl(urlString)

TestObject myButtonTestObject = findTestObject("Object Repository/myButton")

// make sure <button id='myButton'> is displayed in the page initially
WebUI.verifyElementPresent(myButtonTestObject, 10, FailureHandling.STOP_ON_FAILURE)

// get the reference to the HTML element <button id='myButton'>
WebElement myButtonWebElement = WebUI.findWebElement(myButtonTestObject, 10, FailureHandling.STOP_ON_FAILURE)

// the test intentionally does nothing for long enough seconds
WebUI.delay(5)

// At 3 seconds after the page load, JavaScript wil remove and recreate the HTML element

try {
    // at 3 secs after the initial page loading,
    // the old <button id='myButton'> is removed, then
    // a new <button id='myButton'> is recreted.
    myButtonWebElement.click()  // this statement will throw a StaleElementReferenceException
} catch (Exception e) {
    WebUI.comment(">>> An Exception was caught: " + e.getClass().getName() + ": " + e.getMessage() + " <<<")
}

//WebUI.closeBrowser()

The TC1 does the following steps:

open browser, navigate to the targetPage.html, prepare a Test Object that selects the <button id='myButton'> element, makes sure the page is opened.
TC1 creates a variable named myButtonWebElement which is type of org.openqa.selenium.WebElement, refers to the <button id='myButton'> element in the HTML DOM.
TC1 intentionally waits for 5 seconds.
On the other hand, in the opened browser, at 3 seconds after the page load, the <button id='myButton'> in blue color is removed. Then a new <button id='myButton'> in grey color is created.
After 5 seconds of wait, TC1 calls myButtonWebElement.click(). At this call, a StaleElementReferenceException will be thrown.

See the console log emited by TC1:

12月 05, 2024 9:52:44 午後 com.kms.katalon.core.logging.KeywordLogger startTest
情報: START Test Cases/TC1
...
12月 05, 2024 9:52:59 午後 com.kms.katalon.core.logging.KeywordLogger logInfo
情報: An Exception was caught: org.openqa.selenium.StaleElementReferenceException: stale element reference: stale element not found in the current frame
  (Session info: chrome=131.0.6778.109)
For documentation on this error, please visit: https://www.selenium.dev/documentation/webdriver/troubleshooting/errors#stale-element-reference-exception
Build info: version: '4.22.0', revision: 'c5f3146703'
System info: os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '14.7.1', java.version: '17.0.7'
Driver info: com.kms.katalon.selenium.driver.CChromeDriver
Command: [ac30658fb877503145f65a505a4902e5, clickElement {id=f.D3000530D4F4A376D08D574B93D847AB.d.2AC594D17950A4F164BD8602A5C7E00D.e.3}]
Capabilities {acceptInsecureCerts: false, browserName: chrome, browserVersion: 131.0.6778.109, chrome: {chromedriverVersion: 131.0.6778.87 (ce31cae94873..., userDataDir: /var/folders/7m/lm7d6nx51kj...}, fedcm:accounts: true, goog:chromeOptions: {debuggerAddress: localhost:63200}, networkConnectionEnabled: false, pageLoadStrategy: normal, platformName: mac, proxy: Proxy(), se:cdp: ws://localhost:63200/devtoo..., se:cdpVersion: 131.0.6778.109, setWindowRect: true, strictFileInteractability: false, timeouts: {implicit: 0, pageLoad: 300000, script: 30000}, unhandledPromptBehavior: ignore, webSocketUrl: ws://localhost:45754/sessio..., webauthn:extension:credBlob: true, webauthn:extension:largeBlob: true, webauthn:extension:minPinLength: true, webauthn:extension:prf: true, webauthn:virtualAuthenticators: true}
Element: [[CChromeDriver: chrome on mac (ac30658fb877503145f65a505a4902e5)] -> xpath: //button[@id='myButton']]
Session ID: ac30658fb877503145f65a505a4902e5 <<<
12月 05, 2024 9:52:59 午後 com.kms.katalon.core.logging.KeywordLogger endTest
情報: END Test Cases/TC1

The variable myButtonWebElement is an instance of org.openqa.selenium.WebElement class. Initially, the variable got a valid reference to the <button id='myButton'> element in the target page. But after 5 seconds of wait, the reference becomes stale because the JavaScript in the target page dynamically changed the page’s DOM. This is the core reason why a StaleElementReferenceException is thrown.

Test Cases/TC2

Let me show you another sample code. The TC1 referred to a variable of type org.openqa.selenium.WebElement class. But a usual Katalon Studio user won’t use the Selenium WebDriver API. They would primarily use WebUI.* keywords. The next TC2 calls only WebUI keywords, and it stil reproduces a StaleElementReferenceException.

See the source of Test Cases/TC2.

import static com.kms.katalon.core.testobject.ObjectRepository.findTestObject

import java.nio.file.Path
import java.nio.file.Paths

import org.openqa.selenium.WebElement

import com.kms.katalon.core.configuration.RunConfiguration
import com.kms.katalon.core.model.FailureHandling
import com.kms.katalon.core.testobject.TestObject
import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI

/**
 * TC2
 *
 * This script demonstrates the WebUI.waitForElementNotClickable keyword throws
 * Selenium StaleElementReferenceException (SERE).
 *
 * @author kazurayam
 */
Path projectDir = Paths.get(RunConfiguration.getProjectDir())
Path html = projectDir.resolve("docs/targetPage.html")
URL htmlURL = html.toFile().toURI().toURL()
String urlString = htmlURL.toExternalForm()
WebUI.comment("navigating to " + urlString)

// open a browser, navigate to the target web page
WebUI.openBrowser('')
WebUI.navigateToUrl(urlString)
WebUI.setViewPortSize(800, 600)

TestObject myButtonTestObject = findTestObject("Object Repository/myButton")

// make sure <button id='myButton'> is displayed in the page initially
WebUI.verifyElementPresent(myButtonTestObject, 10, FailureHandling.STOP_ON_FAILURE)

try {
    // at 3 secs after the initial page loading,
    // the old <button id='myButton'> was removed, but soon
    // a new <button id='myButton'> was recreated.
    // The keyword will see the HTML node stays clickable untile the timeout expires
    // However, the keyword throws a SERE for some reason.
   // Do you know why?
    WebUI.waitForElementNotClickable(myButtonTestObject,
                                10,
                                FailureHandling.STOP_ON_FAILURE)
} catch (Exception e) {
    println ">>> An Exception was caught: " + e.getClass().getName() + ": " + e.getMessage() + " <<<"
    println "==========================================================================="
    e.printStackTrace()
    println "==========================================================================="

}

WebUI.closeBrowser()

When I ran the TC2, I saw the following messages in the console. You see, a SERE was thrown.

12月 05, 2024 10:01:44 午後 com.kms.katalon.core.logging.KeywordLogger startTest
情報: START Test Cases/TC2
...
12月 05, 2024 10:01:54 午後 com.kms.katalon.core.logging.KeywordLogger logFailed
重大: ❌ Unable to wait for object 'Object Repository/myButton' to be not clickable (Root cause: com.kms.katalon.core.exception.StepFailedException: Unable to wait for object 'Object Repository/myButton' to be not clickable
    at com.kms.katalon.core.webui.keyword.internal.WebUIKeywordMain.stepFailed(WebUIKeywordMain.groovy:117)
    at com.kms.katalon.core.webui.keyword.internal.WebUIKeywordMain.runKeyword(WebUIKeywordMain.groovy:43)
    at com.kms.katalon.core.webui.keyword.internal.WebUIKeywordMain$runKeyword.call(Unknown Source)
    at com.kms.katalon.core.webui.keyword.builtin.WaitForElementNotClickableKeyword.waitForElementNotClickable(WaitForElementNotClickableKeyword.groovy:108)
    at com.kms.katalon.core.webui.keyword.builtin.WaitForElementNotClickableKeyword.execute(WaitForElementNotClickableKeyword.groovy:69)
    at com.kms.katalon.core.keyword.internal.KeywordExecutor.executeKeywordForPlatform(KeywordExecutor.groovy:74)
    at com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords.waitForElementNotClickable(WebUiBuiltInKeywords.groovy:597)
    at TC2.run(TC2:47)
  ...
Caused by: org.openqa.selenium.StaleElementReferenceException: stale element reference: stale element not found in the current frame
  (Session info: chrome=131.0.6778.109)
  ...
==========================================================================
>>> An Exception was caught: com.kms.katalon.core.exception.StepFailedException: Unable to wait for object 'Object Repository/myButton' to be not clickable
===========================================================================
12月 05, 2024 10:01:55 午後 com.kms.katalon.core.logging.KeywordLogger endTest
情報: END Test Cases/TC2

Please find that the Exception was raised at (WaitForElementNotClickableKeyword.groovy:108). So we should check the source code of Katalon Studio at com.kms.katalon.core.webui.keyword.builtin.WaitForElementNotClickableKeyword

package com.kms.katalon.core.webui.keyword.builtin
...
public class WaitForElementNotClickableKeyword extends WebUIAbstractKeyword {
    ..
    public boolean waitForElementNotClickable(TestObject to, int timeOut, FailureHandling flowControl) throws StepFailedException {
        return WebUIKeywordMain.runKeyword({
            ...
            try {
                ...
                try {
                    ...
                    WebElement foundElement = WebUIAbstractKeyword.findWebElement(to, timeOut)   // Line#103
                    WebDriverWait wait = new WebDriverWait(DriverFactory.getWebDriver(), Duration.ofSeconds(timeOut))
                    foundElement = wait.until(new ExpectedCondition<WebElement>() {
                                @Override
                                public WebElement apply(WebDriver driver) {
                                    if (foundElement.isEnabled()) {    // Line#108
                                        return null
                                    } else {
                                        return foundElement
                                    }
                                }
                            })
                    ...
                    return true
                } catch (WebElementNotFoundException e) {
                    ...
                    return false
                } catch (TimeoutException e) {
                    ...
                    return false
                }
            } finally {
                ...
            }
        }, ...)
    }
}

Why a StaleElementReferenceException was thrown at the Line#108?

At the Line#103, a variable named foundElement is declared to have a reference to an org.openqa.selenium.WebElement object, which points to the <button id='myButton'> element in the target page.

At the Line#108, the WebUI.waitForElementNotClickable keyword repeats referring to the foundElement until it find the web element is not clickable any more.

While the keyword is in the loop, in the target web page, the initial <button id='myButton'> element is once removed; and a new <button id='myButton'> element will be inserted.
At this timing, the foundElement becomes stale. It is not referring to a valid HTML element any more.
Therefore the WebUI.waitForElementNotClickable keyword threws a StaleElementReferenceException. A SERA was thrown by TC2 by just the same reason as the TC1.

Which WebUI keywords are likely to throw SERE?

In the TC2, I pointed out that the WebUI.waitForElementNotClickable keyword is likely to throw a StaleElementReferenceException.

Any other keywords would behave the same?

In the Test Cases/TC5, I pick up just a few other WebUI keywords and found the following 2 built-in keywords threw SERE.

WebUI.waitForElementNotHasAttribute
WebUI.waitForElementNotVisible

There could be more.

It was interesting to find that the WebUI.waitForElementNotPresent keyword did not throw SERE. I checked its Groovy source code and found it is implemented nicely so that it prevents SERE.

Stateful keywords, Stateless keywords

I would like to propose a set of categorical terms:

Stateless keyword, e.g, WebUI.waitForElementNotPresent
Stateful keyword, e.g, WebUI.waitForElementNotClickable

A stateful keyword has a variable of type org.openqa.selenium.WebElement, which is once gets initialized with a valid reference to a DOM element
in the target page, and is kept in memory and is referred to repeatedly for long seconds.
When the DOM is changed by JavaScript in the target page, the variable becomes stale. A stateful keyword is unreliable. Occationally it could throw a StaleElementReferenceException.

A stateless keyword doesn’t have such a variable, therefore won’t throw any StaleElementReferenceException.

I believe that all built-in keywords should be stateless so that they never throw StaleElementReferenceException.

So, how many stateful keywords are there in Katalon Studio? — Well, I don’t know. I hope that Katalon will publish the list of stateful keywords A.S.A.P. so that the users can refrain from those risky keywords.

Test Cases/TC3

I would show one more script. The TC3 is almost the same as TC2 except one line different. See the source of Test Cases/TC3.

import static com.kms.katalon.core.testobject.ObjectRepository.findTestObject

import java.nio.file.Path
import java.nio.file.Paths

import org.openqa.selenium.WebElement

import com.kms.katalon.core.configuration.RunConfiguration
import com.kms.katalon.core.model.FailureHandling
import com.kms.katalon.core.testobject.TestObject
import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI

/**
 * TC3
 *
 * A variation derived from the TC2.
 *
 * This script demonstrates that FailureHandling.CONTINUE_ON_FAILURE makes
 * all keywords silent.
 * No exception will be raised by a keyword invokation.
 * Your Test Case script can not catch any Exception.
 *
 * @author kazurayam
 */
Path projectDir = Paths.get(RunConfiguration.getProjectDir())
Path html = projectDir.resolve("docs/targetPage.html")
URL htmlURL = html.toFile().toURI().toURL()
String urlString = htmlURL.toExternalForm()
WebUI.comment("navigating to " + urlString)

WebUI.openBrowser('')
WebUI.navigateToUrl(urlString)
WebUI.setViewPortSize(800, 600)

TestObject myButtonTestObject = findTestObject("Object Repository/myButton")

WebUI.verifyElementPresent(myButtonTestObject, 10, FailureHandling.STOP_ON_FAILURE)

try {
    WebUI.verifyElementNotPresent(myButtonTestObject,
                                10,
                                FailureHandling.CONTINUE_ON_FAILURE)
    // The keyword will throw no Exception
} catch (Exception e) {
    // You can not catch SERE here
    println ">>> An Exception was caught: " + e.getClass().getName() + ": " + e.getMessage() + " <<<"
    println "==========================================================================="
    e.printStackTrace()
    println "==========================================================================="
}

WebUI.closeBrowser()

// In the end, in the Console, you will find a long Stack trace of StepFailedException is printed.

The TC2 has a code fragmen like this:

try {
    WebUI.waitForElementNotClickable(myButtonTestObject,
                                10,
                                FailureHandling.STOP_ON_FAILURE)
    // so the keyword will throw a SERE

The TC3 has a code like this:

try {
    WebUI.waitForElementNotClickable(myButtonTestObject,
                                10,
                                FailureHandling.CONTINUE_ON_FAILURE)
} catch (Exception e) {
    // You can not catch SERE here
    println ">>> An Exception was caught: " + e.getClass().getName() + ": " + e.getMessage() + " <<<"
    println "==========================================================================="
    e.printStackTrace()
    println "==========================================================================="
}

When I ran the TC3, I got the following output in the console:

12月 05, 2024 11:29:53 午後 com.kms.katalon.core.logging.KeywordLogger startTest
情報: START Test Cases/TC3
...
12月 05, 2024 11:30:24 午後 com.kms.katalon.core.logging.KeywordLogger logFailed
重大: ❌ Web element with id: 'Object Repository/myButton' located by 'By.xpath: //button[@id='myButton']' is present after '10' second(s) (Root cause: com.kms.katalon.core.exception.StepFailedException: Web element with id: 'Object Repository/myButton' located by 'By.xpath: //button[@id='myButton']' is present after '10' second(s)
    at com.kms.katalon.core.webui.keyword.internal.WebUIKeywordMain.stepFailed(WebUIKeywordMain.groovy:117)
    at com.kms.katalon.core.webui.keyword.internal.WebUIKeywordMain$stepFailed$0.call(Unknown Source)
    at com.kms.katalon.core.webui.keyword.builtin.VerifyElementNotPresentKeyword$_verifyElementNotPresent_closure1.doCall(VerifyElementNotPresentKeyword.groovy:124)
    at com.kms.katalon.core.webui.keyword.builtin.VerifyElementNotPresentKeyword$_verifyElementNotPresent_closure1.doCall(VerifyElementNotPresentKeyword.groovy)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at com.kms.katalon.core.webui.keyword.internal.WebUIKeywordMain.runKeyword(WebUIKeywordMain.groovy:35)
    at com.kms.katalon.core.webui.keyword.internal.WebUIKeywordMain$runKeyword.call(Unknown Source)
    at com.kms.katalon.core.webui.keyword.builtin.VerifyElementNotPresentKeyword.verifyElementNotPresent(VerifyElementNotPresentKeyword.groovy:133)
    at com.kms.katalon.core.webui.keyword.builtin.VerifyElementNotPresentKeyword.execute(VerifyElementNotPresentKeyword.groovy:70)
    at com.kms.katalon.core.keyword.internal.KeywordExecutor.executeKeywordForPlatform(KeywordExecutor.groovy:74)
    at com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords.verifyElementNotPresent(WebUiBuiltInKeywords.groovy:1557)
    at TC3.run(TC3:40)
    ...
  Caused by: com.kms.katalon.core.exception.StepFailedException: Web element with id: 'Object Repository/myButton' located by 'By.xpath: //button[@id='myButton']' is present after '10' second(s)
    ... 27 more
  ...
12月 05, 2024 11:30:25 午後 com.kms.katalon.core.logging.KeywordLogger endTest
情報: END Test Cases/TC3

You can see there is no output from the statement

println ">>> An Exception was caught: " + e.getClass().getName() + ": " + e.getMessage() + " <<<"

This implies that no exception was thrown out of a keyword call because FailureHandling.CONTINUE_ON_FAILURE was specified.

Test Cases/TC4, custom resolution

The TC2 demonstrates that the WebUI.waitForElementNotClickable keyword occasionally throws a StaleElementReferenceException.

Can I fix this problematic keyword? — Yes, I can propose an idea. Let me show it.

I made 2 codes:

The TC4 is similar to the TC2 but different. The TC4 calls com.kazurayam.hack.StatelessWaitForElementNotClickableKeyword class instead of the WebUI.waitForElementNotClickable keyword. When I ran the TC4, it threws no SERE.

Let’s compare the source of 2 classes to see the difference:

com.kms.katalon.core.webui.keyword.builtin.WaitForElementNotClickable

This code is likely to throw SERE.

...
        try {
          WebElement foundElement = WebUIAbstractKeyword.findWebElement(to, timeOut)
          WebDriverWait wait = new WebDriverWait(DriverFactory.getWebDriver(), Duration.ofSeconds(timeOut))
          foundElement = wait.until(new ExpectedCondition<WebElement>() {
            @Override
            public WebElement apply(WebDriver driver) {
              if (foundElement.isEnabled()) {
                return null
              } else {
                return foundElement
              }
            }
          })
          if (foundElement != null) {
            ...
          }
          return true
...

com.kazurayam.hack.StatelessWaitForElementNotClickable

This code does not throw any SERE.

...
        try {
          WebDriverWait wait = new WebDriverWait(DriverFactory.getWebDriver(), Duration.ofSeconds(timeOut))
          WebElement webElement = wait.until(new ExpectedCondition<WebElement>() {
            @Override
            public WebElement apply(WebDriver driver) {
              // recreate a reference to the <button id='myBook'>
              WebElement foundElement = WebUIAbstractKeyword.findWebElement(to, timeOut)
              if (foundElement.isEnabled()) {
                return null
              } else {
                return foundElement
              }
            }
          })
...

In the built-in keyword, the foundElement variable is created once outside the scope of wait loop and refered to many times inside the wait loop. The reference to the <button id='myButton'> element has enough chance to get stale.

On the other hand, in my class, the foundElement variable is scoped narrow; it resides inside the apply method. The foundElement variable is created once, refered to only once, then immediately thrown away. The reference to the <button id='myButton'> has no chance to get stale.

TC6 WebUI.waitForElementClickable keyword could throw a StaleElementReferenceException

The TC2 showed me there are some problematic keywords with name starting with “WebUI.waitForElementNot”. But how about the keywords of “WebUI.waitForElement*” ? No one throws a SERE? I examined some and found that “WebUI.waitForElementClickable” keyword could throw a SERE.

I made a Test Case:

Test Cases/TC6_waitForElementClickable

This script targets the url

https://kazurayam.github.io/StaleElementReferenceExceptionReproduction/initiallyDisabledLaterEnabled.html

When I ran the TC6, I saw a SERE was thrown:

Why the waitForElementClickable keyword threw a StaleElementReferenceException? There is a pitfall in the target HTML. The HTML initially has a <button id="myButton" disabled>.

The disabled attribute makes the element unable to click. Therefore the WebUI.waitForElementClickable keyword is forced to wait for a while. Then at 3 secs after the initial page loading, JavaScript removes the element and recreate a new <button id="myButton"> with no disabled attribute. As soon as the button element is recreated, the WebUI.waitForElementClickable keyword tries to access to the WebElement object, which has got already stale.

The WebUI.waitForElementClickable is a stateful keyword. There could be more.

Conclusion

I presented the reason how a StaleElementReferenceException is raised by Katlaon WebUI keywords. The Exception could be thrown with the combination of 2 factors:

the target web page is driven by JavaScript, which changes the page’s DOM dynamically: remove an Element, recreate the Element.
the WebUI keyword is implemented stateful like WebUI.waitForElementNotClickable. A variable of type org.openqa.selenium.WebElement is repeatedly referred to while the WebElement turned to be stale due to the dynamic DOM change by JavaScript in the target web page.

Lesson learned:

when you encountered a SERE, you need to study how your target web page is written. You need to be aware how JavaScript works inside the page.
you need to read the source code of WebUI keywords if it threw a SERE.

How can you avoid SERE at all? Well, I don’t know. There is no silver bullet. Please find your way for yourself.

In the TC4, I showed how to fix the defect in a built-in keyword. I hope Katalon to address my suggestion and fix the problematic built-in keywords.

wiky.hendra · December 6, 2024, 6:51am

Thank you for the information. I got this problem, and for now its fixed. I simply used WebUI.waitforelementpresent and clear properties (i made test object in script). Hope this would help other people too.

'Make Test Object'
data1 = new TestObject('list')

///result is an array with expected value
‘loop per result size’
for (int i : (1…result.size())) {
‘Adding Property to data1 with xpath settings and specified locator’
data1.addProperty(‘xpath’, ConditionType.EQUALS, (‘/html/body/div[2]/layout-manager/div/layout/div/div[6]/tab/div/div[7]/div[3]/div/form/div/div/div[2]/div[2]/field-block[’ +
i) + ‘]/div/div[2]/field/readonly-field/div/input’)

	'Wait for element present'
    WebUI.waitForElementPresent(data1, 3)
	
	'Verify Expected and Result is same'
    (result[(i - 1)]) != '' ? WebUI.verifyEqual(WebUI.getAttribute(data1, 'title'), result[(i - 1)], FailureHandling.CONTINUE_ON_FAILURE) : ''

	'Clear the property in data1'
    data1.getProperties().clear()
}

This works on me. Now, when the problem is explained, i understand the problem. Thank you.

Note : Its just 1 of many cases. I just share my opinion regarding this. If there’s another cases coming by, i hope you guys share it too.

kazurayam · December 7, 2024, 12:06am

@vu.tran
@Elly_Tran

I believe that a StaleElementReferenceException is caused due to the small defects in the implementation of Katalon built-in keywords.

Hey, Katalon! You already know how to prevent SERE. Why don’t you do the same for all built-in keywords!?

pasdaryaser · December 8, 2024, 4:03pm

package com.kms.katalon.core.webui.keyword.builtin
…
public class WaitForElementNotClickableKeyword extends WebUIAbstractKeyword {
…
public boolean waitForElementNotClickable(TestObject to, int timeOut, FailureHandling flowControl) throws StepFailedException {
return WebUIKeywordMain.runKeyword({
…
try {
…
try {
…
WebElement foundElement = WebUIAbstractKeyword.findWebElement(to, timeOut) // Line#103
WebDriverWait wait = new WebDriverWait(DriverFactory.getWebDriver(), Duration.ofSeconds(timeOut))
foundElement = wait.until(new ExpectedCondition() {
@Override
public WebElement apply(WebDriver driver) {
if (foundElement.isEnabled()) { // Line#108
return null
} else {
return foundElement
}
}
})
…
return true
} catch (WebElementNotFoundException e) {
…
return false
} catch (TimeoutException e) {
…
return false
}
} finally {
…
}
}, …)
}
}

kazurayam · December 8, 2024, 11:15pm

@pasdaryaser

Please use the Code Block syntax of Discourse Markdown to make your post better readable.

arvind.choudhary · December 10, 2024, 6:51am

Yes, I too have faced same StaleElementReferenceException while automating script. As you also explained, there are 3 ways to deal as mentioned below

Put wait of 5-10 sec
Include waitforElementvisibility keyword before that object with 20-30 sec
Include WaitforElementPresent keyword before the object with 20-30 sec

Either of the above steps will work

Regards,
Arvind Kumar C.

kazurayam · December 10, 2024, 7:20am

As I explained above, a StaleElementReferenceException will be thrown due to a combination of 2 factors:

your target web page is driven by JavaScript which dynamically changes the DOM: remove an Element, recreate the Element
WebUI keyword you used is coded with a defect to be StaleElementReferenceExpetion-prone.

It is pointless to inform others of your reckless choice saying “this keyword worked for me” without any analysis how your target web page is driven by JavaScript, without any analysis on the Groovy source code of the keywords you used.

arvind.choudhary · December 10, 2024, 7:37am

Keep your script as it is, just include lines as shown in the below image, It will work.

So, here in the line 65, was throwing StaleElementReferenceException while automating script in Katalon. Just include either of lines 60, 61, or 62. It will work. Best case scenario is to keep 61 and 62 both and execute the test script.
try it

lordsurya007 · December 10, 2024, 9:55am

I tried and working fine

arvind.choudhary · December 10, 2024, 1:06pm

And if the element is dropdown then use waitforelementclickable it will work

Simply it is working

kazurayam · December 12, 2024, 9:46am

@Elly_Tran

I added this:

kazurayam:

Stateful keywords, Stateless keywords

I would like to propose a set of categorical terms:

Stateful keyword, e.g, WebUI.waitForElementNotClickable

Stateless keyword, e.g, WebUI.waitForElementNotPresent

A stateful keyword has a variable of type org.openqa.selenium.WebElement, which is once gets initialized with a valid reference to a DOM element in the target page, and is kept in memory and is referred to repeatedly for long seconds. When the DOM is changed by JavaScript in the target page, the variable becomes stale. A stateful keyword is unreliable. Occationally it could throw a StaleElementReferenceException.

A stateless keyword doesn’t have such a variable, therefore won’t throw any StaleElementReferenceException.

I believe that all built-in keywords should be stateless so that they never throw StaleElementReferenceException.

How many stateful keywords are there in Katalon Studio? — Well, precisely I don’t know. I hope that Katalon will publish the list of stateful keywords A.S.A.P. so that the users can refrain from those risky keywords.

kazurayam · December 15, 2024, 12:38am

I added a section:

TC6 WebUI.waitForElementClickable keyword could throw a StaleElementReferenceException

The TC2 showed me there are some problematic keywords with name starting with “WebUI.waitForElementNot”. But how about the keywords of “WebUI.waitForElement*” ? No one throws a SERE? I examined some and found that “WebUI.waitForElementClickable” keyword could throw a SERE.

I made a Test Case:

Test Cases/TC6_waitForElementClickable

This script targets the url

https://kazurayam.github.io/StaleElementReferenceExceptionReproduction/initiallyDisabledLaterEnabled.html

When I ran the TC6, I saw a SERE was thrown:

TC6

Why the waitForElementClickable keyword threw a StaleElementReferenceException? There is a pitfall in the target HTML. The HTML initially has a <button id="myButton" disabled>.

TC6 html

The disabled attribute makes the element unable to click. Therefore the WebUI.waitForElementClickable keyword is forced to wait for a while. Then at 3 secs after the initial page loading, JavaScript removes the element and recreate a new <button id="myButton"> with no disabled attribute. As soon as the button element is recreated, the WebUI.waitForElementClickable keyword tries to access to the WebElement object, which has got already stale.

The WebUI.waitForElementClickable is a stateful keyword. There could be more.

kazurayam · December 17, 2024, 11:46am

I created a new topic that concerns the StaleElementReferenceException

Topic		Replies	Views
Katalon Studio Fails to Handle StaleElementReferenceException Bugs Report katalon-studio , bugs-report , try-catch , exception	15	126	December 11, 2024
Stale Element Reference Exception in Firefox (headless mode) Katalon Studio katalon-studio , web-testing	14	2445	February 4, 2022
Is there any way to handle "stale element reference: element is not attached to the page document" Katalon Studio katalon-studio , web-testing	6	2848	August 8, 2022
Stale element reference: element is not attached to the page document. Katalon Studio katalon-studio , web-testing	31	33559	December 31, 2021
[Error] stale element reference: element is not attached to the page document Archive	1	3384	July 30, 2018

Reproducing Selenium StaleElementReferenceException in Katalon Studio

Achronym “SERE”

Problem to solve

Reference

Solution

Decription

The target page

Object Repository/myButton

Test Cases/TC1

Test Cases/TC2

Which WebUI keywords are likely to throw SERE?

Stateful keywords, Stateless keywords

Test Cases/TC3

Test Cases/TC4, custom resolution

com.kms.katalon.core.webui.keyword.builtin.WaitForElementNotClickable

com.kazurayam.hack.StatelessWaitForElementNotClickable

TC6 WebUI.waitForElementClickable keyword could throw a StaleElementReferenceException

Conclusion

TC6 WebUI.waitForElementClickable keyword could throw a StaleElementReferenceException

Related topics