Stale Element Reference

I am getting the “Stale Element Reference” seemingly at random. I have a page where I capture the text on a vehicle make link and store it to a variable. I then click the link which updates the panel with a new report (not reloading the page). One the new report loads, I then capture vehicle makes from all of the rows in the grid and loop through them to verify that all are the same. Then I click on the 1st rows, model link and do the same process of checking.

This is one point (clicking the model link) that I am randomly getting the error. If it passes this point, it then clicks a bread crumb that takes me back to the original report where I click the Make link, but this time it captures the text and clicks the link for the Model. This seems to be the point that throws this error the most. Again, it seems random as to whether it will pass or get the Stale Element Reference.

Here is a snipping of code where it is doing the checks:

CustomKeywords.'dwKeywords.WaitForSpinner.spinnerNotVisible'('#panel3 multi-comp div div datatable progress-bar div')
			
			CustomKeywords.'dwKeywords.MakeModelCheck.verifyMakeModel'(reportName, 'Make')
			
			CustomKeywords.'dwKeywords.WaitForSpinner.spinnerNotVisible'('#panel3 multi-comp div div datatable progress-bar div')
			
			CustomKeywords.'dwKeywords.MakeModelCheck.verifyMakeModel'(reportName, 'Model')
			
			// New Inventory Details Breadcrumb
			WebUI.executeJavaScript('document.querySelectorAll(\'#panel3 a\')[1].click()', null)
			
			CustomKeywords.'dwKeywords.WaitForSpinner.spinnerNotVisible'('#panel3 multi-comp div div datatable progress-bar div')
			
			CustomKeywords.'dwKeywords.MakeModelCheck.verifyMakeModel'(reportName, 'Model')

Here is where the CustomKeyword is defined:

public class WaitForSpinner {
	@Keyword
	def spinnerNotVisible(String selector) {
		def count = 0

		def spinnerHidden = false

		if (count >= 20) {
			return
		} else {
			if (spinnerHidden == false) {
				def classList = WebUI.executeJavaScript('return document.querySelectorAll(\'' + selector + '\')[0].classList', null)

				if (classList.contains('ng-hide')) {
					spinnerHidden = true

					return spinnerHidden
				} else {
					count++

					WebUI.delay(0.5)

					spinnerNotVisible(selector)
				}
			}
		}
	}
}

And here is the Error Log from the Katalon Console:

=============== ROOT CAUSE =====================

For trouble shooting, please visit: https://docs.katalon.com/katalon-studio/docs/troubleshooting.html

07-01-2021 09:18:43 AM Test Cases/VariableOperations/NewCarSalesReports

Elapsed time: 1m - 11.664s

Test Cases/VariableOperations/NewCarSalesReports FAILED.
Reason:
org.openqa.selenium.StaleElementReferenceException: stale element reference: element is not attached to the page document
(Session info: chrome=91.0.4472.124)
For documentation on this error, please visit: Exceptions | Selenium
Build info: version: ‘3.141.59’, revision: ‘e82be7d358’, time: ‘2018-11-14T08:25:53’
System info: host: ‘DWDEVWS008’, ip: ‘172.16.1.220’, os.name: ‘Windows 10’, os.arch: ‘amd64’, os.version: ‘10.0’, java.version: ‘1.8.0_181’
Driver info: com.kms.katalon.selenium.driver.CChromeDriver
Capabilities {acceptInsecureCerts: false, browserName: chrome, browserVersion: 91.0.4472.124, chrome: {chromedriverVersion: 90.0.4430.24 (4c6d850f087da…, userDataDir: C:\Users\DEVIN~1.BRO\AppDat…}, goog:chromeOptions: {debuggerAddress: localhost:51745}, javascriptEnabled: true, networkConnectionEnabled: false, pageLoadStrategy: normal, platform: WINDOWS, platformName: WINDOWS, proxy: Proxy(), setWindowRect: true, strictFileInteractability: false, timeouts: {implicit: 0, pageLoad: 300000, script: 30000}, unhandledPromptBehavior: dismiss and notify, webauthn:extension:largeBlob: true, webauthn:virtualAuthenticators: true}
Session ID: 2d0245a6651b3aa85bb2510110a68abe
at org.openqa.selenium.remote.http.W3CHttpResponseCodec.createException(W3CHttpResponseCodec.java:187)
at org.openqa.selenium.remote.http.W3CHttpResponseCodec.decode(W3CHttpResponseCodec.java:122)
at org.openqa.selenium.remote.http.W3CHttpResponseCodec.decode(W3CHttpResponseCodec.java:49)
at org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:158)
at org.openqa.selenium.remote.service.DriverCommandExecutor.execute(DriverCommandExecutor.java:83)
at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:552)
at com.kms.katalon.selenium.driver.CChromeDriver.execute(CChromeDriver.java:19)
at org.openqa.selenium.remote.RemoteWebElement.execute(RemoteWebElement.java:285)
at org.openqa.selenium.remote.RemoteWebElement.getText(RemoteWebElement.java:166)
at org.openqa.selenium.support.events.EventFiringWebDriver$EventFiringWebElement.lambda$new$0(EventFiringWebDriver.java:404)
at com.sun.proxy.$Proxy11.getText(Unknown Source)
at org.openqa.selenium.support.events.EventFiringWebDriver$EventFiringWebElement.getText(EventFiringWebDriver.java:463)
at dwKeywords.MakeModelCheck.verifyMakeModel(MakeModelCheck.groovy:80)
at dwKeywords.MakeModelCheck.invokeMethod(MakeModelCheck.groovy)
at com.kms.katalon.core.main.CustomKeywordDelegatingMetaClass.invokeStaticMethod(CustomKeywordDelegatingMetaClass.java:50)
at NewCarSalesReports.processReports(NewCarSalesReports:126)
at Script1623172074810$processReports.callCurrent(Unknown Source)
at NewCarSalesReports.run(NewCarSalesReports:57)
at com.kms.katalon.core.main.ScriptEngine.run(ScriptEngine.java:194)
at com.kms.katalon.core.main.ScriptEngine.runScriptAsRawText(ScriptEngine.java:119)
at com.kms.katalon.core.main.TestCaseExecutor.runScript(TestCaseExecutor.java:369)
at com.kms.katalon.core.main.TestCaseExecutor.doExecute(TestCaseExecutor.java:360)
at com.kms.katalon.core.main.TestCaseExecutor.processExecutionPhase(TestCaseExecutor.java:339)
at com.kms.katalon.core.main.TestCaseExecutor.accessMainPhase(TestCaseExecutor.java:331)
at com.kms.katalon.core.main.TestCaseExecutor.execute(TestCaseExecutor.java:248)
at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:142)
at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:133)
at com.kms.katalon.core.main.TestCaseMain$runTestCase$0.call(Unknown Source)
at TempTestCase1625145519841.run(TempTestCase1625145519841.groovy:25)

Any help would be greatly appreciated!

I will provide the long answer, and a tl;dr. I think it’s helpful to understand what this exception means so that you can avoid it more generally in the future.

Long Answer

You may already be aware of what a StaleElementReferenceException is, but I will give some context anyway.

This error occurs when you attempt to use a reference to a WebElement that is no longer valid against the current DOM (i.e. the reference is “stale”). In other words, you have captured a WebElement, but at some point the page has changed in such a way that this element reference is no longer valid. This does not have to be an entire page reload or a redirect; It can happen even if only a portion of the page has changed in some way (after an AJAX request, for example). The randomness of this exception comes from the fact that page load times vary across iterations, so sometimes your script is able to use the element before it is reloaded, sometimes not.

This can be hard to visualize in the HTML itself. Imagine for a moment a very real scenario: You’ve located a link element. At some point, the element for your link is then removed from the DOM, then reloaded into the DOM as the exact same element. To you as the observer, this element doesn’t appear to have changed, but it’s technically not the same element, it’s a “copy”. If you located this link prior to it being removed/added back, then tried to use that reference after the reload, you would get the same exception. This is just one example among countless others.

Now, in reference to your exact custom keyword, I don’t have line numbers, but according to the error log its at dwKeywords.MakeModelCheck.verifyMakeModel(MakeModelCheck.groovy:80). If you can point me to exactly which line of code that is, I can give a specific answer as to what’s going on. However, I think you’d be better off following my advice below to handle this error more generally.

"Short" Answer

More generally, here’s what you can do to avoid StaleElementExceptions across the board. If you follow these simple principles, you will rarely ever see this exception in your future automation efforts:

1.) Ensure that EVERY element reference is 1-time use. In other words, locate the element, use it, and DO NOT use that same reference again. If you want to reference that element again, you should re-locate it in the HTML every single time.

2.) Ensure that you have proper wait conditions in place for every action you take in your application. Always wait for the page to be “static” prior to locating elements and using them.

I’m happy to elaborate, but this might be too much info already. Let me know :slight_smile:

Sorry about that…I posted the code for the spinner check…here is the MakeModelCheck. I believe that line 80 is going to be right near the bottom: If(value.toUpperCase() != rowValue.toUpperCase()) {

package dwKeywords

import static com.kms.katalon.core.checkpoint.CheckpointFactory.findCheckpoint
import static com.kms.katalon.core.testcase.TestCaseFactory.findTestCase
import static com.kms.katalon.core.testdata.TestDataFactory.findTestData
import static com.kms.katalon.core.testobject.ObjectRepository.findTestObject
import static com.kms.katalon.core.testobject.ObjectRepository.findWindowsObject
import com.kms.katalon.core.annotation.Keyword
import com.kms.katalon.core.checkpoint.Checkpoint
import com.kms.katalon.core.cucumber.keyword.CucumberBuiltinKeywords as CucumberKW
import com.kms.katalon.core.mobile.keyword.MobileBuiltInKeywords as Mobile
import com.kms.katalon.core.model.FailureHandling
import com.kms.katalon.core.testcase.TestCase
import com.kms.katalon.core.testdata.TestData
import com.kms.katalon.core.testng.keyword.TestNGBuiltinKeywords as TestNGKW
import com.kms.katalon.core.testobject.TestObject
import com.kms.katalon.core.webservice.keyword.WSBuiltInKeywords as WS
import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI
import com.kms.katalon.core.webui.keyword.builtin.ConvertWebElementToTestObjectKeyword as ConvertWebElementToTestObjectKeyword
import com.kms.katalon.core.windows.keyword.WindowsBuiltinKeywords as Windows
import internal.GlobalVariable
import org.openqa.selenium.Keys as Keys

public class MakeModelCheck {
	@Keyword
	def verifyMakeModel(String reportName, String type) {
		String value = ''

		boolean checkPassed = true
		boolean reportOpened = true

		int columnNumber = 0

		WebUI.delay(2)

		def headerArray = WebUI.executeJavaScript('return document.querySelectorAll(\'#data-table thead th\')', null)

//		WebUI.delay(2)

		for (def val : headerArray) {
			String header = val.text

			if (header == type) {
				break
			} else {
				columnNumber++
			}
		}

//		WebUI.delay(2)

		if (type == 'Make') {
			value = WebUI.executeJavaScript('return document.querySelectorAll(\'#data-table tbody td\')[' + columnNumber.toString() + '].innerText', null)

//			WebUI.delay(2)

			WebUI.executeJavaScript('document.querySelectorAll(\'#data-table tbody td\')[' + columnNumber.toString() + '].children[0].click()', null)
		} else if (type == 'Model') {
			value = WebUI.executeJavaScript('return document.querySelectorAll(\'#data-table tbody td\')[' + columnNumber.toString() + '].innerText', null)

//			WebUI.delay(2)

			WebUI.executeJavaScript('document.querySelectorAll(\'#data-table tbody td\')[' + columnNumber.toString() + '].children[0].click()', null)
		}

//		if (!(reportOpened = CustomKeywords.'dwKeywords.WaitForSpinner.spinnerNotVisible'('#panel3 multi-comp div div datatable progress-bar div'))) {
//			WebUI.comment(reportName + ': ' + type + ' report did not open!')
//
//			assert false
//		}

		if (!(reportOpened = new dwKeywords.WaitForSpinner().spinnerNotVisible('#panel3 multi-comp div div datatable progress-bar div'))) {
			WebUI.comment(reportName + ': ' + type + ' report did not open!')

			assert false
		}

		columnNumber = 0

		headerArray = []

		headerArray = WebUI.executeJavaScript('return document.querySelectorAll(\'#data-table thead th\')', null)

//		WebUI.delay(2)

		for (def val : headerArray) {
			String header = val.text

			if (header == type) {
				break
			} else {
				columnNumber++
			}
		}

//		WebUI.delay(2)

		def rowArray = WebUI.executeJavaScript('return document.querySelectorAll(\'#data-table tbody tr td:nth-child(' + (columnNumber + 1).toString() + ')\')', null)

//		WebUI.delay(2)

		for (def row : rowArray) {
			String rowValue = row.text

			println rowValue

			if (value.toUpperCase()!= rowValue.toUpperCase()) {
				WebUI.comment(rowArray.toString())

				WebUI.comment(reportName + ': ' + type + ' report not all ' + type + 's match!')

				assert false
			}
		}

		return checkPassed
	}
}

As you can see here, I am capturing the elements in my tables header and looping through to determine which column holds Make or Model depending on what is passed in. This to me is why it is so baffling because I am using an executeJavaScript every time it is called, so to me, it should either find the element or return an element not found error.

As for you suggestion on the wait conditions, as you can see, I have them sprinkled throughout trying to find the right balance of speed vs. accuracy. I do not believe this would affect the stale elements issue, but the timing is causing issues with my results by capturing a record count or list of Makes/Models from the current table, after I click the link to open the next child table. It should be capturing that data from the child table.

Using hard waits like WebUI.delay() is a cardinal sin of test automation, and will be the bane of your existence as an automation engineer. Never use them. The only reason to use a hard wait like that is to confirm that you have a timing issue, after which you need to implement a legitimate wait condition to handle it. (Also, I see that they are commented out, so I’m not sure that you are even waiting the 2 seconds like you’re intending to).

This is an art in it’s own right, but usually it involves waiting for some condition on the page to be true. If your application has a spinner or an overlay of some sort that appears when it’s loading, it’s easy enough to wait for that to disappear, indicating that the page is ready for further input. If you’re not that lucky, you’ll have to watch for specific elements and/or attributes in the DOM to appear/disappear, indicating that the page is in a static state. See the built-in WebUI.waitForElement___() family of methods.

Now, as for your specific issue, it violates point #1 that I made above. When you do something like this:

def rowArray = WebUI.executeJavaScript('return document.querySelectorAll(\'#data-table tbody tr td:nth-child(' + (columnNumber + 1).toString() + ')\')', null)

You get a reference to an array of elements, and you are reusing this reference multiple times. While you iterate through these elements and do work, your page has plenty of time to potentially reload, which would completely invalidate your list. There are two ways to mitigate this:

1.) Re-locate the list of elements after each iteration:

def rowArray = WebUI.executeJavaScript('return document.querySelectorAll(\'#data-table tbody tr td:nth-child(' + (columnNumber + 1).toString() + ')\')', null)
for (def row : rowArray) {
    String rowValue = row.text
    println rowValue
    if (value.toUpperCase()!= rowValue.toUpperCase()) {
        WebUI.comment(rowArray.toString())
        WebUI.comment(reportName + ': ' + type + ' report not all ' + type + 's match!')
        assert false
    }
    rowArray = WebUI.executeJavaScript('return document.querySelectorAll(\'#data-table tbody tr td:nth-child(' + (columnNumber + 1).toString() + ')\')', null)
}

2.) (Preferred method) Grab the size of the array once, then grab each row individually by index and get the text:

def size = WebUI.executeJavaScript('return document.querySelectorAll(\'#data-table tbody tr td:nth-child(' + (columnNumber + 1).toString() + ')\').length', null)
for (i = 0; i < size; i++) {
    def row = WebUI.executeJavaScript('return document.querySelectorAll(\'#data-table tbody tr td:nth-child(' + (columnNumber + 1).toString() + ')\').item(' + i + ')', null)
    String rowValue = row.text
    println rowValue
    if (value.toUpperCase()!= rowValue.toUpperCase()) {
        WebUI.comment(rowArray.toString())
        WebUI.comment(reportName + ': ' + type + ' report not all ' + type + 's match!')
        assert false
    }
}

(You should probably check my JavaScript here, I don’t normally use it for element location.)

Method 2 seems to have cleared up the stale element issue. Thank you so much for that!

1 Like