Need assistance understanding stale reference with regards to AJAX table loading

Trying to resolve a consistent Stale Object reference that I’m getting where I have a page that loads the main page first, then if a specific button is pressed loads a “shell table”, then populates the table data for that table, usually taking 7 - 10 seconds to do so because of some intricate calculations and renderings that go on in the background.

The issue is that when I go to check the row and column counts (as outlined in some of my other posts here) I’m running into stale object errors due to the checks being run on the initial empty table object instead of the newly refreshed fully populated table.

So far I’ve “cheated” by adding ever-increasing wait statements, but know that this is a bad solution. Finally getting time to tackle the issue; however I’m not finding a good way around it because "if exists " checks fail due to the shell table being present, and “text check” methods fail because they run into the stale table object.

I there a good way to continually check for an updated and fully populated table (i.e. - the AJAX call to create the table has finished completely) that won’t itself trigger a stale object reference, or [barring that option] a way to force an object refresh on the Katalon side whenever a stale object is detected?

1 Like

Hi there,

Thank you very much for your topic. Please note that it may take a little while before a member of our community or from Katalon team responds to you.

Thanks!

@kevin.jackey

Stale element reference, AJAX call … welcome to the death zone of Selenium testing.

Have a look at the following previous post where I struggled with the issue.

Katalon’s SmartWait is designed to solve this issue. But I am afraid that you are likely to get puzzled with it.

The Stale element reference — this sort of issue highly depends on each individual project’s Application Under Test, the test script codes and the project settings. It is very difficult for others in this public forum to support you remotely.

If you are a paying licensee of the Enterprise license, then it would be an idea to raise an official support request to Katalon. You can communicate with them in a closed manner so that you can disclose everything to them and ask for their solution.

I did try activating Smart Wait prior to the call sequence, but unfortunately it did not appear to have any positive effect

1 Like

I have no more idea.

Understood, only thing I could think of is if there’s some way to trap the stale object error and force a refresh of the object linkage, can you think of any way to do that? Would the stale object error be trappable?

I have no idea because I do not have your project in my hand.

Only you could find out something by hands-on experiments in your project.

Perhaps you can see if the base Exception works if you can catch anything, like below, or you can see if you can catch specific StaleElementReferenceException (perhaps with a safety Exception clause as well ??):

try {
   ...
} catch (Exception) { blah blah}

or

try {
   ...
} catch (StaleElementReferenceException) {
  blah blah
} catch (Exception) { blah blah}

@Brandon_Hein 's suggestion may interest you:

Agree and definitely saw that, the challenge I’m having is in how to find the object that it’s failing on.

What I see when the script runs on a page (some of this I’ve posted before, so may look familiar, also I have trimmed out all the extraneous checks as statement in the interest of time, but can post the entire code blocks if those would help):

  1. Script opens page
  2. Script locates object with currently hidden + unpopulated table
  3. Script selects the button to open the table view section, triggering the table populate
  4. Table populates, visually it appears to be complete and all wait checks I have thought of (have tried VerifyElementPresent, VerifyTextPresent, VerifyElementVisible) have completed
  5. Table Validation begins:

Setup table object using xpath indicator

``` //Define table object TestObject this_table_object = new TestObject (table_name) this_table_object.addProperty('xpath', ConditionType.EQUALS, test_object_xpath) ``` Retrieve row List object:
List<WebElement> table_rows = table_element.findElements(By.tagName('tr'))

Cycle through each of the table rows and retrieve column information:

for(current_row = 0; current_row < table_row_count; current_row++) {

   table_columns = validate_table_columns(table_rows.get(current_row), expected_column_count, current_row, column_match_condition)

}

.....

public List<WebElement> validate_table_columns(WebElement table_row, int expected_column_count, current_row, String column_match_condition = "EQUALS") {


  List<WebElement>table_columns;
  List<WebElement>header_columns;

  table_columns = table_row.findElements(By.tagName('td')) // <---This is where the Stale Element error occurs
		
  header_columns = table_row.findElements(By.tagName('th'))

  table_column_count = table_columns.size() + header_columns.size()

Hey @kevin.jackey

We can fix this. And by the way, smartwait is not a solution.

  1. Where in the code is your test failing (I found it by editing your post)

  2. What is the precise error message?

  3. What technology is being used to retrieve the table row data? Be as specific as you can please.

Thanks!!

Not sure how to answer #3, other than citing the code in my earlier example where we’re pulling the table rows into a list element

List<WebElement> table_rows = table_element.findElements(By.tagName('tr'))

and then using the tablerow.findElements shown - is there another way to answer that you might be looking for?

Precise error message is as follow:

Test Cases/01-Rgsn/02 - UI/02-Menu/02-Exec Rpts/Digital Experience Overview FAILED.
Reason:
org.openqa.selenium.StaleElementReferenceException: stale element reference: stale element not found in the current frame
  (Session info: chrome=126.0.6478.56)
For documentation on this error, please visit: https://www.seleniumhq.org/exceptions/stale_element_reference.html
Build info: version: '3.141.59', revision: 'e82be7d358', time: '2018-11-14T08:25:53'
System info: host: 'BTT0247', ip: '192.168.0.108', os.name: 'Windows 10', os.arch: 'amd64', os.version: '10.0', java.version: '1.8.0_282'
Driver info: com.kms.katalon.selenium.driver.CChromeDriver
Capabilities {acceptInsecureCerts: true, browserName: chrome, browserVersion: 126.0.6478.56, chrome: {chromedriverVersion: 126.0.6478.55 (7616ff175414..., userDataDir: C:\Users\KEVINJ~2\AppData\L...}, fedcm:accounts: true, goog:chromeOptions: {debuggerAddress: localhost:51257}, javascriptEnabled: true, networkConnectionEnabled: false, pageLoadStrategy: normal, platform: WINDOWS, platformName: WINDOWS, proxy: Proxy(), setWindowRect: true, strictFileInteractability: false, timeouts: {implicit: 0, pageLoad: 300000, script: 30000}, unhandledPromptBehavior: dismiss and notify, webauthn:extension:credBlob: true, webauthn:extension:largeBlob: true, webauthn:extension:minPinLength: true, webauthn:extension:prf: true, webauthn:virtualAuthenticators: true}
Session ID: 6d0ff57f9de26c78ee4b1675f0e7067d
*** Element info: {Using=tag name, value=th}
	at org.openqa.selenium.remote.http.W3CHttpResponseCodec.createException(W3CHttpResponseCodec.java:187)
	at org.openqa.selenium.remote.http.W3CHttpResponseCodec.decode(W3CHttpResponseCodec.java:122)
	at org.openqa.selenium.remote.http.W3CHttpResponseCodec.decode(W3CHttpResponseCodec.java:49)
	at org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:158)
	at org.openqa.selenium.remote.service.DriverCommandExecutor.execute(DriverCommandExecutor.java:83)
	at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:552)
	at com.kms.katalon.selenium.driver.CChromeDriver.execute(CChromeDriver.java:19)
	at org.openqa.selenium.remote.RemoteWebElement.execute(RemoteWebElement.java:285)
	at org.openqa.selenium.remote.RemoteWebElement.findElements(RemoteWebElement.java:204)
	at org.openqa.selenium.remote.RemoteWebElement.findElementsByTagName(RemoteWebElement.java:281)
	at org.openqa.selenium.By$ByTagName.findElements(By.java:312)
	at org.openqa.selenium.remote.RemoteWebElement.findElements(RemoteWebElement.java:177)
	at org.openqa.selenium.support.events.EventFiringWebDriver$EventFiringWebElement.lambda$new$0(EventFiringWebDriver.java:404)
	at com.sun.proxy.$Proxy13.findElements(Unknown Source)
	at org.openqa.selenium.support.events.EventFiringWebDriver$EventFiringWebElement.findElements(EventFiringWebDriver.java:504)
	at org.openqa.selenium.WebElement$findElements$0.call(Unknown Source)
	at btt_portal.Widgets.validate_table_columns(Widgets.groovy:665)
	at btt_portal.Widgets$validate_table_columns$1.callCurrent(Unknown Source)
	at btt_portal.Widgets.validate_table_populated(Widgets.groovy:732)
	at btt_portal.Widgets.validate_table_populated(Widgets.groovy)
	at btt_portal.Widgets$validate_table_populated$0.call(Unknown Source)
	at btt_portal.Validate_Std_Pg_Widget.check_page_table_functionality(Validate_Std_Pg_Widget.groovy:102)
	at btt_portal.Validate_Std_Pg_Widget.check_page_table_functionality(Validate_Std_Pg_Widget.groovy)
	at btt_portal.Validate_Std_Pg_Widget$check_page_table_functionality.call(Unknown Source)
	at Digital Experience Overview.run(Digital Experience Overview:180)
	at com.kms.katalon.core.main.ScriptEngine.run(ScriptEngine.java:194)
	at com.kms.katalon.core.main.ScriptEngine.runScriptAsRawText(ScriptEngine.java:119)
	at com.kms.katalon.core.main.TestCaseExecutor.runScript(TestCaseExecutor.java:448)
	at com.kms.katalon.core.main.TestCaseExecutor.doExecute(TestCaseExecutor.java:439)
	at com.kms.katalon.core.main.TestCaseExecutor.processExecutionPhase(TestCaseExecutor.java:418)
	at com.kms.katalon.core.main.TestCaseExecutor.accessMainPhase(TestCaseExecutor.java:410)
	at com.kms.katalon.core.main.TestCaseExecutor.execute(TestCaseExecutor.java:285)
	at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:144)
	at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:135)
	at com.kms.katalon.core.main.TestCaseMain$runTestCase$0.call(Unknown Source)
	at TempTestCase1718648383530.run(TempTestCase1718648383530.groovy:25)

To be clear, you’re pulling the rows from the DOM element. I suspect what’s happening is, the table has been rebuilt in accordance with fresh (i.e. newer) data arriving from the server. Your problem then becomes, any references to it (or it TDs/TRs) are now gone (down a black hole, never to be seen again).

We can of course, spend a bunch of time going over your code layout, hoping to improve the chances that you’re dealing with a table that is complete and ready for testing. But that’s not the best way just yet… first we need to ensure a robust means of waiting for the table to be completed. Hence my question…

When you click the button and the server responds with the table data, a call goes out to the server to fetch the data. I want to know what that code looks like. It could be a fetch call, jQuery, any old XHR stuff…

If it happens to be jQuery, I can give you an API that deals specifically with waiting for that…

It’s xhr as best as I can tell, here are the calls made immediately after the trigger is clicked:

image

What happens if you type jQuery into the console?

If you get “Uncaught ReferenceError” or similar, you’re going to need something that proves the data is finished loading – that can be anything at all that is visible in the UI or present in the DOM that you can detect from your Groovy code.

Is it possible in advance to know how many rows will be returned?

I get the following, does this help?

However I get that same thing right after I push the button and I see the graph loading as well

Don’t know how many rows will be returned, but wondering if there’s a way to loop a row counter over a number of iterations and see when it stabilizes, would that work?

Ultimately not knowing Groovy table handling very well, nor what specifically is triggering this one issue I’m not sure what the best approach might be.

Hopefully, yes. Sit tight…

In JavaScript, you can use jQuery.active to find out if jQuery is “busy”. The property returns true while jQuery is busy (perhaps, as in your case, it’s waiting on a promise to return data from an ajax/xhr call). So, when it returns false, it’s not busy.

To be as certain as possible that ALL jQuery calls are completed, sometimes you might want to be extra paranoid and call it twice, perhaps even three times, separated by WebUI.delay(1) each time.

The following is my jsWaitAjax() method. Add it to your own library of utilities (I assume you have one kicking around).

  /**
   * Wait for ajax (jQuery.active) to complete for maximum <code>timeout</code> seconds.
   */
  static void jsWaitAjax(int timeout = TIMEOUT_SECONDS) {
    // use a closure for the wait loop
    Closure doWaitAjax = {
      boolean active = true
      int count = 0
      String js = "return jQuery.active;"
      while(active) {
        count++
        if(count > timeout) {
          throw new StepErrorException('jsWaitAjax: Ajax is taking too long!')
        }
        comment("doWaitAjax: " + count)
        active = (boolean) WebUI.executeJavaScript(js, null)
        Thread.sleep(AJAX_DELAY_MILLISECONDS)
      }
    }

    doWaitAjax()

    comment("jsWaitAjax complete.")
  }

Somewhere, you’ll need to define:

  1. TIMEOUT_SECONDS – mine is set to 10 for Test Cases, and 30 for Suites.
  2. AJAX_DELAY_MILLISECONDS – try 50 or 100 or whatever.

comment() is merely a wrapper over the standard WebUI.comment() method – you can delete it or fix it as you wish.

For your code, add a call to jsWaitAjax() after you have done all your usual “waits” that you were doing before. And like I said, maybe add a couple more:

// your code here, then...

jsWaitAjax()
WebUI.delay(1)
jsWaitAjax()

// now access the table rows/data...

The only thing I can think of that might screw this up is if the developers are NOT using jQuery behind that button.

Let me know.

1 Like

Certainly, that’s on my mind too. However, that’s the kind of code that once it’s working, you’ll forget about it. Then (maybe months later) you start testing on another server/network and it starts to fail – that’s kind of thing that would be a nightmare to debug, so, best avoided. I’d classify that as “doable, but over engineered”.

I avoid it completely. I go straight to JavaScript every time. It’s always live ← no stale elements, ever.

Yeah, it’s still failing even with the Ajax check, Ajax check passes every time.

If you were going to do a table check in JavaScript (thus no stale elements) how would you suggest doing it? Perhaps something like this?