Stale Element all of a sudden

Product Forums Katalon Studio
,
1

Hi all!

So I am having an odd problem. I was at work and was writing up my Katalon Script for some testing, and everything ran smoothly. I prepared the things I still had to do, and before I left, I ran my script one more time on my work laptop, and it ran perfectly. I backed up the project to my USB.

I am home now, transfer the project to my home pc, and set it to run. I didn’t change anything

Not I have an error popping up at different times!? The error is in the screenshot below. We made no changes to this part of the web software today or whilst I was on my way home. It just… broke? Why? I heard of dynamically creating objects in code, however this script is now 596 lines, and is going to be much more when its done.

I also took @Russ_Thomas’s advice and to use WebUI.waitForElementVisible for all objects to wait for it to load.

Side question: Does Katalon become slower the longer your scripts are? I noticed today at work that Katalon was running very slow compared to yesterday.

I took the advice of someone, that told me to set my “true” statements to “false” for “selectOptionByValue” which sped it up a lot, but now for some reason its so slow, on my work pc and at home (different script). Slower than usual

import static com.kms.katalon.core.checkpoint.CheckpointFactory.findCheckpoint
import static com.kms.katalon.core.testcase.TestCaseFactory.findTestCase
import static com.kms.katalon.core.testdata.TestDataFactory.findTestData
import static com.kms.katalon.core.testobject.ObjectRepository.findTestObject
import static com.kms.katalon.core.testobject.ObjectRepository.findWindowsObject
import com.kms.katalon.core.checkpoint.Checkpoint as Checkpoint
import com.kms.katalon.core.cucumber.keyword.CucumberBuiltinKeywords as CucumberKW
import com.kms.katalon.core.mobile.keyword.MobileBuiltInKeywords as Mobile
import com.kms.katalon.core.model.FailureHandling as FailureHandling
import com.kms.katalon.core.testcase.TestCase as TestCase
import com.kms.katalon.core.testdata.TestData as TestData
import com.kms.katalon.core.testng.keyword.TestNGBuiltinKeywords as TestNGKW
import com.kms.katalon.core.testobject.TestObject as TestObject
import com.kms.katalon.core.webservice.keyword.WSBuiltInKeywords as WS
import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI
import com.kms.katalon.core.windows.keyword.WindowsBuiltinKeywords as Windows
import internal.GlobalVariable as GlobalVariable
import org.openqa.selenium.Keys
import org.openqa.selenium.chrome.ChromeDriver as Keys

import org.openqa.selenium.chrome.ChromeDriver as ChromeDriver
import org.openqa.selenium.chrome.ChromeOptions as ChromeOptions
import com.kms.katalon.core.webui.driver.DriverFactory as DriverFactory
import org.openqa.selenium.Dimension
import org.openqa.selenium.Point
import org.openqa.selenium.WebDriver
import org.openqa.selenium.chrome.ChromeDriver
import org.openqa.selenium.chrome.ChromeOptions
import com.kms.katalon.core.testobject.ConditionType

firstName = 'Error'
surname = 'Test4'
email = 'ErrorTest4@gmail.com'
passportNumber = 'ErrorTest4!'
fileLocation = 'C:\\Users\\Thomas\\Pictures\\random\\random.jpg'

// open normal Chrome browser on the left side
WebDriver normalChrome = openChromeBrowserPlain()

resizeHorizontalHalfLocateLeft(normalChrome)

DriverFactory.changeWebDriver(normalChrome)

WebUI.navigateToUrl('https://inscape-operations-staging.stratusolvecloud.com/App/Student/Workflow/NewEnrolment/2022')

WebUI.waitForPageLoad(10)

// open incognito Chrome browser on the right side
ChromeOptions options = new ChromeOptions()

options.addArguments('--incognito')

WebDriver incognitoChrome = new ChromeDriver(options)

resizeHorizontalHalfLocateRight(incognitoChrome)

DriverFactory.changeWebDriver(incognitoChrome)

WebUI.navigateToUrl('https://inscape-connect-staging.stratusolvecloud.com/UserManagement/login/')

WebUI.waitForPageLoad(10)

WebUI.delay(2)

DriverFactory.changeWebDriver(normalChrome)

// Basic Personal Information Step
// Maximizes the Window
//WebUI.maximizeWindow()

// Enters the First Name

WebUI.waitForElementVisible(findTestObject('Enrolment/Page_Connect - Operations - Student Enrolment/input_Welcome to INCONNECT - our online stu_0eecac'), 10)

WebUI.setText(findTestObject('Enrolment/Page_Connect - Operations - Student Enrolment/input_Welcome to INCONNECT - our online stu_0eecac'), 
    firstName)

// Enters the Last Name

WebUI.waitForElementVisible(findTestObject('Enrolment/Page_Connect - Operations - Student Enrolment/input_Welcome to INCONNECT - our online stu_d2ddfb'), 10)

WebUI.setText(findTestObject('Enrolment/Page_Connect - Operations - Student Enrolment/input_Welcome to INCONNECT - our online stu_d2ddfb'), 
    surname)

// Enters the Cell Phone Number: This is not validated, so it can always be the same

WebUI.waitForElementVisible(findTestObject('Enrolment/Page_Connect - Operations - Student Enrolment/input_land Islands_cell'), 10)

WebUI.setText(findTestObject('Enrolment/Page_Connect - Operations - Student Enrolment/input_land Islands_cell'), 
    '0790000000')

// Enters the Email Address: This is validated, it has to be unique

WebUI.waitForElementVisible(findTestObject('Enrolment/Page_Connect - Operations - Student Enrolment/input_Valid_c3'), 10)

WebUI.setText(findTestObject('Enrolment/Page_Connect - Operations - Student Enrolment/input_Valid_c3'), 
    email)

// Country of Origin Selection of South Africa

WebUI.waitForElementVisible(findTestObject('Enrolment/Page_Connect - Operations - Student Enrolment/select_Country of OriginSouth AfricaUnited _72b9c5'), 10)

WebUI.selectOptionByValue(findTestObject('Enrolment/Page_Connect - Operations - Student Enrolment/select_Country of OriginSouth AfricaUnited _72b9c5'), 
    '1', false)

// ID Type Selection of Passport Number

WebUI.waitForElementVisible(findTestObject('Enrolment/Page_Connect - Operations - Student Enrolment/select_--Select ID Type--RSA ID NumberPassp_e4a2f7'), 10)

WebUI.selectOptionByValue(findTestObject('Enrolment/Page_Connect - Operations - Student Enrolment/select_--Select ID Type--RSA ID NumberPassp_e4a2f7'), 
    '2', false)

// Enters the Passport Number: This is validated, it has to be unique

WebUI.waitForElementVisible(findTestObject('Enrolment/Page_Connect - Operations - Student Enrolment/input_Valid_c7'), 10)

WebUI.setText(findTestObject('Enrolment/Page_Connect - Operations - Student Enrolment/input_Valid_c7'), 
    passportNumber)

1_error
1_error1046×908 43.2 KB

2_dropdown
2_dropdown1074×608 37.7 KB

3_HTML
3_HTML2145×698 86.4 KB

1 Like
2

I don’t see it.

Not generally, no. What can cause scripts to run slower include:

  • Smart wait. Make sure you turn this off.
  • The application under test is slower than usual
  • Your network connection is slower than usual

Generally speaking, StaleElementReferenceExceptions indicate a timing issue. It happens when you have a reference to a web element that no longer exists on the page, or has changed in some way before you try and use it. The only way to solve these permanently is to figure out which statements are causing the exception, and to add wait conditions around those to make sure the page isn’t loading before you try and do work. If you can share the error we may be able to help you do this further.

3

Apologies, I took screenshot but never attached. I attached now

Ok so you suggest adding a WebUI.delay() into the equation where I experience this?

Furthermore, can one add videos onto posts? I think it could help for some circumstances

4

No problem, I see it now.

Yes, with an extremely important caveat. WebUI.delay(), or any hard wait like that, should only be used to diagnose the problem. It is not a permanent solution. It’s useful to put in a wait of say, 5 seconds, run the script several times, and if you don’t see the exception at that point, you know you have a wait issue. But we kinda already know that.

My guess at what’s happening here:

You have a wait for visible, which is fine, but I’m guessing that the element is visible the whole time, but the options below that element (it’s a <select> element I’m fairly sure) may be changing as you are trying to set a value.

Can you share the HTML of the dropdown (and its options) that you are working with at that step?

I’m pretty sure you can just embed a video here directly. I’ve seen others do it. Not sure if there are any special steps. You may need to use the “Upload” button when typing out your post.

5

Sure thing. I added two screenshots to the original post, 1 with the HTML and 1 with the picture of the dropdown.

Note, that sometimes I am getting an error on selecting the drop down, and othertimes getting an error of setting the Text of the ID/Passport number

6

Not surprising. Wait/timing issues are intermittent by nature.

7

You did. You used a completely different network. That’s like changing everything.

With Brandon’s help, you’re on the right track. A move you’re making before you access the dropdown is causing the <select> to (re)populate. Your test steps arrive at the selectOptionByValue() call before it’s either ready or even present.

Looking at your waitForElement* code, you might benefit from making those a single method call, then you can call the method anywhere and use up some time benignly while the browser is busy – it never hurts to wait, your test case code is moving like lightning compared to how the browser responds.

void all_my_waits() {
  WebUI.waitForElementVisible(A, ...) 
  WebUI.waitForElementVisible(B, ...)
  WebUI.waitForElementVisible(C, ...)
  ...
  WebUI.waitForElementVisible(Z, ...)
  // for links, buttons, etc
  WebUI.waitForElementClickable(btn, ...)

}

Some way down the road, you’ll be asking a question to which either Brandon or I will respond with, “You should use a POM”. You’re not quite their yet, but I can assure you it’s coming :sunglasses:

8

So true :smiley:

Specifically regarding your issue:

Here, you are waiting for the dropdown itself (the <select> element) to be visible before selecting an option. This is not the same as waiting for the <option> to be visible.

You can definitely go down the route of waiting for each and every specific element to be “ready” before interacting with it, but to me it seems like there is something else going on with your page. This is apparent because you are seeing stale elements in “random” places, not just the one in question.

Is there any sort of visual indication of your page doing work after you take action? This can often take the form of:

1.) An “overlay” across the whole page. This usually looks like a big gray window covering your page while it’s loading.
2.) A “spinner” somewhere on the page, indicating that some work is being done on the page.
3.) A “progress bar”, often at the top or the bottom of the page, that fills incrementally while the page is doing something.

If your web page has something like this, you may be better served waiting for this to not be visible between each action, instead of waiting for specific elements to be ready.

1 Like
9

Thank you for responding. I do not quite understand what you are saying though. What do you suggest that I Wait For instead of the <select>?

To this point, yes the Drop Down in question is actually dynamically created. If you take a look at the 2nd screenshot, you see that the one drop down is “South Africa”. By selecting South Africa, the second drop down “–Select ID Type–” gets dynamically created to show the options “RSA ID Number” and “Passport Number”. Before selecting the country, this dropdown doesn’t populate any options yet.

I hope that this answers you question.

I am still confused though. The waitForElementVisible is something that I think I am not understanding fully. When and HOW should I be using it, in general and in this specific case?

Well I know what I will be researching after I have my full Katalon script for this test case completed xD.

What test object should I be Waiting for then? The specific selection of the drop down? If so, how do I create the test object of this selection, in regards to the HTML maybe?

10

I am looking at this. I am not sure whats going on though? I am trying to make sense of it :see_no_evil: but I do not understand what is going on here.

Back to my other question though:

How do you suggest I do this properly then? I thought that adding waitForElementVisible was the correct method :see_no_evil: I literally added this before EVERY call in my script as you mentioned before that its beneficial to have the waits for elements to be interactable/visible

11

You answered this yourself:

So it’s a dynamic form, which is pretty common these days. What Russ is saying is that you are doing work in your script that is causing the list to re-populate its options (or causing the list to appear in the first place). Our guess is that, sometimes, your script is moving quickly enough, and your page slowly enough, that something like this happens:

1.) Set some value in another field. This causes your drop down options to start repopulating, which takes time.
2.) Within this timeframe, you try and select the desired option in the dropdown, but the page is still repopulating these options. This is where the stale element exception happens.

However, I’m still not sure if your page has a “global” indication that it’s doing this work. Like I asked previously, is there an overlay/spinner/progress bar that appears on the page after you take action, and the form is being rebuilt? If so, that’s actually really great; you can solve all/most of your timing issues with 1 wait condition.

You are understanding it correctly. You are waiting for the element to be “visible” on the page. Now there’s some serious debate on what “visible” means, but in this context, it means that the element:

1.) Exists in the HTML.
2.) Does not have an @hidden attribute, or an @style=display: none; attribute, or any other attribute which causes the browser to not render that element.

You will, and I can tell that you actually care about getting this stuff right (based on how you write your topics), so I have no doubt you’ll become a great engineer. What we’re referring to as the “POM” is a design pattern (a general approach to how to structure your test automation project) that will resolve a lot of these smaller issues you are having. We will be glad to go through that with you when the time comes.

Yes, exactly. You need to wait for the <option> you are trying to select, not the dropdown itself. You can create a Test Object with the following xpath, then call your waitForElementVisible() on this instead:

//select[./option[text()='--Select ID Type--']]/option[@value='2']

12

Russ is saying that you could create a custom keyword that you can call from your scripts that waits for all sorts of things in one go (instead of waiting for each element you are working with along the way). This may be a good idea for your use case, but again, if your page has some sort of global indicator that the page is loading, an even better solution would be to wait for this indicator to disappear, signaling that your page is ready for further input.

13

If you can share a video of your page while you fill that form, like you said, it might help.

14

Ohhhh I see what you mean. Thank you. I am still learning here.

Nah there is no overall indication or processing that is visible. What I was thinking is to select the dropdown for “South Africa” FIRST, then fill in some other fields, then come back to second drop down, and as you say, select the option as the test object, and not the drop down itself. What would the WebUI look like for this? Would it still be WebUI.selectOptionByValue(findTestObject(–Option of the drop down–)) or what?

Ok thank you! This clears my understanding up.

THANK YOU! I will def start looking into it and see what I can find. I am doing this as a pet project for my work to show them how valuable it can be so I want to make it as perfect as possible.

Thank you for this. I have all this written down to do at work tomorrow. I hope that my systems will work perfectly then. I will just need to go through all the other dropdowns in my test case and fix the issue to select the --Option-- rather than the dropdown itself and setting a value to that. Please just inform me on my previous question of how I actually structure the WebUI to choose the option?

I will try to do that. When I just ran the script it actually worked. So this is what I am saying, sometimes it works and sometimes it doesn’t :rofl: That will make it hard to show you the error

15

It’s not so much the error – we want to see “behavior”.

16

Right, we just want to see what your page does when filling out the form. No need to reproduce the error, we kinda already know that part.

17

The part where you are choosing the option is actually fine. You just want to change your wait condition to use that custom object I mentioned above:

WebUI.waitForElementVisible(findTestObject(path/to/my/custom/object'), 10)
WebUI.selectOptionByValue(findTestObject('Enrolment/Page_Connect - Operations - Student Enrolment/select_--Select ID Type--RSA ID NumberPassp_e4a2f7'), '2', false)
18

Right, it is pretty common for timing issues to behave erratically. But it’s extremely important to resolve issues like these, otherwise there’s not much point in automating your tests. They need to be reliable.

19

Ok I will upload. I will share the link for the google drive at the bottom of this post.

I will upload. I see now, as you will, that there is a quick little page load. I never noticed it because I wasn’t ever llooking for it.

I get that. I want to make sure that it works in EVERY case, on my pc or on a server

https://drive.google.com/file/d/1G4ISv5mmhcc0z1LpgiR09Y6dvGWqQu9G/view?usp=sharing

20

Since there seems to be a global interaction, how do I then get around this? :smiley: I am very keen to make this robust