False error on waitForElementAttributeValue

[Clayton_Friesen]

I have a test case that reports a failure when it should not be and I’m not finding why. The test is simple, I have an input screen with multiple panels that are the same. Each panel has inputs and a button to clear the inputs that I am testing.

My test walk through:

1. I am setting the text to each input
2. I click on the ‘clear’ button to show the confirmation dialog
3. I click on the confirmation ‘OK’ button to
4. JS runs to clear the inputs
5. I am testing that the inputs have a blank value

This test falsely fails about 90% of the time. Sometimes it works, but most of the time it fails. The screen being tested never changes, the inputs are always there and the inputs are clear after the JS runs to clear them. The error when it fails is always “stale element reference: element is not attached to the page document”. The error does not always occur on the same waitForElementAttributeValue line. The error randomly fails on either the 8th, 9th or 10th waitForElementAttributeValue line:

WebUI.waitForElementAttributeValue(findTestObject("input/input08", ["idx":1]), "value", "", 2)
WebUI.waitForElementAttributeValue(findTestObject("input/input09", ["idx":1]), "value", "", 2)
WebUI.waitForElementAttributeValue(findTestObject("input/input10", ["idx":1]), "value", "", 2)

What would cause this error?

My test case:
WebUI.navigateToUrl(GlobalVariable.baseurl + ‘/input/’ + GlobalVariable.editid + ‘/edit#!/packages’)
WebUI.verifyElementAttributeValue(findTestObject(‘common/meta_application’), ‘data-screenid’, ‘input.packages’, 20)
WebUI.waitForElementNotVisible(findTestObject(“common/spinner”), 20)

WebUI.setText(findTestObject("input/input01", ["idx":1]), "input01")
WebUI.setText(findTestObject("input/input02", ["idx":1]), "input02")
WebUI.setText(findTestObject("input/input03", ["idx":1]), "input03")
WebUI.setText(findTestObject("input/input04", ["idx":1]), "input04")
WebUI.setText(findTestObject("input/input05", ["idx":1]), "input05")
WebUI.setText(findTestObject("input/input06", ["idx":1]), "input06")
WebUI.setText(findTestObject("input/input07", ["idx":1]), "input07")
WebUI.setText(findTestObject("input/input08", ["idx":1]), "input08")
WebUI.setText(findTestObject("input/input09", ["idx":1]), "input09")
WebUI.setText(findTestObject("input/input10", ["idx":1]), "3210")
WebUI.setText(findTestObject("input/input11", ["idx":1]), "1,111.11")
WebUI.setText(findTestObject("input/input12", ["idx":1]), "2,222.22")
WebUI.setText(findTestObject("input/input13", ["idx":1]), "3,333.33")

WebUI.click(findTestObject("input/clearbtn", ["idx":1]))
WebUI.waitForElementVisible(findTestObject("common/confirmationdialog"), 2)
WebUI.click(findTestObject("common/confirmationdialogok"))

WebUI.waitForElementAttributeValue(findTestObject("input/input01", ["idx":1]), "value", "", 2)
WebUI.waitForElementAttributeValue(findTestObject("input/input02", ["idx":1]), "value", "", 2)
WebUI.waitForElementAttributeValue(findTestObject("input/input03", ["idx":1]), "value", "", 2)
WebUI.waitForElementAttributeValue(findTestObject("input/input04", ["idx":1]), "value", "", 2)
WebUI.waitForElementAttributeValue(findTestObject("input/input05", ["idx":1]), "value", "", 2)
WebUI.waitForElementAttributeValue(findTestObject("input/input06", ["idx":1]), "value", "", 2)
WebUI.waitForElementAttributeValue(findTestObject("input/input07", ["idx":1]), "value", "", 2)
WebUI.waitForElementAttributeValue(findTestObject("input/input08", ["idx":1]), "value", "", 2)
WebUI.waitForElementAttributeValue(findTestObject("input/input09", ["idx":1]), "value", "", 2)
WebUI.waitForElementAttributeValue(findTestObject("input/input10", ["idx":1]), "value", "", 2)
WebUI.waitForElementAttributeValue(findTestObject("input/input11", ["idx":1]), "value", "", 2)
WebUI.waitForElementAttributeValue(findTestObject("input/input12", ["idx":1]), "value", "", 2)
WebUI.waitForElementAttributeValue(findTestObject("input/input13", ["idx":1]), "value", "", 2)

Object Example: (only ng-model changes for each)
Selection Method: Attributes
Object’s Properties: xpath = //div[@name=“packages”]/div[${idx}]//input[@ng-model=“input05”]
Detect Object: checked

Error:
Stack trace: com.kms.katalon.core.exception.StepFailedException:
Unable to verify if object ‘Object Repository/input/input08’ has attribute ‘value’ with value ‘’
Root cause: org.openqa.selenium.StaleElementReferenceException: stale element reference: element is not attached to the page document
Session info: chrome=70.0.3538.110
Driver info: chromedriver=2.43.600210

[Mate_Mrse]

Hmm. I’ll just throw some ideas and see if anything sticks

I would experiment with ways of refreshing the DOM somehow, after the WebUI.click(findTestObject("common/confirmationdialogok")) line. Maybe with some of these:

WebUI.refresh()
WebUI.waitForPageLoad()

[Russ_Thomas]

Here’s the deal, as I understand it: your step 4 “JS runs to clear the inputs” happens and then the page/elements go through a stale/rebuild cycle. The problem is, your following lines of test code happen asynchronously in the driver not knowing when your step 4 is done and finished. That is why you get potentially different results on each run.

Note: It’s not clear to me (but it should be clear to you) if angular (JS) is doing a server round trip at step 4. If it is, then you might consider inserting https://docs.katalon.com/katalon-studio/docs/webui-wait-for-angular-load.html at that point.

In a nutshell, if you find out when step 4 is complete, you’re on your way to fixing your issue.

Good luck (and report back with your progress, please).

[Clayton_Friesen]

When the JS runs to clear the values, it is setting the value of the bound object. It is not doing any server round trip with this. I tried the wait for angular, unfortunately that did not solve the problem. Following on the async theory, I added a wait for 1 second after the OK confirmation. This seems to have remove the problem, I ran it 4 times in a row without issue. Thanks.

[Russ_Thomas]

@Clayton_Friesen

This seems to have remove the problem, I ran it 4 times in a row without issue.

I’m not convinced. If you have an unpredictable problem (which is how you described the nature of the issue) you cannot fix its cause using a (predictable) fixed metric like “one second”. What you’ve done is mask the symptoms in the hope that the mask doesn’t “slip” one day and the problem reveal itself again.

Find a state that appears at the true end of your step 4. That’s the place/time to be looking/waiting for.

Good luck.

[Clayton_Friesen]

No. (responses must be 20 characters long)

[Brandon_Hein]

I totally agree with Russ here, adding an explicit 1 second wait is simply masking your issue. If you want to definitively solve the issue, you need a more custom solution (a wait condition, really). Generally speaking, if you have an intermittent, unpredictable error, it’s from one of two things:

1.) An intermittent, unpredictable error in the application you are testing (doesn’t seem likely in your case).

2.) A timing issue, where you are trying to interact with an element on the page when there’s still some JS or back-end processing going on, OR you have reference to an element that has since changed AFTER the JS has finished.

Option 2 seems most likely in your case, seeing that you’re getting a StaleElementReferenceException. This usually happens when you have reference to an element that has since changed in some way, and you try to do something with it after the fact. Either you properly wait for the JS to finish, or you re-locate the element, then try to do something with it.

Or you can continue to ignore the advice that you’ve asked for, and hope that your 1 second explicit wait doesn’t fail you

[Russ_Thomas]

@Brandon_Hein

I totally agree with Russ here, adding an explicit 1 second wait is simply masking your issue.

Thanks, Brandon.

And in the interests of extending our knowledge-base, here I’d like to add to what we’ve both said (my thinking is, future visitors will find this thread and appreciate the knowledge shared).

Quoting the OP again:

@Clayton_Friesen

… Following on the async theory, I added a wait for 1 second after the OK confirmation.

(Emphasis added). Clayton is referring back to my post where I say:

@Russ_Thomas

The problem is, your following lines of test code happen asynchronously in the driver not knowing when your step 4 is done and finished.

(Original emphasis untouched).

The point I want to bring out is about “asynchronicity”, which Clayton refers to as a “theory”.

There is no “theory” here. Webdriver code (our Groovy code, in essence) is running asynchronously with respect to the code executing in the browser. It is the challenge we face as automation test engineers to coerce the webdriver into cooperating with the browser code. If you (we) don’t do that with care and due diligence, Katalon is capable of running the entire test case before the browser has even rendered the page (errors willing, of course).

So we use “waits”.

We wait for X to appear,
We wait for Y to be populated,
We wait for Z to disappear, etc.

And we do that because …

• test code runs asynchronously and at warp factor 9 compared to browser code

if you’ll allow the metaphor.

Finally, back to the OP’s issue: I’d start by adding a wait for one of those inputs becoming blank/empty. I’d repeat that probably for a few more of them. I’d add more waits for any other changes that may occur on the page at the same time. Then, maybe, just maybe, I’d add a second or two as a fixed delay right at the end. Fact is, you won’t need a fixed delay if you get the criteria for the wait-for-states correct.

[Brandon_Hein]

Fact is, you won’t need a fixed delay if you get the criteria for the wait-for-states correct.

This is the crux of it. It is (in my opinion) blasphemous to have any hard waits in your code. You can always find a proper wait condition with enough diligence, and it will serve you tremendously if you make it a mantra.

That being said, I do use thread sleeps to confirm that I indeed have a timing issue. But you should always (again, in my opinion) go back and do the legwork to observe how the DOM is actually changing; your code will be water-tight. Otherwise, things like code changes, or even a load on your server/application that is more than usual will extend your wait time from 1 second to 5 seconds, and bingo, your test fails and you can’t understand why your wait fell short.

One last note. If you make it a habit to use explicit waits to “solve” all of your problems, you’re incrementally adding to your overall test execution time. One second here or there doesn’t seem like an issue until you have that all over the place, and your script takes 5 minutes to run instead of 30 seconds…

[Brandon_Hein]

OP: * fixes engine with duct tape *
engine: * breaks again *
OP:

(I’m so sorry, I couldn’t resist… , he was just so dismissive about all this…)

[Clayton_Friesen]

Russ, I apologize for my short response. You took the time to respond, it was helpful and I do appreciate it. I should not have done that.

With my issue, to find a proper solution that is not a timed delay, I am not sure what that is. My current understanding is:

1. The error “org.openqa.selenium.StaleElementReferenceException: stale element reference: element is not attached to the page document” means that the input I’m trying to test can not be found. The input is on my page before and after my JS runs. My inputs are there when the page loads and never hide or remove.

2. My test is using “waitForElementAttributeValue” with a timeout of 2 seconds. This test should wait for 2 seconds to see that the value has changed to the expected result. It is failing because the input can not be found and not because the value is not the expected result.

You mentioned that my error can occur when an element has changed. I am using AngularJS v1 so the “class” attribute on my input element does change from before my JS runs to after. Example:

Before JS runs:
<input ng-model="input08" maxlength="20" class="form-control id-form-data ng-pristine ng-untouched ng-valid ng-not-empty ng-valid-maxlength" autocomplete="off" style="width:182px;" type="text" value="">

After JS runs (“class” attribute has changed, ng-not-empty => ng-empty)
<input ng-model="input08" maxlength="20" class="form-control id-form-data ng-pristine ng-untouched ng-valid ng-empty ng-valid-maxlength" autocomplete="off" style="width:182px;" type="text" value="">

You are recommending that after my JS runs, I need a wait for a recognized state before testing my value changes. Is there a Katalon method that will refresh the dom/elements so my objects are found?

Bit of a head scratcher. My first six “waitForElementAttributeValue” always test success.

[Russ_Thomas]

@Clayton_Friesen

Russ, I apologize for my short response. You took the time to respond, it was helpful and I do appreciate it. I should not have done that.

No need to apologize. Let’s move on and get you working (and not just working, working robustly so that you can satisfy yourself your code is doing exactly what you expect, when you expect it to do it).

So. Deep breath…

Gaining control over asynchronous code is complex… but not complicated (there is a difference!).

Statement: your control of interest has been destroyed and then rebuilt. Period. Suck it up. It’s happening. (If it isn’t, I’ll eat my hat. And yours.)

Your framework (ng, probably) is destroying the control, then constructing a new one, in the same position in the dom fooling you (and everyone else) into thinking all it’s done is change a class ← not true!

What you want to do, is make another test object “live” in memory – see: https://docs.katalon.com/katalon-studio/docs/creation-of-test-object-in-object-repository-in-runtime.html and https://docs.katalon.com/katalon-studio/docs/manage-test-object.html#test-objects-in-scripting-view - it’s likely you can use the same criteria as your stored test object to construct it (whether that’s css or xpath, etc).

Do that for each control you want to access at your (now infamous) step 4 from your original post.

Go easy on yourself - build a mock test that merely targets one control (and clicks the button to trigger the state in question). Increase the wait timeout to something silly like 60, and prove to yourself, it doesn’t wait that long, once the in-memory test object is found, the test case moves on.

def ctrl = makeTO("whatever css")
WebUI.waitForElementPresent(ctrl, 60)
WebUI.waitForElementAttributeValue(ctrl, "value", "", 60)
WebUI.comment("It works!")

Aside: Strictly, we should be checking for a property value (not an attribute). But Katalon seems to muddy the waters here so try attribute first. (And for all I know, angular might muddy it even further.) Try something like the above, and lets see how you get on.