Problem hand webelements with dynamic IDs

a.bach · November 19, 2020, 3:24pm

Hi,
it feels like I read the whole internet and did not find a solution for my problem. I also searched this forum but did not find my exact problem.

Katalon Studio Version: 7.7.2
OS Version: W10
Browser Version: Firefox 82.0.3

Steps to Reproduce:

I test our webapplication and run into problems when a right-click is involved. The context menu opens and the elements in this context menu do have dynamic ids.

Like this: CGAFynVaXC3p_ZutritteContainerDiv_AxCMenu_new
the cryptic part at the beginning is dynamic, the readable second part is always the same.

I tried to access this element with XPath and attributes (id contains … , xpath contains… ) to only look for the static part.
But I always get an ElementNotInteractableException error.

I also tried with javascript, like suggested in Katalon doc
https://docs.katalon.com/katalon-studio/docs/webui-execute-javascript.html#returns

Then I get some kind of compiling error when running the testscript

the error which is thrown:

Caused by: org.openqa.selenium.ElementNotInteractableException: Element could not be scrolled into view
Build info: version: ‘3.141.59’, revision: ‘e82be7d358’, time: ‘2018-11-14T08:25:53’
System info: host: ‘localhost’, ip: ‘127.0.0.1’, os.name: ‘Windows 10’, os.arch: ‘amd64’, os.version: ‘10.0’, java.version: ‘1.8.0_181’

From the dev team I got the hint that this menu is in an container, also with a dynamic ID.
I should first get the container ID and then navigate with xpath to the element I want to clics and do this in java.

Is this really the only way to handle this problem?

Russ_Thomas · November 19, 2020, 4:11pm

Possibly. But first, post some of the HTML so I can take a look at the problem more closely.

If this is a public website, let me have the URL.

a.bach · November 25, 2020, 3:46pm

Hi, thanks for the help and sorry for the late reply (in case you where wondering); I always thought I was close to the solution, but wasnt…

The site is not public, I will post an excerpt of it below because it is rather large.
It should represend the context menu on which I want to click this item
<tr class=“sub_item” id=“xaL1o5Fsh3Am_ZutritteContainerDiv_AxCMenu_new”>

I am able to get generate the Xpath to this item by now, but I am not able to click it.
I tried a few things I found in Katalon docs and in the forum but nothing works out. Is there a best practice for this?

Thx!

<div class="dhtmlxMenu_dhx_terrace_SubLevelArea_Polygon " dir="ltr" id="polygon_xaL1o5Fsh3Am_dhxWebMenuTopId" style="display: none;">
	<div style="position:relative;"/>
	<div style="position: relative; overflow:hidden;">
		<table class="dhtmlxMebu_SubLevelArea_Tbl" cellspacing="0" cellpadding="0" border="0">
			<tbody>
				<tr class="sub_item" id="xaL1o5Fsh3Am_ZutritteContainerDiv_AxCMenu_new">
					<td class="sub_item_icon">
						<img id="image_xaL1o5Fsh3Am_ZutritteContainerDiv_AxCMenu_new" class="sub_icon" src="/CLICS/img/menu/new.png">
						</td>
						<td class="sub_item_text">
							<div class="sub_item_text">New</div>
						</td>
						<td class="sub_item_hk">&nbsp;</td>
					</tr>
					<tr class="sub_item" id="xaL1o5Fsh3Am_ZutritteContainerDiv_AxCMenu_SetLDTPath">
						<td class="sub_item_icon">&nbsp;</td>
						<td class="sub_item_text">
							<div class="sub_item_text">Edit LDT path POE</div>
						</td>
						<td class="sub_item_hk">&nbsp;</td>
					</tr>
				</tbody>
			</table>
		</div>
		<div style="position:relative;"/>
	</div>

grylion54 · November 25, 2020, 9:10pm

Russ_Thomas will probably get back to you. I have never tried to click on the tr tag, but you can click on any of the td tags associated with it, which I do in web tables regularly. Also, you can use the tr tag you want, to create a path to the associated td tags, something like:

//tr[contains(@id,'ZutritteContainerDiv_AxCMenu_new')]/td[3]

Russ_Thomas · November 26, 2020, 12:32am

I don’t know the context in which you grabbed that HTML, but know this: you can’t (and absolutely should not) click on anything inside an element that is not displayed in the viewport. That said…

This will click on the <tr> you’re trying to reach. In support of @grylion54’s misgivings, whether that is meaningful or not is down to you:

String selector = 'table.dhtmlxMebu_SubLevelArea_Tbl tr[id$=ZutritteContainerDiv_AxCMenu_new]'
String js = 'document.querySelector("'+selector+'").click();'
WebUI.executeJavaScript(js, null)

Based on the limited HTML I’ve seen, that will work. If it doesn’t work, it will be because…

There’s more context needed to improve that selector string, or…
It’s not the <tr> that needs to be clicked.

To be certain about which element is waiting for a click, load the page in Firefox and view the element in DevTools. Whichever element is waiting for a click will have an event button which you can inspect. Here’s an example using this forum post:

Aside. That’s poorly written HTML, at least 10 if not 20 years out of date. Just saying.

a.bach · November 26, 2020, 6:06pm

Thx a lot for the help and attention! I am obviously new to this webstuff.

I copied the html from firefox inspector, wasnt sure if a screenshot would be sufficient. Here is a more complete screenshot. Actually the <tr> holds the click event. Does this look usable?

Russ_Thomas, I tried your approach, but get an Error: cannot execute
org.openqa.selenium.JavascriptException: TypeError: document.querySelector(…) is null

Maybe the problems are connected to the attempt clicking on the <tr>…?

This is my selector, I placed the element with the dyn ID in it.

document.querySelector(“table.dhtmlxMenu_SubLevelArea_Tbl tr[id=jWq5ZeuX4FzY_ZutritteContainerDiv_AxCMenu_new]”).click();

Russ_Thomas · November 27, 2020, 3:18am

Sorry, my mistake. The $= needs to be escaped: \$=

Try this:

String selector = 'table.dhtmlxMebu_SubLevelArea_Tbl tr[id\$=ZutritteContainerDiv_AxCMenu_new]'
String js = 'document.querySelector("'+selector+'").click();'
WebUI.executeJavaScript(js, null)

In future, please post the exact error from Groovy/Katalon or the browser console for JavaScript. A screenshot is good, too.

No. All visible elements are clickable. In your case, the developers are using the <tr> element as a delegate click handler for all elements in the row.

But that will include the dynamic part - try the new one.

By the way, “$=” means “ends with” so we are looking for an id which ends with “ZutritteContainerDiv_AxCMenu_new”.

And yes, that screenshot of the HTML is much better, thank you.

a.bach · November 27, 2020, 11:42am

thx again, much appriciated.

Also with escaping the $ I still get the same error.
I can use this command in Firefox console to get the element though
$(‘#tr[id$=ZutritteContainerDiv_AxCMenu_new]’)

calling the document.queryselector actually returns null

Here the Katalon console, I have some println of my values in there

2020-11-27 12:32:11.778 DEBUG testcase.                                - 18: selector = "table.dhtmlxMenu_SubLevelArea_Tbl tr[id$=ZutritteContainerDiv_AxCMenu_new]"
2020-11-27 12:32:11.778 DEBUG testcase.                                - 19: js = "document.querySelector("" + selector + "").click();"
2020-11-27 12:32:11.779 DEBUG testcase.                                - 20: jsBacA = "('#" + strElementID + "')"
2020-11-27 12:32:11.780 DEBUG testcase.                                - 21: out.println("BacA1: " + jsBacA)
BacA1: ('#TNwkgkvEDfjW_ZutritteContainerDiv_AxCMenu_new')
2020-11-27 12:32:11.781 DEBUG testcase.                                - 22: out.println("BacA2: " + js)
BacA2: document.querySelector("table.dhtmlxMenu_SubLevelArea_Tbl tr[id$=ZutritteContainerDiv_AxCMenu_new]").click();
2020-11-27 12:32:11.783 DEBUG testcase.                                - 23: executeJavaScript(jsBacA, null)


2020-11-27 12:32:11.810 DEBUG testcase.                                - 24: executeJavaScript(js, null)
2020-11-27 12:32:12.043 ERROR c.k.k.core.keyword.internal.KeywordMain  - ? Unable to execute JavaScript. (Root cause: com.kms.katalon.core.exception.StepFailedException: Unable to execute JavaScript.
	at com.kms.katalon.core.webui.keyword.internal.WebUIKeywordMain.stepFailed(WebUIKeywordMain.groovy:64)
	at com.kms.katalon.core.webui.keyword.internal.WebUIKeywordMain.runKeyword(WebUIKeywordMain.groovy:26)
	at com.kms.katalon.core.webui.keyword.builtin.ExecuteJavaScriptKeyword.executeJavascript(ExecuteJavascriptKeyword.groovy:42)
	at com.kms.katalon.core.webui.keyword.builtin.ExecuteJavaScriptKeyword.execute(ExecuteJavascriptKeyword.groovy:37)
	at com.kms.katalon.core.keyword.internal.KeywordExecutor.executeKeywordForPlatform(KeywordExecutor.groovy:73)
	at com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords.executeJavaScript(WebUiBuiltInKeywords.groovy:4244)
	at com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords$executeJavaScript$5.call(Unknown Source)
	at WSVerification1606476708226.run(WSVerification1606476708226:84)
	at com.kms.katalon.core.main.ScriptEngine.run(ScriptEngine.java:194)
	at com.kms.katalon.core.main.ScriptEngine.runScriptAsRawText(ScriptEngine.java:119)
	at com.kms.katalon.core.main.WSVerificationExecutor.runScript(WSVerificationExecutor.java:146)
	at com.kms.katalon.core.main.WSVerificationExecutor.doExecute(WSVerificationExecutor.java:140)
	at com.kms.katalon.core.main.WSVerificationExecutor.processExecutionPhase(WSVerificationExecutor.java:123)
	at com.kms.katalon.core.main.WSVerificationExecutor.accessMainPhase(WSVerificationExecutor.java:115)
	at com.kms.katalon.core.main.WSVerificationExecutor.execute(WSVerificationExecutor.java:103)
	at com.kms.katalon.core.main.TestCaseMain.runWSVerificationScript(TestCaseMain.java:127)
	at com.kms.katalon.core.main.TestCaseMain$runWSVerificationScript$0.call(Unknown Source)
	at TempTestCase1606476704888.run(TempTestCase1606476704888.groovy:25)
Caused by: org.openqa.selenium.JavascriptException: TypeError: document.querySelector(...) is null
Build info: version: '3.141.59', revision: 'e82be7d358', time: '2018-11-14T08:25:53'
System info: host: 'ABCD-N0060', ip: '10.35.31.2', os.name: 'Windows 10', os.arch: 'amd64', os.version: '10.0', java.version: '1.8.0_181'
Driver info: com.kms.katalon.core.webui.driver.ExistingRemoteWebDriver
Capabilities {javascriptEnabled: true, platform: ANY, platformName: ANY}
Session ID: c79ce2db-ca18-4097-94af-c1d57e6b711e
	at org.openqa.selenium.remote.http.W3CHttpResponseCodec.createException(W3CHttpResponseCodec.java:187)
	at org.openqa.selenium.remote.http.W3CHttpResponseCodec.decode(W3CHttpResponseCodec.java:122)
	at org.openqa.selenium.remote.http.W3CHttpResponseCodec.decode(W3CHttpResponseCodec.java:49)
	at org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:158)
	at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:552)
	at com.kms.katalon.core.webui.driver.ExistingRemoteWebDriver.execute(ExistingRemoteWebDriver.java:108)
	at org.openqa.selenium.remote.RemoteWebDriver.executeScript(RemoteWebDriver.java:489)
	at org.openqa.selenium.support.events.EventFiringWebDriver.lambda$new$1(EventFiringWebDriver.java:105)
	at com.sun.proxy.$Proxy9.executeScript(Unknown Source)
	at org.openqa.selenium.support.events.EventFiringWebDriver.executeScript(EventFiringWebDriver.java:229)
	at com.kms.katalon.core.webui.keyword.builtin.ExecuteJavaScriptKeyword$_executeJavascript_closure1.doCall(ExecuteJavascriptKeyword.groovy:48)
	at com.kms.katalon.core.webui.keyword.builtin.ExecuteJavaScriptKeyword$_executeJavascript_closure1.call(ExecuteJavascriptKeyword.groovy)
	at com.kms.katalon.core.webui.keyword.internal.WebUIKeywordMain.runKeyword(WebUIKeywordMain.groovy:20)
	... 16 more
)
2020-11-27 12:32:12.061 ERROR c.k.k.core.main.WSVerificationExecutor   - ? Verification FAILED.
Reason:
com.kms.katalon.core.exception.StepFailedException: Unable to execute JavaScript.
	at com.kms.katalon.core.webui.keyword.internal.WebUIKeywordMain.stepFailed(WebUIKeywordMain.groovy:64)
	at com.kms.katalon.core.webui.keyword.internal.WebUIKeywordMain.runKeyword(WebUIKeywordMain.groovy:26)
	at com.kms.katalon.core.webui.keyword.builtin.ExecuteJavaScriptKeyword.executeJavascript(ExecuteJavascriptKeyword.groovy:42)
	at com.kms.katalon.core.webui.keyword.builtin.ExecuteJavaScriptKeyword.execute(ExecuteJavascriptKeyword.groovy:37)
	at com.kms.katalon.core.keyword.internal.KeywordExecutor.executeKeywordForPlatform(KeywordExecutor.groovy:73)
	at com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords.executeJavaScript(WebUiBuiltInKeywords.groovy:4244)
	at com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords$executeJavaScript$5.call(Unknown Source)
	at WSVerification1606476708226.run(WSVerification1606476708226:84)
	at com.kms.katalon.core.main.ScriptEngine.run(ScriptEngine.java:194)
	at com.kms.katalon.core.main.ScriptEngine.runScriptAsRawText(ScriptEngine.java:119)
	at com.kms.katalon.core.main.WSVerificationExecutor.runScript(WSVerificationExecutor.java:146)
	at com.kms.katalon.core.main.WSVerificationExecutor.doExecute(WSVerificationExecutor.java:140)
	at com.kms.katalon.core.main.WSVerificationExecutor.processExecutionPhase(WSVerificationExecutor.java:123)
	at com.kms.katalon.core.main.WSVerificationExecutor.accessMainPhase(WSVerificationExecutor.java:115)
	at com.kms.katalon.core.main.WSVerificationExecutor.execute(WSVerificationExecutor.java:103)
	at com.kms.katalon.core.main.TestCaseMain.runWSVerificationScript(TestCaseMain.java:127)
	at com.kms.katalon.core.main.TestCaseMain$runWSVerificationScript$0.call(Unknown Source)
	at TempTestCase1606476704888.run(TempTestCase1606476704888.groovy:25)
Caused by: org.openqa.selenium.JavascriptException: TypeError: document.querySelector(...) is null
Build info: version: '3.141.59', revision: 'e82be7d358', time: '2018-11-14T08:25:53'
System info: host: 'ABCD-N0060', ip: '10.35.31.2', os.name: 'Windows 10', os.arch: 'amd64', os.version: '10.0', java.version: '1.8.0_181'
Driver info: com.kms.katalon.core.webui.driver.ExistingRemoteWebDriver
Capabilities {javascriptEnabled: true, platform: ANY, platformName: ANY}
Session ID: c79ce2db-ca18-4097-94af-c1d57e6b711e
	at org.openqa.selenium.remote.http.W3CHttpResponseCodec.createException(W3CHttpResponseCodec.java:187)
	at org.openqa.selenium.remote.http.W3CHttpResponseCodec.decode(W3CHttpResponseCodec.java:122)
	at org.openqa.selenium.remote.http.W3CHttpResponseCodec.decode(W3CHttpResponseCodec.java:49)
	at org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:158)
	at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:552)
	at com.kms.katalon.core.webui.driver.ExistingRemoteWebDriver.execute(ExistingRemoteWebDriver.java:108)
	at org.openqa.selenium.remote.RemoteWebDriver.executeScript(RemoteWebDriver.java:489)
	at org.openqa.selenium.support.events.EventFiringWebDriver.lambda$new$1(EventFiringWebDriver.java:105)
	at com.sun.proxy.$Proxy9.executeScript(Unknown Source)
	at org.openqa.selenium.support.events.EventFiringWebDriver.executeScript(EventFiringWebDriver.java:229)
	at com.kms.katalon.core.webui.keyword.builtin.ExecuteJavaScriptKeyword$_executeJavascript_closure1.doCall(ExecuteJavascriptKeyword.groovy:48)
	at com.kms.katalon.core.webui.keyword.builtin.ExecuteJavaScriptKeyword$_executeJavascript_closure1.call(ExecuteJavascriptKeyword.groovy)
	at com.kms.katalon.core.webui.keyword.internal.WebUIKeywordMain.runKeyword(WebUIKeywordMain.groovy:20)
	... 16 more

2020-11-27 12:32:12.066 INFO  c.k.k.core.main.WSVerificationExecutor   - END Verification

kazurayam · November 27, 2020, 12:38pm

The following log looks strange to me:

2020-11-27 12:32:11.778 DEBUG testcase.                                - 19: js = "document.querySelector("" + selector + "").click();"

Did you write as this?

js = "document.querySelector("" + selector + "").click();"

This looks wrong.

You should rather write perhaps:

js = "document.querySelector(" + selector + ").click();"

a.bach · November 27, 2020, 1:17pm

I think it is escaped “wrong” in the Katalon console, the script should be correct
Here are the script lines as screenshot (because in here I would have to escape “” too)

Russ_Thomas · November 27, 2020, 1:42pm

I don’t believe this didn’t cause a very different error:

Bring up the page in Firefox and type this into the devtools console:

document.querySelector('table.dhtmlxMebu_SubLevelArea_Tbl tr[id$=ZutritteContainerDiv_AxCMenu_new]')

If that works then you need to figure what context I’m missing.

Using this forum post, here is the proof that the method works:

kazurayam · November 27, 2020, 1:43pm

I agree with this.

kazurayam · November 27, 2020, 1:51pm

@a.bach

In the stacktrace, what’s most important is this line:

Caused by: org.openqa.selenium.JavascriptException: TypeError: document.querySelector(...) is null

You should identify at which line of your test case script you got this Exception. It’s only you who can find it; the others can not as they do not have your project on their PC.

The log shows as follows:

2020-11-27 12:32:11.779 DEBUG testcase.                                - 20: jsBacA = "('#" + strElementID + "')"
2020-11-27 12:32:11.780 DEBUG testcase.                                - 21: out.println("BacA1: " + jsBacA)
BacA1: ('#TNwkgkvEDfjW_ZutritteContainerDiv_AxCMenu_new')
...

Therefore your code effectively does the same thing as follows:

WebUI.executeJavaSript("('#TNwkgkvEDfjW_ZutritteContainerDiv_AxCMenu_new')", null)

This would work strange. The string

('#TNwkgkvEDfjW_ZutritteContainerDiv_AxCMenu_new')

is a valid JavaScript code that contains a string literal. Just that. It is not interpreted as an instruction to select HTML nodes.

Remember, $(string) you typed in the Chrome DevTools Console is a special command syntax supported by Chrome DevTools only. It is not valid as JavaScript code unless specifically configured.

Russ_Thomas · November 27, 2020, 2:07pm

I just noticed this:

# means id so you are looking for any element with “id=tr[id$=zut…” which is clearly meaningless.

Please do us all a favor, including you - learn how css selectors work. The cost to us now is having to comb over your code with great precision because you’re making stuff up and hoping it works. That was not the point of your original post.

a.bach · November 30, 2020, 2:28pm

Sorry guys, for the confusion. I just wanted to show that the element returns something in js and only the document queryselector was null. Thats it. Didnt plan to use it for something.
This went deeper into web knowledge than I expected at the beginning and so we/I drifted a bit far from my original post and my personal knowledge - true.

Apologies for missunderstandings, this was the last of my intenstion, I usually prepare these sort of things thoroughly.

And now, to destroy my reputation once and for all, the solution:
This first did not work:
String selector = 'table.dhtmlxMebu_SubLevelArea_Tbl tr[id$=ZutritteContainerDiv_AxCMenu_new]'

I tried to find out about dhtmlxMebu and found dhtmlxMenu. Mebu and Menu. I thought it was a type-o (b and n are keyboard neighbours) and changed it to Menu -> which of course also did not work

I tried a million things in the meantime and forgot about the assumed type-o.

So to sum up, the following lines work:
String selector = ‘table.dhtmlxMebu_SubLevelArea_Tbl tr[id$=ZutritteContainerDiv_AxCMenu_new]’
String js = ‘document.querySelector("’+selector+’").click();’
WebUI.executeJavaScript(js, null)

Now I have to create a new forum account if I ever want to post anything again

Russ_Thomas · November 30, 2020, 2:54pm

That’s silly. Taking ownership of your issues is HUGE. Keep at it, NONE of us were born with this knowledge - it takes time and a mountain of mistakes to become expert (in anything).

Welcome to the club

Glad you got it fixed

kazurayam · November 30, 2020, 11:40pm