How to capture multiple objects from multiple iframes on a webpage

tgeabler · February 10, 2021, 8:31pm

I am working on building navigation and regression of webpages that have iframes with interactable graphs. I am trying to capture multiple objects within a single iframe and when I try capturing the first object, it captures the iframe object and path object. The iframe object is accurate, but the path xpath is empty and although there are attributes none of the attributes work to identify the specific object. Also if I try to capture another object within the same iframe, that object does not get added under the iframe, but seems to write over the first object. Any assistance would be greatly appreciated.

Brandon_Hein · February 10, 2021, 9:06pm

Sounds like you need to write your own xpaths for these, and not rely on captures. Can you share the HTML you are working with, as well as the objects you are trying to capture?

kazurayam · February 11, 2021, 1:53am

@tgeabler

You want to locate HTML nodes within <iframe>s. This is a bit difficult thing to achieve. Capturing tools like Spy and Recorder can not handle <iframe> properly.

Your test case script should use WebUI.switchToFrame()
You can not be ignorant of XPath and CSS selector technology. You need to learn how to write expressions for yourself, rather than relying on the capturing tools.
As for XPath, this could be a read for you.

tgeabler · February 15, 2021, 5:52pm

I usually haven’t had much trouble working with xpaths up to now. Iframes is something new that we are working with trying to integrate report software into our product. I am trying to get down to the #document within the iframe in the picture. I can spy and locate the iframe using Katalon Studio, but it won’t locate any children. Trying to search and see if maybe I need to specifically use Selenium. SwitchToFrame doesn’t seem to find any objects within the frame.

Russ_Thomas · February 15, 2021, 6:25pm

Ignore #document. It does not mean anything to the Katalon or Selenium APIs you are trying to use. Browsers put that there to signify the point at which the contentDocument “begins”, otherwise, it’s essentially useless.

WebUI.switchToFrame() works. It is based on underlying Selenium code which also works.

What you need to do from here:

Use WebUI.switchToFrame()
Use regular WebUI methods to access the elements using Test Objects that are designed to work with the iframe document as if it was loaded WITHOUT an iframe.

#2 will not work until you have successfully performed #1.

tgeabler · February 15, 2021, 7:05pm

Thanks for the clarification. I am trying to click on a pie chart sector with the path information and am trying to use that in the xpath, but still do not seem to be having luck even after using WebUI.swithcToFrame() first. Assume using the d locator should be accurate for the path element, but still not recognizing it.

Russ_Thomas · February 15, 2021, 7:38pm

Well, I’m NOT an XPath guru or even an xpath guy, but I think you need stuff that looks something like…

 //*[local-name()='svg']//*[local- name()='g' ...

Let’s see if @Brandon_Hein can figure out something for you…

Brandon_Hein · February 15, 2021, 8:21pm

I am assuming here that you are working with one single <iframe> element, and that this element is not nested within any other <iframe> element. I will also use pure selenium, but you can do this with the WebUI keywords too:

1.) Switch the context to the iframe:

WebDriver driver = DriverFactory.getWebDriver();
WebElement element = driver.findElement(By.xpath("//iframe"));
driver.switchTo().frame(element);

2.) Now that you are within the context of the iframe, find the element you want to click. The <path> element you want to click is a bit tricky, because it looks like most of the attributes are dynamically valued, but I would use the @elementargument attribute with an index (since there appears to be 2 elements for each unique @elementargument attribute):

element = driver.findElement(By.xpath("(//path[@elementargument='Subs'])[1]"));
element.click();

tgeabler · February 15, 2021, 9:15pm

There are actually four <iframe> elements so I am trying to use a variant of what you showed and am trying:

WebDriver driver = DriverFactory.getWebDriver();
WebElement element = driver.findElement(By.xpath(“(//iframe)[4]”));
driver.switchTo().frame(element)
element = driver.findElement(By.xpath(“(//path[@elementargument=‘Subs’])[1]”));
element.click()

I tried running and got this error

Test Cases/Navigate/Overview/CostsByCostClass/ChooseCostsByCostClass FAILED.
Reason:
org.openqa.selenium.NoSuchElementException: no such element: Unable to locate element: {"method":"xpath","selector":"(//path[@elementargument='Subs'])[1]"}
  (Session info: chrome=88.0.4324.150)
For documentation on this error, please visit: https://www.seleniumhq.org/exceptions/no_such_element.html
Build info: version: '3.141.59', revision: 'e82be7d358', time: '2018-11-14T08:25:53'
System info: host: 'W10LCODEACAD11', ip: '10.101.0.42', os.name: 'Windows 10', os.arch: 'amd64', os.version: '10.0', java.version: '1.8.0_181'
Driver info: com.kms.katalon.selenium.driver.CChromeDriver
Capabilities {acceptInsecureCerts: false, browserName: chrome, browserVersion: 88.0.4324.150, chrome: {chromedriverVersion: 88.0.4324.96 (68dba2d8a0b14..., userDataDir: C:\Users\tgeabler\AppData\L...}, goog:chromeOptions: {debuggerAddress: localhost:59794}, javascriptEnabled: true, networkConnectionEnabled: false, pageLoadStrategy: normal, platform: WINDOWS, platformName: WINDOWS, proxy: Proxy(), setWindowRect: true, strictFileInteractability: false, timeouts: {implicit: 0, pageLoad: 300000, script: 30000}, unhandledPromptBehavior: dismiss and notify, webauthn:extension:largeBlob: true, webauthn:virtualAuthenticators: true}
Session ID: 3f86b9b0636160d5006026e6aa2dcb86
*** Element info: {Using=xpath, value=(//path[@elementargument='Subs'])[1]}
	at org.openqa.selenium.remote.http.W3CHttpResponseCodec.createException(W3CHttpResponseCodec.java:187)
	at org.openqa.selenium.remote.http.W3CHttpResponseCodec.decode(W3CHttpResponseCodec.java:122)
	at org.openqa.selenium.remote.http.W3CHttpResponseCodec.decode(W3CHttpResponseCodec.java:49)
	at org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:158)
	at org.openqa.selenium.remote.service.DriverCommandExecutor.execute(DriverCommandExecutor.java:83)
	at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:552)
	at com.kms.katalon.selenium.driver.CChromeDriver.execute(CChromeDriver.java:19)
	at org.openqa.selenium.remote.RemoteWebDriver.findElement(RemoteWebDriver.java:323)
	at org.openqa.selenium.remote.RemoteWebDriver.findElementByXPath(RemoteWebDriver.java:428)
	at org.openqa.selenium.By$ByXPath.findElement(By.java:353)
	at org.openqa.selenium.remote.RemoteWebDriver.findElement(RemoteWebDriver.java:315)
	at org.openqa.selenium.support.events.EventFiringWebDriver.lambda$new$1(EventFiringWebDriver.java:105)
	at com.sun.proxy.$Proxy10.findElement(Unknown Source)
	at org.openqa.selenium.support.events.EventFiringWebDriver.findElement(EventFiringWebDriver.java:194)
	at org.openqa.selenium.WebDriver$findElement.call(Unknown Source)
	at ChooseCostsByCostClass.run(ChooseCostsByCostClass:38)
	at com.kms.katalon.core.main.ScriptEngine.run(ScriptEngine.java:194)
	at com.kms.katalon.core.main.ScriptEngine.runScriptAsRawText(ScriptEngine.java:119)
	at com.kms.katalon.core.main.TestCaseExecutor.runScript(TestCaseExecutor.java:398)
	at com.kms.katalon.core.main.TestCaseExecutor.doExecute(TestCaseExecutor.java:389)
	at com.kms.katalon.core.main.TestCaseExecutor.processExecutionPhase(TestCaseExecutor.java:368)
	at com.kms.katalon.core.main.TestCaseExecutor.accessMainPhase(TestCaseExecutor.java:360)
	at com.kms.katalon.core.main.TestCaseExecutor.execute(TestCaseExecutor.java:255)
	at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:114)
	at com.kms.katalon.core.keyword.builtin.CallTestCaseKeyword$_callTestCase_closure1.doCall(CallTestCaseKeyword.groovy:59)
	at com.kms.katalon.core.keyword.builtin.CallTestCaseKeyword$_callTestCase_closure1.call(CallTestCaseKeyword.groovy)
	at com.kms.katalon.core.keyword.internal.KeywordMain.runKeyword(KeywordMain.groovy:68)
	at com.kms.katalon.core.keyword.builtin.CallTestCaseKeyword.callTestCase(CallTestCaseKeyword.groovy:81)
	at com.kms.katalon.core.keyword.builtin.CallTestCaseKeyword.execute(CallTestCaseKeyword.groovy:44)
	at com.kms.katalon.core.keyword.internal.KeywordExecutor.executeKeywordForPlatform(KeywordExecutor.groovy:73)
	at com.kms.katalon.core.keyword.BuiltinKeywords.callTestCase(BuiltinKeywords.groovy:334)
	at CostsByCostValueClass.run(CostsByCostValueClass:19)
	at com.kms.katalon.core.main.ScriptEngine.run(ScriptEngine.java:194)
	at com.kms.katalon.core.main.ScriptEngine.runScriptAsRawText(ScriptEngine.java:119)
	at com.kms.katalon.core.main.TestCaseExecutor.runScript(TestCaseExecutor.java:398)
	at com.kms.katalon.core.main.TestCaseExecutor.doExecute(TestCaseExecutor.java:389)
	at com.kms.katalon.core.main.TestCaseExecutor.processExecutionPhase(TestCaseExecutor.java:368)
	at com.kms.katalon.core.main.TestCaseExecutor.accessMainPhase(TestCaseExecutor.java:360)
	at com.kms.katalon.core.main.TestCaseExecutor.execute(TestCaseExecutor.java:255)
	at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:114)
	at com.kms.katalon.core.keyword.builtin.CallTestCaseKeyword$_callTestCase_closure1.doCall(CallTestCaseKeyword.groovy:59)
	at com.kms.katalon.core.keyword.builtin.CallTestCaseKeyword$_callTestCase_closure1.call(CallTestCaseKeyword.groovy)
	at com.kms.katalon.core.keyword.internal.KeywordMain.runKeyword(KeywordMain.groovy:68)
	at com.kms.katalon.core.keyword.builtin.CallTestCaseKeyword.callTestCase(CallTestCaseKeyword.groovy:81)
	at com.kms.katalon.core.keyword.builtin.CallTestCaseKeyword.execute(CallTestCaseKeyword.groovy:44)
	at com.kms.katalon.core.keyword.internal.KeywordExecutor.executeKeywordForPlatform(KeywordExecutor.groovy:73)
	at com.kms.katalon.core.keyword.BuiltinKeywords.callTestCase(BuiltinKeywords.groovy:334)
	at NavigateOverview.run(NavigateOverview:21)
	at com.kms.katalon.core.main.ScriptEngine.run(ScriptEngine.java:194)
	at com.kms.katalon.core.main.ScriptEngine.runScriptAsRawText(ScriptEngine.java:119)
	at com.kms.katalon.core.main.TestCaseExecutor.runScript(TestCaseExecutor.java:398)
	at com.kms.katalon.core.main.TestCaseExecutor.doExecute(TestCaseExecutor.java:389)
	at com.kms.katalon.core.main.TestCaseExecutor.processExecutionPhase(TestCaseExecutor.java:368)
	at com.kms.katalon.core.main.TestCaseExecutor.accessMainPhase(TestCaseExecutor.java:360)
	at com.kms.katalon.core.main.TestCaseExecutor.execute(TestCaseExecutor.java:255)
	at com.kms.katalon.core.main.TestSuiteExecutor.accessTestCaseMainPhase(TestSuiteExecutor.java:203)
	at com.kms.katalon.core.main.TestSuiteExecutor.accessTestSuiteMainPhase(TestSuiteExecutor.java:152)
	at com.kms.katalon.core.main.TestSuiteExecutor.execute(TestSuiteExecutor.java:95)
	at com.kms.katalon.core.main.TestCaseMain.startTestSuite(TestCaseMain.java:157)
	at com.kms.katalon.core.main.TestCaseMain$startTestSuite$0.call(Unknown Source)
	at TempTestSuite1613422311763.run(TempTestSuite1613422311763.groovy:39)

Brandon_Hein · February 16, 2021, 12:12am

You are either in the wrong frame, or you are in the correct frame and the xpath is not valid for that frame. One nifty trick you can use to test any xpath:

1.) Open devtools by pressing F12. Open the Console tab in devtools:

2.) On the Console tab, you can select an iframe to switch to with this setting:

3.) Enter an xpath into the console using the $x() syntax, like so:

This way, you can both choose the context, and test an xpath within that context, before ever writing code in Katalon.

kazurayam · February 16, 2021, 3:01am

@tgeabler

Your xpath is wrong.

Try this:

WebDriver driver = DriverFactory.getWebDriver();
WebElement element = driver.findElement(By.xpath("(//iframe)[4]"));
driver.switchTo().frame(element)

element = driver.findElement(By.xpath("(//*[local-name='svg']//*[local-name()='path' and @elementargument='Subs'])[1]"));

element.click()

kazurayam · February 16, 2021, 3:04am

Your XPath expression:

This will not select the node you want.

Rather you should write:

(//*[local-name()='svg']//*[local-name()='path' and @elementargument='Subs'])[1]

You may wonder what //*[local-name='svg']//*[local-name()='path'' ....' is. Before talking about it, let me ask you some questions:

Are you aware of the concept of XML Namespace?
Are you aware that the node you want to click belongs to the Namespace of Scalable Vector Graphics?
Are you aware that the node you want to click does not belong to the Namespace of XHTML?
Are you aware that you have a node <html xmlns="http://www.w3.org/1999/xhtml"> as the ancestor of your target node?
The screenshot you provided does not show it, but most probably you would have a node <svg xmlns="http://www.w3.org/2000/svg"> as the ancestor of your target node. Am I right?

If you do not understand my questions, you need to study “XML Namespace” specification first. Otherwise you would be lost forever.

Also you would want to study XPath local-name() and name() functions:

xml - What is the difference between name() and local-name()? - Stack Overflow

Here is my previous post about selecting a SVG node contained in a HTML document. Please have a look.

tgeabler · February 16, 2021, 7:35pm

@kazurayam Thank you very much. I will be honest I am still pretty new with some of this stuff. I have been getting better at xpath stuff, just haven’t come across iframe stuff to automate until now. I will definitely start looking in to some of the material you suggested. The changes you have recommended are working when I run in Chrome with the browser, but not when it is running headless so I am trying to look in to why that might be. I will let you know if I figure it out. Thanks again!

kazurayam · February 17, 2021, 2:13am

Similar issue is here:

kazurayam · February 17, 2021, 2:40am

I guess loading=“lazy” of iframe may have something to do with this.

Standardized lazy-loading of iframes defers offscreen iframes from being loaded until the user scrolls near them. This saves data, speeds up the loading of other parts of the page, and reduces memory usage.

I would recommend you to try inserting

WebUI.scrollToElement( testobject_for_the_iframe )

before getting access to the <iframe> and its contents. Scrolling to the <iframe> element will give headless browser a trigger to fire loading the external page source.

It could be the case that a headless browser is too lazy to load the documents inside <iframe> … I am not sure.

Or you may want to insert

WebUI.delay(5)

to wait for the document inside <iframe> to be loaded completely.

Brandon_Hein · February 17, 2021, 5:03pm

Wasn’t aware of this, as I’ve never dealt with these types of elements in our applications. Thanks kaz!

tgeabler · February 17, 2021, 8:25pm

Thank you so much for the added insight and assistance. I am setting it to run with the GUI. We use the Enterprise to build our tests and the we schedule and run them in TFS with the Runtime Engine. Is there a chance this could have an effect on the test?

kazurayam · February 17, 2021, 10:58pm

The sole question is about the behavior of browsers — if <iframe> loads the linked document?

I think the type of Browser (Chrome/FireFox/Edge/Safari, headful/headless) would matter.

Type of CI/CD server — I don’t think it matters.

KS/KSE/KRE — I don’t think it matters.

Topic		Replies	Views
Iframe issue on katalon Katalon Studio katalon-studio , web-testing	10	1469	February 13, 2023
Cannot click on iframe objects Katalon Studio katalon-studio , web-testing	4	444	March 13, 2020
Problem capturing html content inside another html Katalon Studio katalon-studio , web-testing	1	330	February 11, 2021
How to find an object within an iframe Katalon Studio katalon-studio	8	874	February 24, 2023
Facing issue with capturing objects using Object spy Archive integrations , katalon-studio	2	360	August 8, 2022

How to capture multiple objects from multiple iframes on a webpage

Related Topics