Cannot get List of divs from the parent div by xpath

Hello!
The general idea is I have items, get the list of them and then click randomly on any, but I’m stuck.
My code below is:
System.setProperty(“webdriver.chrome.driver”, “C:\Users\Mila\Downloads\Automation testing\chromedriver.exe”);
WebDriver driver = new ChromeDriver();

WebUI.openBrowser('')
WebUI.navigateToUrl('*********')
WebUI.maximizeWindow()
WebUI.waitForPageLoad(80)
WebUI.verifyElementPresent(findTestObject("Object Repository/Page_ID Accessories  Leader Products/div_Accessories options"), 80)
Thread.sleep(3000)

List<WebElement> list = driver.findElements(By.xpath("//div[@class='product-grid xs-2 sm-2 md-3 lg-4 xl-4']"))
System.out.println(list)
Random r = new Random()

int randomValue = r.nextInt(list.size())
list.get(randomValue).click()

So I printed the “list” and got empty .
And of course, after running the test I was getting “bound must be positive”.
The whole error text is below:
eason:
java.lang.IllegalArgumentException: bound must be positive
at java_util_Random$nextInt.call(Unknown Source)
at Random Playground.run(Random Playground:25)
at com.kms.katalon.core.main.ScriptEngine.run(ScriptEngine.java:194)
at com.kms.katalon.core.main.ScriptEngine.runScriptAsRawText(ScriptEngine.java:119)
at com.kms.katalon.core.main.TestCaseExecutor.runScript(TestCaseExecutor.java:337)
at com.kms.katalon.core.main.TestCaseExecutor.doExecute(TestCaseExecutor.java:328)
at com.kms.katalon.core.main.TestCaseExecutor.processExecutionPhase(TestCaseExecutor.java:307)
at com.kms.katalon.core.main.TestCaseExecutor.accessMainPhase(TestCaseExecutor.java:299)
at com.kms.katalon.core.main.TestCaseExecutor.execute(TestCaseExecutor.java:233)
at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:114)
at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:105)
at com.kms.katalon.core.main.TestCaseMain$runTestCase$0.call(Unknown Source)
at TempTestCase1589418618472.run(TempTestCase1589418618472.groovy:25)

I played around xpaths, but did not succeed

image

Try …

List<WebElement> list = driver.findElements(
  By.xpath("//div[@class='product-grid item']"))

I’m guessing you’ll want to dig into the <a ...> elements, too.

Hii! Thanks for the prompt reply
Yeah, I guess for clicking I’d need elements, sure
I’ve tried what you’ve suggested, but no luck image
I’ve also tried //div[@class=‘product-grid xs-2 sm-2 md-3 lg-4 xl-4’]/child::* and there was the same result

I the screenshot, I can see 2 underline characters between grid and item in the class attribute’s value.

Wrong:

List<WebElement> list = driver.findElements(
    By.xpath("//div[@class='product-grid item']"))

Right:

List<WebElement> list = driver.findElements(
    By.xpath("//div[@class='product-grid__item']"))

Oh, that’s right
but result the same with the proper xpath
I even entered that xpath in the DevTools and the required elements were highlighted


image

Does your HTML contains <iframe> ?

not as I can see, searched for “iframe” in the code
Can’t understand why it does not want to work

The DevTools can find the elements with the XPath. It proves that the XPath is OK. But it does not necessarily mean that your script can find the element soon after the page is loaded.

Possibly your script should make sure that the elements are loaded before calling driver.findElements(...) by calling WebUI.waitForElementsPresent or WebDriverWait(driver, seconds) { closure }

I had verification to check if element present, but exchanged it for what you suggested but still


I’m wondering if the issue can be that the element is dymanic
Sorry, I’m not that good at autotesting yet, posted here when was desperate already :sweat_smile:

In order to think about the XPath’s validity etc, we need to look at the HTML source from the 1st line to the end.

Is the URL of your AUT is public? If so, please share it.

If it is not public, then could you save the full HTML source code of your target page into a file and attach it here?

Not a public yet, here is the code in the attachment
Thanks
Updated: HTML removed

I saved the file you attached into a Katlaon project’ directory with the name “page.html”.

I wrote the following testcase.

import java.nio.file.Path
import java.nio.file.Paths

import org.openqa.selenium.By
import org.openqa.selenium.WebDriver
import org.openqa.selenium.WebElement

import com.kms.katalon.core.configuration.RunConfiguration
import com.kms.katalon.core.testobject.ConditionType
import com.kms.katalon.core.testobject.TestObject as TestObject
import com.kms.katalon.core.webui.driver.DriverFactory
import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI

String filePath = "./page.html"

Path html = Paths.get(RunConfiguration.getProjectDir()).resolve(filePath)
URL htmlURL = html.toFile().toURI().toURL()
WebUI.openBrowser(htmlURL.toString())

String xpath = "//div[@class='product-grid__item']"

TestObject tObj = new TestObject()
tObj.addProperty("xpath", ConditionType.EQUALS, xpath)

// wait for the elements to be loaded
WebUI.verifyElementPresent(tObj,10)

WebDriver driver = DriverFactory.getWebDriver();
List<WebElement> elements = driver.findElements(By.xpath(xpath))
WebUI.comment("elements.size()=${elements.size()}")

WebUI.closeBrowser()

When I ran it, it worked fine.

2020-05-14 15:15:28.138 INFO  c.k.katalon.core.main.TestCaseExecutor   - START Test Cases/TC1
2020-05-14 15:15:29.168 DEBUG testcase.TC1                             - 1: filePath = "./page.html"
2020-05-14 15:15:29.173 DEBUG testcase.TC1                             - 2: html = get(getProjectDir()).resolve(filePath)
2020-05-14 15:15:29.188 DEBUG testcase.TC1                             - 3: htmlURL = toURI().toURL()
2020-05-14 15:15:29.205 DEBUG testcase.TC1                             - 4: openBrowser(htmlURL.toString())
2020-05-14 15:15:29.902 INFO  c.k.k.core.webui.driver.DriverFactory    - Starting 'Firefox' driver
5 14, 2020 3:15:29 午後 org.openqa.selenium.remote.DesiredCapabilities firefox
情報: Using `new FirefoxOptions()` is preferred to `DesiredCapabilities.firefox()`
2020-05-14 15:15:30.156 INFO  c.k.k.core.webui.driver.DriverFactory    - Action delay is set to 0 seconds
1589436932011	mozrunner::runner	INFO	Running command: "/Applications/Firefox.app/Contents/MacOS/firefox-bin" "-marionette" "-foreground" "-no-remote" "-profile" "/var/folders/7m/lm7d6nx51kj0kbtnsskz6r3m0000gn/T/rust_mozprofile.kjXHa9RYBTFM"
1589436932644	addons.webextension.<unknown>	WARN	Loading extension 'null': Reading manifest: Warning processing permissions: Error processing permissions.12: Value "debugger" must either: must either [must either [be one of ["clipboardRead", "clipboardWrite", "geolocation", "idle", "notifications"], be one of ["bookmarks"], be one of ["find"], be one of ["history"], be one of ["menus.overrideContext"], be one of ["search"], be one of ["topSites"], be one of ["activeTab", "tabs", "tabHide"], be one of ["browserSettings"], be one of ["cookies"], be one of ["downloads", "downloads.open"], be one of ["privacy"], be one of ["webRequest", "webRequestBlocking"], or be one of ["webNavigation"]], be one of ["alarms", "mozillaAddons", "storage", "unlimitedStorage"], be one of ["browsingData"], be one of ["captivePortal"], be one of ["devtools"], be one of ["identity"], be one of ["menus", "contextMenus"], be one of ["normandyAddonStudy"], be one of ["pkcs11"], be one of ["geckoProfiler"], be one of ["sessions"], be one of ["urlbar"], be one of ["contextualIdentities"], be one of ["dns"], be one of ["activityLog"], be one of ["management"], be one of ["networkStatus"], be one of ["proxy"], be one of ["nativeMessaging"], be one of ["telemetry"], be one of ["theme"], or match the pattern /^experiments(\.\w+)+$/], or must either [be one of ["<all_urls>"], must either [match the pattern /^(https?|wss?|file|ftp|\*):\/\/(\*|\*\.[^*/]+|[^*/]+)\/.*$/, or match the pattern /^file:\/\/\/.*$/], or match the pattern /^resource:\/\/(\*|\*\.[^*/]+|[^*/]+)\/.*$|^about:/]
1589436932645	addons.webextension.<unknown>	WARN	Loading extension 'null': Reading manifest: Warning processing offline_enabled: An unexpected property was found in the WebExtension manifest.
1589436932747	addons.webextension.doh-rollout@mozilla.org	WARN	Loading extension 'doh-rollout@mozilla.org': Reading manifest: Invalid extension permission: networkStatus
1589436932768	addons.webextension.screenshots@mozilla.org	WARN	Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid extension permission: mozillaAddons
1589436932768	addons.webextension.screenshots@mozilla.org	WARN	Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid extension permission: telemetry
1589436932769	addons.webextension.screenshots@mozilla.org	WARN	Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid extension permission: resource://pdf.js/
1589436932769	addons.webextension.screenshots@mozilla.org	WARN	Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid extension permission: about:reader*
1589436933169	addons.webextension.{54dcd9e9-903f-43e9-95f0-22bd0e0f3c0a}	WARN	Loading extension '{54dcd9e9-903f-43e9-95f0-22bd0e0f3c0a}': Reading manifest: Warning processing permissions: Error processing permissions.12: Value "debugger" must either: must either [must either [be one of ["clipboardRead", "clipboardWrite", "geolocation", "idle", "notifications"], be one of ["bookmarks"], be one of ["find"], be one of ["history"], be one of ["menus.overrideContext"], be one of ["search"], be one of ["topSites"], be one of ["activeTab", "tabs", "tabHide"], be one of ["browserSettings"], be one of ["cookies"], be one of ["downloads", "downloads.open"], be one of ["privacy"], be one of ["webRequest", "webRequestBlocking"], or be one of ["webNavigation"]], be one of ["alarms", "mozillaAddons", "storage", "unlimitedStorage"], be one of ["browsingData"], be one of ["captivePortal"], be one of ["devtools"], be one of ["identity"], be one of ["menus", "contextMenus"], be one of ["normandyAddonStudy"], be one of ["pkcs11"], be one of ["geckoProfiler"], be one of ["sessions"], be one of ["urlbar"], be one of ["contextualIdentities"], be one of ["dns"], be one of ["activityLog"], be one of ["management"], be one of ["networkStatus"], be one of ["proxy"], be one of ["nativeMessaging"], be one of ["telemetry"], be one of ["theme"], or match the pattern /^experiments(\.\w+)+$/], or must either [be one of ["<all_urls>"], must either [match the pattern /^(https?|wss?|file|ftp|\*):\/\/(\*|\*\.[^*/]+|[^*/]+)\/.*$/, or match the pattern /^file:\/\/\/.*$/], or match the pattern /^resource:\/\/(\*|\*\.[^*/]+|[^*/]+)\/.*$|^about:/]
1589436933169	addons.webextension.{54dcd9e9-903f-43e9-95f0-22bd0e0f3c0a}	WARN	Loading extension '{54dcd9e9-903f-43e9-95f0-22bd0e0f3c0a}': Reading manifest: Warning processing offline_enabled: An unexpected property was found in the WebExtension manifest.
1589436935863	Marionette	INFO	Listening on port 56359
1589436935916	Marionette	WARN	TLS certificate errors will be ignored for this session
1589436935928	Marionette	INFO	Proxy settings initialised: {"proxyType":"direct"}
5 14, 2020 3:15:36 午後 org.openqa.selenium.remote.ProtocolHandshake createSession
情報: Detected dialect: W3C
2020-05-14 15:15:36.128 INFO  c.k.k.core.webui.driver.DriverFactory    - sessionId = d8d7262a-7598-f648-9ef8-b035b588cf44
2020-05-14 15:15:36.263 INFO  c.k.k.core.webui.driver.DriverFactory    - browser = Firefox 75.0
2020-05-14 15:15:36.264 INFO  c.k.k.core.webui.driver.DriverFactory    - platform = Mac OS X
2020-05-14 15:15:36.265 INFO  c.k.k.core.webui.driver.DriverFactory    - seleniumVersion = 3.141.59
2020-05-14 15:15:36.269 INFO  c.k.k.core.webui.driver.DriverFactory    - proxyInformation = ProxyInformation{proxyOption=NO_PROXY, proxyServerType=HTTP, password=, proxyServerAddress=, proxyServerPort=0, executionList=}
2020-05-14 15:15:38.984 DEBUG testcase.TC1                             - 5: xpath = "//div[@class='product-grid__item']"
2020-05-14 15:15:38.986 DEBUG testcase.TC1                             - 6: tObj = new com.kms.katalon.core.testobject.TestObject()
2020-05-14 15:15:39.025 DEBUG testcase.TC1                             - 7: tObj.addProperty("xpath", EQUALS, xpath)
2020-05-14 15:15:39.061 DEBUG testcase.TC1                             - 8: verifyElementPresent(tObj, 10)
2020-05-14 15:15:39.686 DEBUG testcase.TC1                             - 9: driver = getWebDriver()
2020-05-14 15:15:39.701 DEBUG testcase.TC1                             - 10: elements = driver.findElements(By.xpath(xpath))
2020-05-14 15:15:39.959 DEBUG testcase.TC1                             - 11: comment(elements.size()=$elements.size())
2020-05-14 15:15:40.057 INFO  c.k.k.c.keyword.builtin.CommentKeyword   - elements.size()=2
2020-05-14 15:15:40.067 DEBUG testcase.TC1                             - 12: closeBrowser()
1589436940163	Marionette	INFO	Stopped listening on port 56359
2020-05-14 15:15:41.773 INFO  c.k.katalon.core.main.TestCaseExecutor   - END Test Cases/TC1

I do not see why you have a problem. HTML is fine, XPath is OK. Then what? Your test case script must have something mysterious.

I ran your test and it works well
Will try to find a difference, thanks :blush:
Now I know all works well

and incorporated parts of your tests in my test and I’m able to random click
Phew, now I can sleep well

I think that the difference between your original script and mine is this:

Your script waited for something else.

A part of your HTML is pasted here.

<div class="product-grid__item">
    <a href="/en_CA/products/id-accessories/accessories/" class="product-grid__link product-grid__link--small" data-pid="16" title="Accessories">
        <figure class="photo product-grid__photo">
            <img src="/assets/Uploads/hqdbRBoA__PadWzI4MCwyMDAsIkZGRkZGRiIsMF0.png" class="photo__image" alt="Accessories"/>
        </figure>

I have a doubt that the png file loaded by <img> tag is big and would takes some milliseconds to load. I suppose that the milliseconds taken for loading the image delayed the element <div class="product-grid__item"> to appear in the DOM.

If your test script waits for that exact element, it would work fine. But your original script waited for something else

therefore your script was too hasty (in the milliseconds order) before checking the elements of your interest.

Thank you for such detailed explanation :pray:
So little details matter so much

Devil is in the details.

1 Like