A custom keyword which returns a List of Selenium WebElement from a Web page

# Defining a problem

Let me define a problem with a simple HTML as target.

1. I have a Web page to test. The target page has a list of web elements. Let me name it as the found elements. Please find a list of elements in the following HTML snippet:

<select id="combo_facility" name="facility" class="form-control">
  <option value="Tokyo">Tokyo CURA Healthcare Center</option>
  <option value="Hongkong">Hongkong CURA Healthcare Center</option>
  <option value="Seoul">Seoul CURA Healthcare Center</option>
</select>

The way how the web elements are marked up is not so significant. You may have

  • , , or whatever. I would assume that we are able to code any XPath expression appropriate to select the HTML elements of our interest out of the target page.

    2. My test case has a list of texts. Let me name it as the expected data. Let me suppose I have the following literal in my test case code, for example:

    List<Map<String,String>> data =  [
      ["text":"Hongkong CURA Healthcare Center"],
      ["text":"Seoul CURA Healthcare Center"],
      ["text":"Tokyo CURA Healthcare Center"],
      ["text":"New York CURA Healthcare Center"]
    ]
    

    3. I want to determine ‘yes’ or ‘no’ for each items of the expected data. My test case should emit message for each text if it is “displayed:yes” or “displayed:no” in the target Web page.

    4. The two lists roughly correspond; but these are not 100% correspondent. I mean:

    • The size of the found elements is not necessarily equal to the size of the expected data. Making one-to-one correspondence between entries of each lists may results remainders.
    • These lists may be sorted differently. Either of these may be unsorted at all.

    I want to iterate over the expected data to find out if each text is displayed in the target page. Therefore I want to perform nested iteration over the found elements for each text in the expected data.

    I think that this problem is a frequently-asked-question in the in the Katalon forum.

    # Blocking problem of Katalon Studio

    Now I want to perform, in my test case, an iteration over the found elements. However, Katalon Studio does not provide any built-in keyword which returns a list of web elements out of the target Web page.

    # Solution proposed

    ## My custom keyword

    Here I have developed a Groovy class as custom keyword for Katalon Studio: com.kazurayam.ksbackyard.FindElementsByXPath. This class implements a method: List<org.openqa.selenium.WebElement> getWebElementsAsList(String xpath). The source code is here. Here I copy&paste it:

    package com.kazurayam.ksbackyard
    import org.openqa.selenium.By
    import org.openqa.selenium.WebDriver
    import org.openqa.selenium.WebElement
    import com.kms.katalon.core.annotation.Keyword
    import com.kms.katalon.core.webui.driver.DriverFactory
     
    class FindElementsByXPath {
        @Keyword
        List<WebElement> getWebElementsAsList(String xpath4elements) {
            WebDriver webDriver = DriverFactory.getWebDriver()
            List<WebElement> elements = webDriver.findElements(By.xpath(xpath4elements))
            return elements
        }
    }
    

    Please note that this keyword returns List<org.openqa.selenium.WebElement>. Hence, those who use this custom keyword have to study WebDriver’s API document/ WebElement.

    ## My test case

    Here is my test case TC_getWebElementsAsList:

    import org.openqa.selenium.By
    import org.openqa.selenium.WebElement
    import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI
    WebUI.openBrowser('')
    WebUI.navigateToUrl('http://demoaut-mimic.kazurayam.com/6967_testbed.html')
    List<Map<String,String>> data =  [
        ["text":"Tokyo CURA Healthcare Center"],
        ["text":"Hongkong CURA Healthcare Center"],
        ["text":"Seoul CURA Healthcare Center"],
        ["text":"New York CURA Healthcare Center"]
    ]
    List<WebElement> webElementsInPage =
        CustomKeywords.'com.kazurayam.ksbackyard.FindElementsByXPath.getWebElementsAsList'('//select[@name="facility"]/option')
    WebUI.comment("webElementsInPage.size()=${webElementsInPage.size()}")
    for (int i = 0; i < data.size(); i++) {
        data[i].found = 'no'
        for (WebElement el: webElementsInPage) {
            WebElement node = el.findElement(By.xpath('.'))  // you can further search for any descendant nodes of the WebElement
            String t = node.getText().trim()
            if (t == data[i].text) {
                data[i].found = 'yes'
            }
        }
    }
    for (Map m : data) {
        WebUI.comment("${m.text} is displayed:${m.found}")
    }
    WebUI.closeBrowser()
    

    ## Result

    When I execute my test case, I got the following output in the log:

    >>> Tokyo CURA Healthcare Center is displayed: yes
    ...
    >>> Hongkong CURA Healthcare Center is displayed: yes
    ...
    >>> Seoul CURA Healthcare Center is displayed: yes
    ...
    >>> New York CURA Healthcare Center is displayed: no
    

    This is what I wanted to see.

    ## Extensibility

    The custom keyword getWebElementsAsList, which I presented here, returns List<org.openqa.selenium.WebElement> into the scope of test case in the Katalon Studio. This keyword would encourage you to develop your test cases using both of Katalon Studio’s built-in keywords and the native API of the Selenium WebDriver. How can you extend this example? — Well, it is up to you. You can do in the Katalon Studio whatever as far as the WebDriver API supports.

    ## Demo at GitHub

    You can check out a demo project from the following GitHub repository:

  • 3 Likes

    I have made one more example here:

    This demo shows you how to perform nested iterations over an instance of List and multiple instances of List contained there.

    ----

    The target HTML of this demo is like this:

    <!DOCTYPE html>
    <html>
    <head>
      <meta http-equiv="X-UA-Compatible" content="IE=edge">
      <meta charset="utf-8">
      <meta name="author" content="">
      <meta name="viewport" content="width=device-width, initial-scale=1">
      <link rel="stylesheet" href="">
      <!--[if lt IE 9]>
      <script src="//cdn.jsdelivr.net/html5shiv/3.7.2/html5shiv.min.js"></script>
      <script src="//cdnjs.cloudflare.com/ajax/libs/respond.js/1.4.2/respond.min.js"></script>
      <![endif]-->
      <link rel="shortcut icon" href="">
      <title>Discussion #7520</title>
      <meta name="description" content="client specified description">
      <link rel="canonical" href="https://example.canonical.com" />
      <link rel="stylesheet"
      href="https://stackpath.bootstrapcdn.com/bootstrap/4.1.1/css/bootstrap.min.css"
      integrity="sha384-WskhaSGFgHYWDcbwN70/dfYBj47jz9qbsMId/iRN3ewGhXQFZCSftd1LZCfmhktB"
      crossorigin="anonymous">
    </head>
    <body>
    <div class="container">
        <p><a href="https://forum.katalon.com/discussion/7520/the-xpath-for-the-url-is-correct-but-not-visible-if-added-via-addproperty#latest">
            The "xpath" for the url is correct but not visible if added via addProperty</a></p>
        <table class="table">
            <tbody>
                <tr>
                    <th>1<th>
                    <td>
                        <div>
                            <div>div1</div>
                            <div>div2</div>
                            <div>div3</div>
                            <div>div4</div>
                            <div>div5</div>
                            <div>div6</div>
                            <div>div7</div>
                            <div>div8
                                <div>
                                    <table class="table">
                                        <tbody>
                                            <tr><th>A</th></tr>
                                            <tr><th>B</th></tr>
                                            <tr><th>C</th></tr>
                                            <tr><th>D</th></tr>
                                            <tr><th>E</th></tr>
                                            <tr>
                                                <th>F</th>
                                                <td>(1)</td>
                                                <td>(2)</td>
                                                <td>415074</td>
                                                <td><div><div><a href="#">Money Map Press</a></div></div></td>
                                                <td>(5)</td>
                                                <td>(6)</td>
                                                <td><a href="#">1</a></td>
                                            </tr>
                                        </tbody>
                                    </table>
                                </div>
                            </div>
                        </div>
                    </td>
                </tr>
                <tr>
                    <th>2</th>
                </tr>
            </tbody>
        </table>
    </div>
    <!-- Optional JavaScript -->
    <!-- jQuery first, then Popper.js, then Bootstrap JS -->
    <script src="https://code.jquery.com/jquery-3.3.1.slim.min.js"
        integrity="sha384-q8i/X+965DzO0rT7abK41JStQIAqVgRVzpbzo5smXKp4YfRvH+8abtTE1Pi6jizo"
        crossorigin="anonymous"></script>
    <script src="https://cdnjs.cloudflare.com/ajax/libs/popper.js/1.14.3/umd/popper.min.js"
        integrity="sha384-ZMP7rVo3mIykV+2+9J3UJ46jBk0WLaUAdn689aCwoqbBJiSnjAK/l8WvCWPIPm49"
        crossorigin="anonymous"></script>
    <script src="https://stackpath.bootstrapcdn.com/bootstrap/4.1.1/js/bootstrap.min.js"
        integrity="sha384-smHYKdLADwkXOn1EmN1qk/HfnUcbVRZyYmZ4qpPea6sjB/pTJ0euyQp0Mk8ck+5T"
        crossorigin="anonymous"></script>
    </body>
    </html>
    

    Please find here nested

    elements. In more realistic case I will have far larger number of ,
    elements. This sample HTML is a testbed.

    I want to verify text displayed in the page against data in hand.

    My test case is as follows:

    import static com.kms.katalon.core.testobject.ObjectRepository.findTestObject
    import org.openqa.selenium.By as By
    import org.openqa.selenium.WebElement as WebElement
    import com.kms.katalon.core.model.FailureHandling as FailureHandling
    import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI
    WebUI.openBrowser('')
    WebUI.navigateToUrl('http://demoaut-mimic.kazurayam.com/7520_testbed.html')
    WebUI.verifyElementPresent(findTestObject('Page_Discussion 17958/td_415074'), 10, FailureHandling.STOP_ON_FAILURE)
    List<Map<String,String>> expectedContents = [
      ["CID":"1", "Campain":"dummy", "Clicks": "dummy"],
      ["CID":"2", "Campain":"dummy", "Clicks": "dummy"],
      ["CID":"3", "Campain":"dummy", "Clicks": "dummy"],
      ["CID":"4", "Campain":"dummy", "Clicks": "dummy"],
      ["CID":"5", "Campain":"dummy", "Clicks": "dummy"],
      ["CID":"415074", "Campain":"Money Map Press", "Clicks": "1"]
    ]
    List<WebElement> TRs =  CustomKeywords.
        'com.kazurayam.ksbackyard.FindElementsByXPath.getWebElementsAsList'(
            '/html/body/div[@class="container"]/table/tbody/tr[1]/td/div/div[8]/div/table/tbody/tr')
    WebUI.comment("TRs.size()=${TRs.size}")
    for (int i = 0; i < TRs.size(); i++) {
      // debug print
      List<WebElement> children = TRs[i].findElements(By.xpath('./child::*'))
      for (WebElement child :children) {
        WebUI.comment("indexFound=${i} child.getTagName()=${child.getTagName()} child.getText()=${child.getText()}")
      }
      //
      List<WebElement> TDs = TRs[i].findElements(By.xpath('./td'))
      if (TDs.size() >= 7) {
        String cid     = TDs[2].getText()
        String campain = TDs[3].getText()
        String clicks  = TDs[6].getText()
        WebUI.comment("indexFound=${i}, cid=${cid}, campain=${campain}, clicks=${clicks}")
        // now do verification    WebUI.verifyEqual(cid,     expectedContents[i].CID,     FailureHandling.CONTINUE_ON_FAILURE)
        WebUI.verifyEqual(campain, expectedContents[i].Campain, FailureHandling.CONTINUE_ON_FAILURE)
        WebUI.verifyEqual(clicks,  expectedContents[i].Clicks,  FailureHandling.CONTINUE_ON_FAILURE)
      }
    }
    WebUI.closeBrowser()
    

    When I execute this test case, I got this output in the log:

    [INFO]   - indexFound=5, cid=415074, campain=Money Map Press, clicks=1
    [INFO]   - Comparing actual object '415074' with expected object '415074'
    [PASSED] - Actual object '415074' and expected object '415074' are equal
    [INFO]   - Comparing actual object 'Money Map Press' with expected object 'Money Map Press'
    [PASSED] - Actual object 'Money Map Press' and expected object 'Money Map Press' are equal
    [INFO]   - Comparing actual object '1' with expected object '1'[PASSED] - Actual object '1' and expected object '1' are equal
    

    This output is what I wanted to see. My success.

    2 Likes

    I happened to find that com.kms.katalon.core.webui.keyword.WebUIBuiltinKeywords class implements a method:

    static List findWebElements(TestObject to, int timeout)

    The API document says “Internal method to find web elements by test object”

    ーーーーーーーーーーーーー

    Ah, this is exactly the same as I have made. I have reinvented a wheel!

    3 Likes

    Katalon team labeled “Internal method” to it. If the Katalon Documentation page “[WebUI] Element” covered this ‘internal’ method, I would not have spent time for reinventing the wheel.

    1 Like

    Hi ,

    I have used your custom method,
    like this
    println(getWebElementsAsList("//input[@type=‘checkbox’]").size())
    WebUI.click(getWebElementsAsList("//input[@type=‘checkbox’]").get(2), FailureHandling.STOP_ON_FAILURE)

    iam able to print number of elements but second code webui.click is not working ,
    Throwing the below error

    01-21-2020 03:35:54 PM User selects single test from location list

    Elapsed time: 19.851s

    Location.selectSingleLocation:160

    User selects single test from location list FAILED.
    Reason:
    groovy.lang.MissingMethodException: No signature of method: static com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords.click() is applicable for argument types: (org.openqa.selenium.support.events.EventFiringWebDriver$EventFiringWebElement, com.kms.katalon.core.model.FailureHandling) values: [[[CChromeDriver: chrome on WINDOWS (9f70dd5a87d2f2222e856d1269e94770)] -> xpath: //input[@type=‘checkbox’]], …]
    Possible solutions: click(com.kms.katalon.core.testobject.TestObject, com.kms.katalon.core.model.FailureHandling), click(com.kms.katalon.core.testobject.TestObject), check(com.kms.katalon.core.testobject.TestObject, com.kms.katalon.core.model.FailureHandling), back(), back(com.kms.katalon.core.model.FailureHandling), check(com.kms.katalon.core.testobject.TestObject)
    at Location.selectSingleLocation(Location.groovy:160)
    at ✽.User selects single test from location list(D:/mywork/KS/Webproject/TestEhswatch/Include/features/Location.feature:45)

    Please help

    getWebElementsAsList() returns a List<org.openqa.selenium.WebElement>.

    WebUI.click(xxx) requires argument of type com.kms.katalon.core.test.object.TestObject.

    Type mismatching.

    Therefore WebUI.click(getWebElementsAsList(...).get(x)) will never work.


    If you want to use WebUI.click(...) then you should NOT use getWebElementsAsList(). If you want to use WebUI.click(...) you should stick to findTestObject("...")

    If you rather want to use getWebElementsAsList(), then you can no longer use WebUI.xxxx keywords. You need use Selenium API to work on objects of class org.openqa.selenium.WebElement in the Test Case script. Using Selenium API in the KS Test Case is totally valid, would certainly work.

    There is a function
    WebUiBuiltInKeywords.convertWebElementToTestObject( … )

    Is it an answer to your problem ?

    Oh, I didn’t know that. It looks nice, worth trying.

    quoting the javadoc of the method:

    @CompileStatic@com.kms.katalon.core.annotation.Keyword(keywordObject = StringConstants.KW_CATEGORIZE_ELEMENT)static com.kms.katalon.core.testobject.TestObject convertWebElementToTestObject(org.openqa.selenium.WebElement webElement, com.kms.katalon.core.model.FailureHandling flowControl)
    Convert a WebElement to a TestObject. It will create a Test Object with no name that wraps around the given WebElement. When the Test Object is used by built-in keywords, it is unwrapped and the given WebElement will be used
    throws:
    StepFailedException
    Returns:
    a TestObject that wraps around the given WebElement
    Parameters:
    webElement - the WebElement retrieved by Selenium or other APIs
    flowControl - failureHandling
    Since:
    6.2.0