Suggestions for navigating a Windows Tree View object

Product Forums Katalon Studio
, ,
1

Is there a recommended or preferred way to navigate arbitrary Windows application treeviews?

Background, research and code…

I am automating a test of TreeView navigation in a circa 2014 native Windows application.

The app behaves similarly to how Windows File Explorer displays files and folders in its left-hand navigation panel.

My goal is to programatically navigate the tree without direct knowledge of the child element names in advance.

I know the parent and can find it. I’m not sure how to find the children.

For example, given only the “windows” folder, find and process the children “ASUS”, then “Shortcuts”, etc.:

c:\windows
|-- ASUS
| |-- Shortcuts

My first broad assumption is that the tree view is internally similar to nested web page elements in HTML (DOM tree): , ,

, etc.

I’ve tried a few possibilities, such as one I found on StackOverflow for a web app that does a findElement() for the parent, then does findElements() for the children.

My code is almost identical to the StackOverflow code, except for the “Windows” references.

  WindowsTestObject wo2 = new WindowsTestObject('')
  wo2.setLocatorStrategy(WindowsTestObject.LocatorStrategy.NAME)
  wo2.setLocator('XYZ')
  Windows.doubleClick(wo2)   // This works!  The tree under XYZ has many children that appear after double-clicking
  WebElement parent = Windows.findElement(wo2);
  println "Parent: " + parent.toString()

  List<WebElement> treeItems = parent.findElements(By.xpath("*"))
  println "TreeItems.size: " + treeItems.size()
  treeItems.each {println it}

Here is the console output:

  //> Parent: [[io.appium.java_client.windows.WindowsDriver, Capabilities: {appTopLevelWindow=8059e, javascriptEnabled=true, platform=WINDOWS, platformName=WINDOWS}] -> name: XYZ]
  //> TreeItems.size: 1
  //> treeItems.each({ -> ... })
  //> [[[[io.appium.java_client.windows.WindowsDriver, Capabilities: {appTopLevelWindow=2305c8, javascriptEnabled=true, platform=WINDOWS, platformName=WINDOWS}] -> name: XYZ]] -> xpath: *]

I’m not sure this is even a good way to do it, but I’ve exhausted my scanty knowledge and Google skills.

My thoughts:

  • Windows may not work quite the same as the WebUI
  • Tree view elements appear nested in XPATH, but may not actually be
  • Its possible the first findElement call for “parent” doesn’t refresh the page information with the newly opened tree after double-clicking
  • I’m not sure how to get Spy to work since the app requires login, then switches Windows - native recorder works, but of course the results are static

Using the XPATH from the Object Repository doesn’t seem to fair any better, for example:

WebElement parent = Windows.findElement(By.xpath("/Window/Custom[1]/Custom[2]/TreeView[1]/TreeViewItem[1]/TreeViewItem[8]/Text[1]"));

This seems to work, but still no children are found, even though I know the XPATH for the children. such as:
"/Window/Custom[1]/Custom[2]/TreeView[1]/TreeViewItem[1]/TreeViewItem[8]/Text[1]/TreeViewItem[3]"));

Any suggestions are welcome.

If I missed a similar post, a pointer would be great!

Thanks,

Jim

3

To be clear… the last lines of the code above FAIL. Even though I doubleclick and expand the treeview and see elements beneath the parent, the wildcard Xpath search returns nothing, as shown in the console output.

So my question is actually “IS there a way?”, not “Is there a better way?”, because I haven’t found a way to access children in the treeview.

4

Hi @james_bay,
First of all, sorry for my late response

Windows may not work quite the same as the WebUI

It’s true because Windows test is based on WinAppDrivers, WebUI test based on chromedriver for Chrome, geckodriver for Firefix, safaridriver for Safari,… Both of them follow W3C API standards but some of them don’t implement some of APIs.

  • Its possible the first findElement call for “parent” doesn’t refresh the page information with the newly opened tree after double-clicking

When your app/page need to be refreshed, you should find the element again.

List treeItems = parent.findElements(By.xpath(“*”))
println "TreeItems.size: " + treeItems.size()
treeItems.each {println it}

This code should work for Web, Mobile but not for Windows because WinAppDriver doesn’t support this. You can try the solution in this post: Quickly finding all child elements · Issue #669 · microsoft/WinAppDriver · GitHub.

Another recommendation: You should upgrade WinAppDriver to the latest version: Release WinAppDriver v1.2.1 · microsoft/WinAppDriver · GitHub. Microsoft has recently released this new version and may have some improvements for the driver.

Thanks

5

Thanks SO much for your reply!

I did three things:

  • I upgraded the WinAppDrv to 1.2.1
  • I added a “refresh” to find the parent element again after the double-click
  • Per the article you suggested, I changed the Xpath search to “star-slash-star” ("*/*")

The results are different! But not quite what I was hoping for. I’m going over the article you referenced in more detail, because I feel like I’m on the right track.

But I thought I’d post the output in case you had any other recommendations.

In the application, when I double click on the parent, the tree opens to reveal 11 children. Three of the children have children in turn, as indicated by a small arrowhead pointing to the right (“closed”).

The code that iterates through the results of the “*/*” returns the same toString 14 times.

I believe that what is being returned are each of the children and EACH OF THE ARROWHEADS (11 children + three arrows = 14 “objects”).

If the objects being returned are some sort of pointers to the children, how do I access them? I expected to see a list of the children’s NAMES. But instead, the toStrings are identical.

Here is the console output:

2020-11-19 15:50:33.541 DEBUG testcase.fullexport2                     - 46: parent = findElement(wo2)
2020-11-19 15:50:34.375 DEBUG testcase.fullexport2                     - 47: treeItems = parent.findElements(By.xpath("*/*"))
2020-11-19 15:50:35.388 DEBUG testcase.fullexport2                     - 48: treeItems.each({ -> ... })
[[[[io.appium.java_client.windows.WindowsDriver, Capabilities: {appTopLevelWindow=7052c, javascriptEnabled=true, platform=WINDOWS, platformName=WINDOWS}] -> name: AIZENBERG]] -> xpath: */*]
[[[[io.appium.java_client.windows.WindowsDriver, Capabilities: {appTopLevelWindow=7052c, javascriptEnabled=true, platform=WINDOWS, platformName=WINDOWS}] -> name: AIZENBERG]] -> xpath: */*]
[[[[io.appium.java_client.windows.WindowsDriver, Capabilities: {appTopLevelWindow=7052c, javascriptEnabled=true, platform=WINDOWS, platformName=WINDOWS}] -> name: AIZENBERG]] -> xpath: */*]
[[[[io.appium.java_client.windows.WindowsDriver, Capabilities: {appTopLevelWindow=7052c, javascriptEnabled=true, platform=WINDOWS, platformName=WINDOWS}] -> name: AIZENBERG]] -> xpath: */*]
[[[[io.appium.java_client.windows.WindowsDriver, Capabilities: {appTopLevelWindow=7052c, javascriptEnabled=true, platform=WINDOWS, platformName=WINDOWS}] -> name: AIZENBERG]] -> xpath: */*]
[[[[io.appium.java_client.windows.WindowsDriver, Capabilities: {appTopLevelWindow=7052c, javascriptEnabled=true, platform=WINDOWS, platformName=WINDOWS}] -> name: AIZENBERG]] -> xpath: */*]
[[[[io.appium.java_client.windows.WindowsDriver, Capabilities: {appTopLevelWindow=7052c, javascriptEnabled=true, platform=WINDOWS, platformName=WINDOWS}] -> name: AIZENBERG]] -> xpath: */*]
[[[[io.appium.java_client.windows.WindowsDriver, Capabilities: {appTopLevelWindow=7052c, javascriptEnabled=true, platform=WINDOWS, platformName=WINDOWS}] -> name: AIZENBERG]] -> xpath: */*]
[[[[io.appium.java_client.windows.WindowsDriver, Capabilities: {appTopLevelWindow=7052c, javascriptEnabled=true, platform=WINDOWS, platformName=WINDOWS}] -> name: AIZENBERG]] -> xpath: */*]
[[[[io.appium.java_client.windows.WindowsDriver, Capabilities: {appTopLevelWindow=7052c, javascriptEnabled=true, platform=WINDOWS, platformName=WINDOWS}] -> name: AIZENBERG]] -> xpath: */*]
[[[[io.appium.java_client.windows.WindowsDriver, Capabilities: {appTopLevelWindow=7052c, javascriptEnabled=true, platform=WINDOWS, platformName=WINDOWS}] -> name: AIZENBERG]] -> xpath: */*]
[[[[io.appium.java_client.windows.WindowsDriver, Capabilities: {appTopLevelWindow=7052c, javascriptEnabled=true, platform=WINDOWS, platformName=WINDOWS}] -> name: AIZENBERG]] -> xpath: */*]
[[[[io.appium.java_client.windows.WindowsDriver, Capabilities: {appTopLevelWindow=7052c, javascriptEnabled=true, platform=WINDOWS, platformName=WINDOWS}] -> name: AIZENBERG]] -> xpath: */*]
[[[[io.appium.java_client.windows.WindowsDriver, Capabilities: {appTopLevelWindow=7052c, javascriptEnabled=true, platform=WINDOWS, platformName=WINDOWS}] -> name: AIZENBERG]] -> xpath: */*]
[[[[io.appium.java_client.windows.WindowsDriver, Capabilities: {appTopLevelWindow=7052c, javascriptEnabled=true, platform=WINDOWS, platformName=WINDOWS}] -> name: AIZENBERG]] -> xpath: */*]
2020-11-19 15:50:35.393 INFO  c.k.katalon.core.main.TestCaseExecutor   - END Test Cases/fullexport2

Here is the output from WinAppDrv:

User-Agent: selenium/3.141.59 (java windows)

{
  "id": "7.15012.49211517",
  "using": "xpath",
  "value": "*\u002f*"
}
HTTP/1.1 200 OK
Content-Length: 538
Content-Type: application/json

`{"sessionId":"EB987586-28E0-4EEC-B1F1-9C7DD9B44112","status":0,"value":[{"ELEMENT":"7.15012.41852331"},{"ELEMENT":"7.15012.41126667"},{"ELEMENT":"7.15012.34595685"},{"ELEMENT":"7.15012.42925716"},{"ELEMENT":"7.15012.10917859"},{"ELEMENT":"7.15012.31151867"},{"ELEMENT":"7.15012.11931350"},{"ELEMENT":"7.15012.40273294"},{"ELEMENT":"7.15012.26915331"},{"ELEMENT":"7.15012.40911390"},{"ELEMENT":"7.15012.32658192"},{"ELEMENT":"7.15012.2548827 [...]`

I believe the ELEMENT value shown in the WinAppDrv output is a unique object ID.

But how can I use that in my script? I would have hoped to see that in the toString output.

Jim

(P.S. Thanks again)

6

I happened to enter this code in another post, and then remembered this post I made, and thought I’d put the answer here as well, in case someone else comes looking for it.

The problem is processing a treeview, like the folder view in the left-hand pane of Windows File Explorer. The nodes are all collapsed. I could open them all up before the run, but I wanted to automate the process (which in turn depends on the nodes being closed… which I figured was easier than opening them all, since many apps have shortcut key to collapse all the nodes).

I eventually determined that first I need to click on a node in order to make it visible if its “offscreen”. Once its visible, I have to move focus away, because the next action on the selected node will go into edit mode. To do this, I send an up-arrow key. Originally I tried to click on the previous element, but I was not able to get it to “remember” the previous element. The click occurred somewhat randomly, occasionally working, but mostly not.

After the up-arrow, I can then double-click on the element, which then opens the node.

I can then do a findElment, which returns a list of the children of the node. If its empty, there are no children, so I pop out to the previous level and carry on.

I was delighted with how well it works. But as I said, the problem is that its stateful - it relies on the nodes all being closed in advance. If thats not the case, the double click will CLOSE the node, making it appear there are no children.

I’d love to find a better way to do this, and I have a feeling there will be one eventually, because although treeviews I don’t think are that common in web applications, they are very common in Windows applications, so a better way to do it would be very helpful.

Anyway, here’s the code. Hope it helps someone.

    // Open notebook treelist element and process contained experiments
    //
    //    .
    //    |-- Notebook1
    //    |   |-- Experiment 1
    //    |   |-- Experiment 2
    //    |-- Notebook2
    //    |   |-- Experiment 1
    //    |   |-- Experiment 2

    notebookNameTO.setLocatorStrategy(WindowsTestObject.LocatorStrategy.NAME)
    notebookNameTO.setLocator(notebookName) // e.g., "Notebook1"
    notebookNames = Windows.findElement(notebookNameTO);

    'Click on notebook'
    Windows.click(notebookNameTO)

    'Press up-arrow to take focus from current element'
    Windows.sendKeys(notebookNameTO, Keys.chord(Keys.ARROW_UP))

    'Doubleclick on notebook'
    Windows.doubleClick(notebookNameTO)

    'Populate list of experiment names for notebook - makes it "appear"'
    experimentNames = Windows.findElement(notebookNameTO);

    'Create list of experiments in notebook'
    List<WebElement> experiments = experimentNames.findElements(By.xpath("*/*"))

    // Iterate through experiments.
    for (experiment in experiments)  {
    .
    .
    .
   }

I also found this function helpful for “copying” Windows Test Objects:

/*
 * Copy properties of one Windows Test Object (WTO) to another.
 */
void copyWTO( WindowsTestObject toWTO, WindowsTestObject fromWTO ) {
    toWTO.setLocatorStrategy(fromWTO.getLocatorStrategy())
    toWTO.setLocator(fromWTO.getLocator())
}
1 Like