Katalon Studio doesn't spy or record object from windows desktop application

Product Forums Katalon Studio
,
1

Good morning,
I am trying to record actions on a Desktop application but the Record function does not detect the objects; the screen is composed by buttons and text but the record (and also the spy) function does not detect anithing in the screen. Only top bar of the application is detected with close/minimize buttons for the window.
The application, and related objects, is written in QT framework c++ by means of Visual Studio.

I am using Katalon 9.x

On another similar application I also received: org.openqa.selenium.WebDriverException: An unknown error occurred in the remote end while processing the command. (WARNING: The server did not provide any stacktrace information)

Thanks

1 Like
2

Hi @l.gori,

Welcome to our community. Have you ever been able to detect it successfully? Can you please help share your code, error log, and steps to reproduce? Thank you

3

Good Morning,

I am using katalon version 9.x.

I open Katalon Studio, I access the Record Windows Actions function Ă  Windows recoreder, I edit WinAppDriver URL http://127.0.0.1:4723/ and capabilities ms:waitForAppLaunch value 25.

I put as Application File the path of the .exe and in the Application Title the name of the application window.

I press Start to recordĂ  the application starts, in the View section the application screen is displayed, but only buttons of the application top bar are detected by the recording function; the QML objects composing the application functionalities (written in QT framework) are not detected. I also try to press on Refresh Screen but nothing changes.

I do not receive any errors but not object are detected.

4

To resolve the issue of Katalon Studio not detecting objects in a QT-based Windows desktop application:

1. Verify QT Accessibility Settings

Ensure the QT application exposes UI elements to accessibility tools:

  • Work with Developers: Request they add AccessibleName or AutomationId properties to QT controls in the code:
// Example for a QPushButton
QPushButton *button = new QPushButton("Submit", this);
button->setAccessibleName("submitButton"); // Mandatory for detection
  • Enable QT Automation Support: The app may need the QAxContainer module or compile flags for UI automation.

2. Use Inspection Tools

Confirm if elements are detectable by Windows automation frameworks:

  • Download inspect.exe from the Windows SDK.
  • Launch the app and use inspect.exe to check if elements have Name, AutomationId, or ControlType properties.

3. Configure WinAppDriver Properly

  • Start WinAppDriver Manually:
# Run as Administrator
"C:\Program Files (x86)\Windows Application Driver\WinAppDriver.exe"
  • Adjust Capabilities in Katalon:
"app": "C:\\Path\\To\\YourApp.exe",
"platformName": "Windows",
"deviceName": "WindowsPC",
"ms:waitForAppLaunch": "30" // Increase if app loads slowly

4. Use XPath or Custom Selectors

Manually define objects if the recorder fails:

  • Spy Object Manually:
    1. Open Object Repository > Add > Windows Object.
    2. Use inspect.exe to find the element’s RuntimeId or AutomationId.
    3. Define the object using XPath:
// Example XPath for a button
ObjectRepository.addWindowsObject("SubmitButton", "//Button[@AutomationId='submitButton']")

5. Modify Katalon Settings

  • Increase Timeouts:
    • In katalon.conf (under settings), add:
waitForElementTimeout = 60
  • Enable Advanced Logging:
    • In Katalon, go to Project Settings > Execution > Default Executors and check Enable advanced logging.

6. Alternative Tools for QT Automation

If WinAppDriver still fails:

  • Use QT-specific tools like Squish or Ranorex to automate the QT app, then integrate results into Katalon via reports.
  • Request a Hybrid Approach: Use C++ test hooks within the QT app to trigger actions via CLI or APIs.

7. Update Katalon & Dependencies

  • Upgrade to Katalon 10.x: Newer versions have improved Windows Desktop support.
  • Update WinAppDriver to the latest version.

8. Debugging Tips

  • Check WinAppDriver Logs:
    • Look for errors in the WinAppDriver console when interacting with the app.
  • Use Accessibility Insights: Microsoft’s Accessibility Insights for Windows can diagnose accessibility issues.

Example Workaround for Undetectable Elements

If elements still aren’t detected, use image-based automation as a last resort:

// Click using screen coordinates (relative to the window)
Windows.clickImage(findWindowsObject("YourWindow"), "C:\\submit_button.png")

Final Notes

  • QT apps often require explicit accessibility configurations. Collaboration with developers is key.
  • If deadlines are tight, consider using pywinauto or AutoHotkey scripts for QT automation, then call them from Katalon via Runtime.getRuntime().exec().
5

Perfect, thank you, the problem is solved.

It was necessary to set QT accessibility.

I tried with your solution to set setAccessibleName but of course it works only for the specific QML element.

I solved assigning the accessibility property to the type (es. graphical) of QML object instead. In this way It worked for all the QML object of that type.

Now all the QML object can be detected.

1 Like