Integrate desktop application test execution on Katalon TestOps

[sontm9_cert2]

I want to integrate desktop application test execution on Katalon TestOps, but cannot find information related to it.

Is it possible to run desktop application tests from Katalon TestOps?

[Elly_Tran]

Hi @sontm9_cert2,

Welcome to our community. To execute desktop app on TestOps, you can follow:

• Build test case on KS
• Configure TestOps Integration & Test Execution
• Use KRE to execute your desktop tests from the command line

Hope this can help. Thank you

[sontm9_cert2]

Thanks for you quick response, but my test scripts interact with Desktop app on windows (for example Calculator, NotePad++), not desktop browser.

[dineshh]

Here’s how to execute desktop application tests (Windows/macOS) using Katalon TestOps:

Yes, It’s Possible!

While Katalon TestOps doesn’t natively support desktop app testing out-of-the-box, you can achieve this by:

1. Using Katalon Studio to author desktop tests (via WinAppDriver/Appium for Windows or AppleScript for macOS).
2. Configuring a self-hosted TestOps Agent on a machine where the desktop app is installed.
3. Orchestrating executions via TestOps schedules.

Step-by-Step Guide

1. Author Desktop Tests in Katalon Studio

• For Windows Apps:
  Use Windows Application Driver (WinAppDriver) with Katalon’s Windows keyword library:

import com.kms.katalon.core.windows.keyword.WindowsBuiltinKeywords as Windows  
Windows.startApplication('path/to/your/app.exe')  
Windows.click(findWindowsObject('Object Repository/Button'))  

• For macOS Apps:
  Use Appium/AppleScript with custom keywords:

CustomKeywords.'com.example.mac.Keywords.launchApp'('AppName')  

2. Set Up a Self-Hosted TestOps Agent

• Install the Agent:
  1. In TestOps, go to Configurations > Environments > Add Environment > On-Premise.
  2. Download and install the TestOps Agent on the machine where the desktop app resides.
     
• Requirements:
  • Machine must have the desktop app installed.
  • For Windows: Enable WinAppDriver (run WinAppDriver.exe in background).
  • For macOS: Allow Accessibility Permissions for the app.

3. Link Tests to TestOps

• Upload Tests:
  1. Upload your Katalon project (with desktop test cases) to a Git repository.
  2. Connect the repo to TestOps via Integrations > Git.
• Create a Test Suite:
  In TestOps, create a test suite containing your desktop test cases.

4. Schedule Execution via TestOps

• Create a Schedule:
  1. Go to Test Planning > Schedules > Create Schedule.
  2. Select your test suite and the On-Premise Environment (with your self-hosted agent).
     TestOps Schedule Setup
• Run Tests:
  The agent will execute tests on the desktop app and report results back to TestOps.

Troubleshooting Tips

• WinAppDriver Issues:
  • Verify WinAppDriver is running: http://127.0.0.1:4723/status.
  • Use WindowsBuiltinKeywords.refreshDesktop() to reset the session.
• Agent Connectivity:
  • Ensure the agent machine has outbound internet access (for TestOps communication).
  • Check agent logs at %ProgramData%\Katalon_TestOps_Agent\logs.
• macOS Permissions:
  • Grant Terminal/TestOps Agent Full Disk Access in System Preferences > Security & Privacy.

Limitations

• Single-Session Only: TestOps can’t parallelize desktop tests on the same agent (unless multiple VMs are used).
• GUI Dependency: The agent machine must not lock/sleep during execution (use caffeinate on macOS or powercfg on Windows).

By following this approach, you can seamlessly integrate desktop app testing into your TestOps workflow.

[sontm9_cert2]

I have tried to setup Agent on local machine, but Katalon TestOps doesn’t support Window app when select an Agent as a test environment.

[dineshh]

To execute Windows desktop application tests via Katalon TestOps, even though there’s no native “Windows app” environment option, you can use a self-hosted TestOps Agent with a workaround. Here’s how:

Step 1: Prepare the Windows Machine & TestOps Agent

1. Install Prerequisites:

• Install WinAppDriver and start it as a service:

# Run WinAppDriver in the background (default port 4723)
"C:\Program Files (x86)\Windows Application Driver\WinAppDriver.exe"

• Install the desktop application you want to test (e.g., MyApp.exe).

2. Install TestOps Agent:

• In TestOps, create an On-Premises Environment:
  • Go to Configurations > Environments > Add Environment > On-Premise.
  • Download the agent and install it on the Windows machine.
• Ensure the agent runs with administrator privileges (required for WinAppDriver interactions).

Step 2: Configure Tests for Windows Desktop in Katalon Studio

1. Write Tests Using Windows Keywords:

import com.kms.katalon.core.windows.keyword.WindowsBuiltinKeywords as Windows
// Launch the Windows app
Windows.startApplication('C:\\Path\\To\\MyApp.exe')
// Interact with elements
Windows.click(findWindowsObject('Object Repository/Button_Submit'))

2. Commit Tests to Git:

• Upload your Katalon project (with Windows test cases) to a Git repository linked to TestOps.

Step 3: Set Up Execution in TestOps

1. Create a Generic Test Environment:

• In TestOps, label your on-premises environment as Windows-Desktop (manually tag it for clarity).

2. Schedule a Test Run:

• Go to Test Planning > Schedules > Create Schedule.
• Select your test suite and choose the On-Premises Environment (your Windows machine’s agent).
• Add a Tag Filter to target Windows-specific tests (optional).

Step 4: Handle Dependencies in the Agent Machine

1. Add Required Capabilities:

• In your Katalon test script, specify WinAppDriver’s endpoint:

"windows:winAppDriverUrl" -> "http://127.0.0.1:4723"

2. Kill Residual Processes:

• Use a @BeforeTestCase method to ensure no lingering app instances:

Runtime.getRuntime().exec("taskkill /f /im MyApp.exe")

Step 5: Run & Debug

1. Execute the Schedule:

• TestOps will trigger the agent to run tests on the Windows machine.

2. Troubleshoot Failures:

• Check Agent Logs: Located at C:\ProgramData\Katalon_TestOps_Agent\logs.
• Verify WinAppDriver Logs: Default logs at C:\Program Files (x86)\Windows Application Driver\Logs.

Workaround for “Unsupported Environment” Issues

If TestOps blocks execution for non-mobile environments:

1. Use a Mobile Capability Profile (hack):

• In Katalon Studio, create a mobile profile with dummy values:

"platformName" -> "Windows",  // Override later
"appium:app" -> "Microsoft.WindowsCalculator_8wekyb3d8bbwe!App"  // Example

• In TestOps, map this profile to your on-premises environment.

Alternative: Use CLI with Custom Scripts

1. Run Tests via Command Line on the Agent:

# On the Windows agent machine
katalonc -noSplash -runMode=console -projectPath="C:\Tests\Project.prj" -testSuitePath="Test Suites/Windows_Tests" -reportFolder="Reports" -reportFileName="report"

2. Push Results to TestOps:
   Use the TestOps API to upload results:

curl -X POST "https://analytics.katalon.com/api/v1/results/import" -H "Authorization: Bearer YOUR_API_KEY" -F "file=@C:\Reports\report.html"

Final Notes

• Parallel Execution: Use multiple agents on different Windows machines (each with WinAppDriver on unique ports).
• CI/CD Integration: Trigger TestOps schedules via Jenkins/GitHub Actions using the TestOps API.

By following these steps, you can bypass TestOps’ lack of native Windows app support and execute desktop tests reliably

[sontm9_cert2]

@dineshh
Could you please provide in detail for below steps?

When scheduling test execution with an agent as test environment, it only supports running with browsers or web service. I am not sure if this is a valid solution.

[dineshh]

Katalon TestOps categorizes environments as “Mobile”, “Web”, or “Web Service” – but not “Desktop.” By creating a fake mobile profile, you can bypass TestOps’ environment checks and run desktop tests on a self-hosted Windows agent.

Step 1: Create a Mobile Capability Profile in Katalon Studio

1. Navigate to Profile Settings:

• In Katalon Studio, go to Profiles > Default (or create a new profile).
• Under Mobile, add dummy capabilities to trick TestOps into accepting the environment:

"platformName": "Android", // Dummy value (required by TestOps)
"appium:app": "Microsoft.WindowsCalculator_8wekyb3d8bbwe!App" // Example dummy package

• Save the profile as WinAppDriver_Workaround.

Step 2: Override Capabilities in Test Scripts

In your Windows desktop test scripts, ignore the dummy profile and explicitly set the correct WinAppDriver capabilities:

import com.kms.katalon.core.windows.keyword.WindowsBuiltinKeywords as Windows

// Override capabilities for WinAppDriver
@Keyword
def startDesktopApp() {
  Map<String, String> capabilities = new HashMap<>()
  capabilities.put("platformName", "Windows") // Real value
  capabilities.put("app", "C:\\Path\\To\\YourApp.exe")
  capabilities.put("deviceName", "WindowsPC")
  capabilities.put("windows:winAppDriverUrl", "http://127.0.0.1:4723")

  // Start the Windows app
  Windows.startApplicationWithKeywords(capabilities)
}

// In your test case
startDesktopApp()
Windows.click(findWindowsObject('Button_Save'))

Step 3: Link the Profile to a TestOps Environment

1. Upload Tests to TestOps:

• Commit your Katalon project (with the WinAppDriver_Workaround profile) to Git.
• Connect the Git repo to TestOps (Integrations > Git).

2. Map the Profile to Your On-Premises Agent:

• In TestOps, go to Configurations > Environments.
• Select your on-premises (Windows) environment and assign the WinAppDriver_Workaround profile

Step 4: Schedule Execution in TestOps

1. Create a Test Suite:

• Add your Windows desktop test cases to a test suite.

2. Schedule a Run:

• Go to Test Planning > Schedules > Create Schedule.
• Select your test suite and the On-Premises Environment mapped to the WinAppDriver_Workaround profile.
• TestOps will treat this as a “mobile” test but execute it on your Windows agent.

How It Works

• TestOps Validation: TestOps checks if the selected environment has a valid mobile profile (which it does, thanks to the dummy values).
• Runtime Execution: During test execution, your scripts override the dummy profile with WinAppDriver settings, allowing Windows app interaction.

Critical Checks for Success

1. Agent Machine Configuration:

• WinAppDriver must be running (http://localhost:4723).
• The desktop app must be installed on the agent machine.
• The TestOps Agent service should run with Admin privileges.

2. Capability Overrides:

• Ensure your test scripts do not inherit the dummy profile values. Explicitly set WinAppDriver capabilities in code.

3. Execution Logs:

• Check Agent Logs at C:\ProgramData\Katalon_TestOps_Agent\logs for connectivity issues.
• Monitor WinAppDriver Logs at C:\Program Files (x86)\Windows Application Driver\Logs.

Troubleshooting

• Error: “No mobile device connected”:
  • Ensure the WinAppDriver_Workaround profile is assigned to the environment.
  • TestOps validates profiles during scheduling but doesn’t enforce device connectivity for on-premises agents.
• WinAppDriver Fails to Start:
  • Run WinAppDriver manually before tests:

cd "C:\Program Files (x86)\Windows Application Driver"
WinAppDriver.exe

Alternative Approach: CLI Execution

If TestOps still blocks execution, run tests directly on the agent machine via Katalon CLI and push results to TestOps:

katalonc -noSplash -runMode=console -projectPath="C:\Tests\Project.prj" -testSuitePath="Test Suites/Windows_Tests" -reportFolder="Reports" -reportFileName="report"

Then upload results using the TestOps API:

curl -X POST "https://analytics.katalon.com/api/v1/results/import" -H "Authorization: Bearer YOUR_API_KEY" -F "file=@C:\Reports\report.html"

By following these steps, you can effectively run Windows desktop tests on Katalon TestOps despite its lack of native support.

[sontm9_cert2]

@dineshh
Thank for your response but I cannot find “Under Mobile” in step 1, I am using Katalon Studio 9.7.5. Could you please provide the screenshot for this step?

[dineshh]