How to use Katalon TestOps Agent to schedule and remotely execute a Katalon Studio project

Hi everyone,

In this tutorial, I’ll show you how to use Katalon TestOps Agent to schedule and remotely execute a Katalon Studio project.

Katalon TestOps Agent will help you:

  • Install and manage multiple versions of Katalon Runtime Engine on your test machines without any manual intervention.
  • Retrieve the latest version of your Katalon Studio projects that are stored on Katalon TestOps or Git to execute, and submit the test results back to Katalon TestOps for review.
  • Keep the number of concurrent executions under a configurable limit. Katalon TestOps will distribute jobs among Agents to balance their workload. This feature is especially helpful if you only have a limited number of licenses and want to queue your executions periodically.

Prepare a sample Katalon Studio project

You can find the sample project here https://github.com/katalon-studio-samples/test-project-sample. This project contains a single test case (Form) that fills information into a form. The test case is put in a Test Suite (Simple Test Suite) and this test suite is put in a Test Suite Collection (Simple Test Suite Collection).

Integrate with Katalon TestOps

Connect this sample project to a Katalon TestOps project.

This way execution results will be submitted back to Katalon TestOps. It also helps monitor the progress of long-running executions.

Install Katalon TestOps Agent on a test machine

Navigate to Katalon TestOps CI.

From here, navigate to the Agent Setup wizard.

In this wizard, provide my machine’s OS (Windows x64) and download the package from the link (https://github.com/katalon-studio/katalon-agent/releases/download/v1.4.1/katalon-agent-win-x64-v1.4.1.zip). In this tutorial, I will use Alex laptop as my Agent’s name, and select an API key.

Next, unzip the Agent package and open the folder in the CMD window. Copy the two commands shown on the wizard and run them one by one.

After about a minute, the new Test Environment should be added.

Submit the Katalon Studio project to Katalon TestOps CI

Click on the “Cloud” button, set the Test Project name, and click Upload.

You can navigate to Test Projects in Katalon TestOps CI to see the newly uploaded Test Project.

Create a Test Plan on Katalon TestOps CI

Navigate to Test Plan in Katalon TestOps CI and click on the Create Plan button.

Next, fill out the information in the wizard. After that, click on the Create button. It’s possible to select multiple Test Environments - the workload will be distributed evenly between them.

To verify my configuration, click on the Run button.

Navigate to the detail page of the newly created Job, and it shows that the Agent is running the Job. You can monitor the Job via the logs that are pushed on-the-fly by the Agent.

The running execution can also be monitored in Katalon TestOps Center.

Use Git repository as a Test Project

Alternatively, Github repository can be used as a Test Project. Since this is a public Github repository, I will left Username and Password empty. For private repositories, an account with username and password (or personal access token in case they are hosted on Github) is required.

The wizard to create a Plan will change a bit. Note that -apiKey and -projectPath are unnecessary since they will be appended by the Agent.

Limit Agent’s capacity based on the number of Katalon Runtime Engine licenses

Katalon TestOps provide a way to impose a limit on how many jobs an Agent can run at the same time. Exceeding jobs will not be canceled - they are put into a queue and will be executed when slots are available.

This is a very handy feature. Say, you have two Katalon Runtime Engine licenses. In this case, there is no point running three executions at the same time, since Katalon Runtime Engine will cancel the third session.

Next step

You can keep the Agent always running by installing it as a service.

Thank you for all the feedback and suggestions for the features.

Happy testing!

12 Likes

Hi Everyone,

I created a Katalon TestOps Agent to schedule and remote execution of katalon studio project stored on Git.

Able to create the Git Project successfully in Katalon TestOps.


but Unable to fetch the Test Suite Collection/Test Suite from the Git Project repository.
I tried to execute the project using Katalon Command in Test Plan of TestOps which throws the following error.

Could someone Please help on this

1 Like

Hi @puneethm,

Thank you for your feedback.

We are working on this issue and will let you know as soon as it’s ready.

Cheers,
Beo

Hi @puneethm,

Please add or update the logLevel property in the agentconfig file to logLevel=ALL to enable all log levels in the debug.log. Then, rerun the plan.

Also, if possible, please send us the app.log and errors.log files in the log directory of the Agent’s setup location.

Thank you!

Hi sonpham,

As mentioned, I have added the logLevel=ALL in agentconfig file to enable all log levels in debug.log and tried to execute even then I am getting the same in debug.log

app.log and errors.log
app.log (139.8 KB) errors.log (68.5 KB)

Could someone help on this.

Thanks in advance.

2 Likes

Hi @puneethm,

It looks like your test project cannot be checked out due to file path length restriction on Windows. Please try enabling long path configuration for Git following this thread.

Hope it helps!

2 Likes

Thanks! it worked.

1 Like

Hi @sonpham

Expecting the below features in Katalon TestOps

  1. Unable to find the Test Suite collection in TestPlan. While configuring/creating the TestPlan, Option to select the Test Suite collection for git project. If it already exists helps me how to find it.
  2. While creating/Configuring a TestPlan in TestOps, In Katalon Command mode->Dropdown for Changing the parameters like browserType and executionProfile, so it makes more flexible for selction of parameters or adding new parameters.
  3. If we buy katalon license, is perfecto plugin will be accessed?

Please help me with the above details.

Thanks!

1 Like

Hi @puneethm,

Please find below the information on your feature requests.

We are working on these features and they will be available in the next release.

Yes, you can access all plugins if you purchase the Enterprise package.

Hope this helps.

Hi team,
Can Katalon Testops Agent support Webhook to trigger test execution? For example, push code on GitHub to trigger the automatic execution of test cases on the Katalon Testops CI.
Thank you very much!

Hi @Cedar67,
Thanks for your suggestion, we are working on this item and will release it soon.

2 Likes

@Thong Thanks for your reply! We are looking forward to it.

1 Like

is this still supported? This can be done without runtime engine license?

I did all the steps above but my test doesn’t run - the execution just has the spinner but nothing happens

Hello! I’ve scheduled a test run and I can see it’s failed, but it still has the blue wheel with the message ‘this test run is in progress’.

Can anyone explain why this is happening? I know I can stop it myself, but I don’t understand why it won’t automatically stop by itself.

I’m still fairly new to Katalon, so apologies if it’s a simple solution that I may not yet know. :slight_smile:

I’m still on the trial version, if this makes any difference.

@Lisa1,

Thank you for let us know. This is a known issue when the agent encounters an error. We will fix it soon.

I’ve checked your Test Run and the error happened because your machine did not have git installed. Please install git and try again.

1 Like

@ashraf.madina KRE license is required for this function.

We have KSE, was asking if Runtime engine license was needed…but i don’t think it does. I got mostly thru the instructions here…but the test run does not execute in the laptop that I set as agent.

Is it possible to choose which profile test suites should be using when running the tests?

Yes, you can generate the command line from Katalon Studio which includes an option to specify a profile.

Anyone could help with this error ?

Do i need to have Katalon Enteprise license to use the agent?
It was working fine a couple days ago