Test Suite Collection execution N-number of times

Web Testing
,
1

Hello to the community,

Here I am, again with a new limitation of Katalon studio.

I am using katalon studio Enterprise v-10.4.3 offline license, for automating web and desktop applications.

Scenario: I have created a TestSuiteCollection which run and cover 3 applications

TestSuiteCollection1

  1. TS1: WebApplication
    a. Group of TCs
  2. TS2:Desktop Application 1
    a. Group of TCs
  3. TS3:Desktop Application 2
    a. Group of TCs

Now I want to run TestSuiteCollection1 n-times. I search online and created TC which loops on the testSuiteCollection

suiteCollectionName = ‘01-Create and produce a batch to close without autobatches’
n= 10

for (int i = 0; i < n; i++) {
println “Running suite collection: ${suiteCollectionName} - Run #${i + 1}”
TestSuiteCollectionRunner.runTestSuiteCollection(suiteCollectionName)
}

but it looks like Katalon does not support and I get the below error while executing the above testcase

Test Cases/TestCase - Scenarios/Test Suite Collections Wrapper/Produce and close batch from IP-OFF-INT FAILED.Reason:org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:file:/C:/GIT/Projects/Parvis.Automation.Test/Scripts/TestCase%20-%20Scenarios/Test%20Suite%20Collections%20Wrapper/Produce%20and%20close%20batch%20from%20IP-OFF-INT/Script1775033157942.groovy: 19: unable to resolve class com.kms.katalon.core.main.TestSuiteCollectionRunner@ line 19, column 1.import com.kms.katalon.core.main.TestSuiteCollectionRunner^

1 error

at com.kms.katalon.core.main.ScriptEngine.getScript(ScriptEngine.java:199)
at com.kms.katalon.core.main.ScriptEngine.run(ScriptEngine.java:194)
at com.kms.katalon.core.main.ScriptEngine.runScriptAsRawText(ScriptEngine.java:119)
at com.kms.katalon.core.main.TestCaseExecutor.runScript(TestCaseExecutor.java:490)
at com.kms.katalon.core.main.TestCaseExecutor.doExecute(TestCaseExecutor.java:481)
at com.kms.katalon.core.main.TestCaseExecutor.processExecutionPhase(TestCaseExecutor.java:460)

Is there any way to loop over a testSuiteCollection?

Many thanks in advance

1 Like
2

Hi there, and thanks for posting in the Katalon community! :hugs:

To help you faster, please review our guide on Git here:

Double-checking the steps and configurations might resolve the issue.

If the doc doesn’t help, feel free to provide more details, and a community member will assist you soon. Thanks for being a part of our community!

Best,
Elly Tran

3

I have never seen the TestSuiteCollectionRunner class. Could you explain what it is?

4

Exactly, this is something katalon doesnt support but this class was suggested by Katalon AI assistant

5

Do you have a Katalon Runtime Engine license?

If so you would be able to do something in a bash shell (or PowerShell) script that invoke your test suites in the command line.

If you do not have KRE license and you want to do it in Katalon Studio IDE, you can repeat delaring TS1 for n-times in your Test Suite Collection. I think it would be good enough. Aren’t you satisfied with it? Then please explain why.

6

Hello @kazurayam Thanks for your suggestions

Related to KRE, we do not have KRE yet and we are not planning in near future.
For repeating the TS multiple time is not a good solution to manage large number of tests.

  1. We will have alot of duplications in case we have to run 20 or 30 times
  2. What if after few day we need to increment or decrement the re-execution TS. in case of loop we just have to change the N variable value

I again ask, is there any other way to do it? without KRE and duplicating TS?

7

I think your requirement is really an Enterprise requirement, which deserves purchasing KRE.

I am sure, Katalon Stuido does not provide such a feature.

If you look into the project on disk, you will find a Test Suite Collection is represented as a simple XML file. For example,

<?xml version="1.0" encoding="UTF-8"?>
<TestSuiteCollectionEntity>
   <description></description>
   <name>TSC1</name>
   <tag></tag>
   <delayBetweenInstances>0</delayBetweenInstances>
   <executionMode>SEQUENTIAL</executionMode>
   <maxConcurrentInstances>8</maxConcurrentInstances>
   <testSuiteRunConfigurations>
      <TestSuiteRunConfiguration>
         <configuration>
            <groupName>Web Desktop</groupName>
            <profileName>default</profileName>
            <requireConfigurationData>false</requireConfigurationData>
            <runConfigurationId>Chrome</runConfigurationId>
         </configuration>
         <runEnabled>true</runEnabled>
         <testSuiteEntity>Test Suites/main/TS1_demoaut.katalon.com</testSuiteEntity>
      </TestSuiteRunConfiguration>
      <TestSuiteRunConfiguration>
         <configuration>
            <groupName>Web Desktop</groupName>
            <profileName>default</profileName>
            <requireConfigurationData>false</requireConfigurationData>
            <runConfigurationId>Chrome</runConfigurationId>
         </configuration>
         <runEnabled>true</runEnabled>
         <testSuiteEntity>Test Suites/main/TS3_process_large_HAR</testSuiteEntity>
      </TestSuiteRunConfiguration>
   </testSuiteRunConfigurations>
</TestSuiteCollectionEntity>

No more complicated data structure is involved.

You would be able to write a script code that generates such XML file so that it satisfies whatever you require. Any programming tools you can choose for this custom task.

Be aware: when you update the XML file on disk, you need to “reflesh” your project; or close&open your project to let Katalon Studio be informed.

9

Ah, how interesting! :laughing:

The Katalon AI assistant must be a dreamy boy.

2 Likes
10

Thanks @kazurayam for the response

currently this specific testSuiteCollection takes upto 15 minutes with addition of testsuites in future as we are trying to create a E2E testCollection,

For me the duration is not a problem at the moment, If you have a solution to run the testSuiteCollection multiple times without duplication, I can give a try

11

but looks like its dreaming towards a better solutions

12

I personally do not agree with you.

Of course you can express your requirement to Katalon through the official support channel. I think that this small topic in the user forum is not significant enough for the Katalon’s serious attention.

1 Like
13

Dear Mr Kazu :grinning_face:

We do care and pay attention, and every piece of feedback is invaluable to us. When it is shared through the official support channel or in the user forum, we truly appreciate it because it helps us improve each day.

So guys please keep them coming :partying_face:

1 Like