How to abort test execution after triggering through KRE manually

Product Forums Katalon Studio
1

Hi, I have few test suites with test cases to automate web flows.
Few of TCs in the suite are dependent on previous test cases in the same test suite. So if some test cases at the beginning FAIL, i dont want Kataloc process to trigger the subsequent test cases. The number of TCs to decide the to stop the failure is not deterministic due to which i cannot use the -maxFailedTests arguent to control it.

So i am manually killing the Katalonc process from my task manager, but even after killing that process i can see the subsequent test cases are still being launched. I understand it because each Test case launches a separate browser instance. And this will continue till the last test case in the test suite which is triggered.

How should i stop the execution even after Kataloc process is killed manually?
What other process should i kill so that the execution stops immediately?
Does Kataloc invoke some other deamon which keeps running in background so that even if the Katalon process is killed, that component keeps running.

2

Katalon’s Test Suite is a dum chain of invoking a sequence of Test Cases. Test Suite can not control if it should invoke the TC2 when the preceding TC1 failed. Test Suite will just invoke the TC2 regardless the TC1 has passed or failed. This is what we have.

3

You can introduce an inversion of control.

Each of your Test Cases should check if the preceding Test Cases passed or failed. Based on that information, It should determine if it runs or stops.

The TC2 should, at the very start of its processing body, check if the preceding TC1 passed or failed. The TC2 should determine if it should run or stop based on the result of the the TC1’s result.

How to inform the TC1’s result to the TC2? ---- Use GlobalVariable of type Map. For example, you have TC1, TC2 and TC3. Then the GlobalVariable Results could be initialized ad:

GlobalVariable.Results = ["TC1": false, "TC2": false, "TC3": false]

And TC1, TC2 and TC3 should turn the result into the “Results” GlobalVariable to true whe it passed. For example TC1 will do at the end of its processing body:

GlobalVariable.Results['TC1'] = true

Or, you would rather like to implement this line (recording the status of a Test Case result into GlobalVaraible) in a @AfterTestCase-annotated method of a Test Listener.

The TC2 will check the TC1’s result like this:

import com.kms.katalon.core.util.KeywordUtil

if ( ! GlobalVariable.Results['TC1']) {
    KeywordUtil.markFailedAndStop("TC1 failed so that TC2 quits immediately")
}

// continue its own business ...

TC2, TC3 and other test cases want to follow this approach.

Having TC1, TC2, TC3 in a Test Suite, all of 3 Test Cases will be called by the Test Suite anyway. When TC1 failed, TC2 and TC3 can, if appropriate, immediately quit before doing their own long-running business. Each constituant Test Cases decides its own life.

The control could be customized. For example, when TC1 failed, TC2 may quit soon but TC3 may continue. It is up to you how to design the flow. You just want to implement the decision as a part of each Test Cases.

This approach will apply to Katalon Runtime Enigine as well as Katalon Studio. You do not have to stop Test Cases manually any longer.

1 Like
4

I have developed a reusable lib for this issue:

5

Thanks al for the wonderful replies and ideas, but the issue is i cannot go ahead and modify all the test cases one by one to add the logic to check for the execution of previous test case and to check if the previous one has passed or failed, so that to decide whether to continue execution of the current test case or not. as suggested by @ kazurayam.

Lets say i am still building my test suite and i want to test once all the test suites and in turn the test cases through runtime, and if required i need to abort the execution if something goes wrong. Currently even if i kill the Katalonc.exe through Task manager, i see the execution still continues and the browsers per each test case keeps launching and i have to manually close them all, since i dont want the test data to be corrupted.
Since i am using KRE v7.9, i cant use the maxFailedTests =2 argument to the runtime, because i guess its not suported.

So the only way i am seeing currently is to manually close the other process which is responsible for trigerring the test cases even after the katalonc.exe is killed.
I hope the problem is clear now, can i expect some expertise please?

7

Why can’t you do it?

8

We cannot change it because I have a huge repository of Test cases and even the test cases are referenced in multiple test suites. issues are

  1. So the sequence of a TC1 will be different in Test Suite1 and will be different in test Suite
  2. Also existing test suites are huge and no time for modifying all those TCs 1 by one every time we change the test suite.
  3. Also every week the test suite has to be changed due to some addition of Test case or deletion because of changes in functionalities in underlying application which is being tested through automation.
9

I am just curious how many Test cases you have. 20? 50? or more?

10

There are more than 2000 Test cases in the Library and we pull in or out them in Test suites as and when required, its based on Regression testing scope, since every week the application deployment and to test the application after deployment through automation, we need this to be executed through command line.

11

Thank you for sharing your story. Your story far exceeds my experience.

I wonder how you managed to create 2000 test cases. How did you make it? Did you used Katalon Studio’s Web Recording tool to generate 2000 test cases?

I suppose you did not use the Script mode for manually writing that number of Test Case sources. You would never do so, I guess. Or did you?

12

@digvijak

This parameter was supported since KRE v8.1.0. Please update your KRE to v8.1.0 onward to use this parameter.

Screenshot 2023-05-23 at 22.52.45
Screenshot 2023-05-23 at 22.52.451726Ă—476 52.1 KB

13

Thanks for confirming.
I guess the arguments in the command line are case sensitive and they are place sensistive as well right?
Because when i tried as -maxfailedtests=2 and at the end of the command line it did not work as expected and when i changed it to be -maxFailedTests=2 and moved the argument just after the -reportFolder=“” it actually worked.
is it correct?

14

Also, i noticed once thing in this case, lets say i have 10 active Test cases in my test suite, i.e. all of them should be executed one after the other and if i trigger the execution through runtime and specify -maxFailedTests=2, it just terminated the execution immediately after first 2 TCs got failed, the failed tests count reached to 2 and in the html report generated also shows the Total tests as 2 and Failed count is also 2.
Now, What about the rest of the test cases in the test suite, they should either be reported as Blocked or Skipped, but they are not reported at all in the HTML file, as if i did not include them in the test suite
My manager thought that i could only think of 2 Test cases to test a functionality assigned to me :slight_smile:
What my manager and off course me are expecting is something below:
Total Tests =10
Failures =2
Passed = few of them (if not only first 2 tests got failed)
Skipped = # TCs count which actually did not get triggered after the -maxFailedTests reached to 2