Use case:
Tests are run manually via command line using KRE. Tests are in a test collection which comprises of around 100s of test suites which each test suite having around 200-300 data variations in data files. The tests would run more than 24 hours and sometimes there is a need to abort the remaining of the tests.
How do we cleanly stop the KRE tests? Observations by killing it via Ctrl-c would result to locked up Reports directory and sometimes tests are still running (by looking at resource monitor).
If I were you, I would stop using Test Suite Collection. Test Suite Collection provides almost no feature to manage the sub-processes (100 Test Suites) forked from the parent process (Test Suite Collection).
I would use Jenkins Pipeline to execute processes of 100 Test Suites in a managed way.
With Jenkins Pipeline you can program how you want to run and manage 100 sub-process (KRE with Test Suites).
Thank you for the information. Unfortunately, we donât have the luxury to use Jenkins. We just need a good way of aborting tests. We have a work around to use task manager and resource monitor to kill âghostâ processes and âhangingâ handles.
I thought that you are not satisfied with Task Manager and Resource Monitor.
Let me add a word, Jenkins is not luxurious at all, I think. Jenkins is an open source project, free to use. You can run a Jenkins on your Windows PC locally. It just requires intensive learning; but will give you full control over subprocesses.
Thank you. Luxury (no time and resource to add that extra infrastructure). Iâm not contented with the workaround. That is why asking here if they addressed this already or someone complained already.
The test suite collection is a âworkaroundâ to handle bigger data file. This will allow us to get reports right away (after a test suite run) and not lose reports if ever KS decide to stop.
I think it would make sense to move this topic to the âFeature Requestâ category.
The OP requires
KRE should provide a feature to gracefully stop an executing Test Suite Collection which comprises many Test Suites (e.g, 100). This would possibly be implented some command line like
$ katalonc stop -testSuiteCollectionPath=<path>
even stopped, KRE should compile a report of the Test Suite Collection; KRE should not lose the report when the TSC is stopped.
KRE v8.1.0 and onward provides a feature call Terminate Execution Conditionally. KRE will automatically stop if the total failed tests reach the given limit number. You can give a try with it.
@duyluong Thank you! That is a feature that Iâm not aware and that seems to be really useful. However, there will be cases that we need to stop the current execution. I donât know if there is a signaling method that you can introduce to safely terminate the execution of the tests.
Pressing Ctrl-C will send the SIGINT (interrupt from keyboard) signal to the process.
You can try to use CTRL - \ instead, which should send SIGQUIT (I donât know exactly how it may behave)
Also, you can try to issue from another terminal a standard kill (that will send a SIGTERM which is similar to SIGINT somehow) vs kill -9 (SIGKILL).
I donât know if will help you, in theory for SIGINT and SIGTERM the shutdown hooks should be executed by JVM, SIGKILL should do a âforce stopâ, but based on the behavior it may give some clues to the katalon devs what and how to implement/improve (catch the right signal and execute a graceful shutdown from the java process)
