Stopping executing test collection in KRE

Use case:
Tests are run manually via command line using KRE. Tests are in a test collection which comprises of around 100s of test suites which each test suite having around 200-300 data variations in data files. The tests would run more than 24 hours and sometimes there is a need to abort the remaining of the tests.

How do we cleanly stop the KRE tests? Observations by killing it via Ctrl-c would result to locked up Reports directory and sometimes tests are still running (by looking at resource monitor).

Thanks!

If I were you, I would stop using Test Suite Collection. Test Suite Collection provides almost no feature to manage the sub-processes (100 Test Suites) forked from the parent process (Test Suite Collection).

I would use Jenkins Pipeline to execute processes of 100 Test Suites in a managed way.
image
With Jenkins Pipeline you can program how you want to run and manage 100 sub-process (KRE with Test Suites).

Thank you for the information. Unfortunately, we donā€™t have the luxury to use Jenkins. We just need a good way of aborting tests. We have a work around to use task manager and resource monitor to kill ā€œghostā€ processes and ā€œhangingā€ handles.

I thought that you are not satisfied with Task Manager and Resource Monitor.

Let me add a word, Jenkins is not luxurious at all, I think. Jenkins is an open source project, free to use. You can run a Jenkins on your Windows PC locally. It just requires intensive learning; but will give you full control over subprocesses.

Thank you. Luxury (no time and resource to add that extra infrastructure). Iā€™m not contented with the workaround. That is why asking here if they addressed this already or someone complained already.

As far as I (3 years in this forum) am aware, no they didnā€™t, I was wrong. See the following post by @duyluong.

As far as I am aware, nobody.

I suppose that you are an exceptional user who has this big Test Suite Collection.

The test suite collection is a ā€œworkaroundā€ to handle bigger data file. This will allow us to get reports right away (after a test suite run) and not lose reports if ever KS decide to stop.

@Russ_Thomas

I think it would make sense to move this topic to the ā€œFeature Requestā€ category.

The OP requires

  • KRE should provide a feature to gracefully stop an executing Test Suite Collection which comprises many Test Suites (e.g, 100). This would possibly be implented some command line like
$ katalonc stop -testSuiteCollectionPath=<path>
  • even stopped, KRE should compile a report of the Test Suite Collection; KRE should not lose the report when the TSC is stopped.
1 Like

I know nothing about TestOps. So I would like to ask others.

Does anybody know if Katalonā€™s TestOps provides favourable solution to @david.casiano?

Does TestOps provide a feature to gracefully stop an executing Test Suite Collection?

Does TestOps compile a report of the stopped Test Suite Collection?

We will not be able to use test ops unless it supports air gapped (not connected to the world) environment.

OK. I understand it.

Hi @david.casiano,

KRE v8.1.0 and onward provides a feature call Terminate Execution Conditionally. KRE will automatically stop if the total failed tests reach the given limit number. You can give a try with it.

@duyluong Thank you! That is a feature that Iā€™m not aware and that seems to be really useful. However, there will be cases that we need to stop the current execution. I donā€™t know if there is a signaling method that you can introduce to safely terminate the execution of the tests.

@duyluong Question on this feature, is the test failure count for the whole test collection or is it per test suite in the collection?

Pressing Ctrl-C will send the SIGINT (interrupt from keyboard) signal to the process.
You can try to use CTRL - \ instead, which should send SIGQUIT (I donā€™t know exactly how it may behave)

Also, you can try to issue from another terminal a standard kill (that will send a SIGTERM which is similar to SIGINT somehow) vs kill -9 (SIGKILL).

I donā€™t know if will help you, in theory for SIGINT and SIGTERM the shutdown hooks should be executed by JVM, SIGKILL should do a ā€˜force stopā€™, but based on the behavior it may give some clues to the katalon devs what and how to implement/improve (catch the right signal and execute a graceful shutdown from the java process)

ref1: signal(7) - Linux manual page
ref2: Handle Signals and Exceptions
ref3: Runtime (Java Platform SE 6)

hints: How to gracefully handle the SIGKILL signal in Java - Stack Overflow

Just FYI

2 Likes