Stopping executing test collection in KRE

#1

Use case:
Tests are run manually via command line using KRE. Tests are in a test collection which comprises of around 100s of test suites which each test suite having around 200-300 data variations in data files. The tests would run more than 24 hours and sometimes there is a need to abort the remaining of the tests.

How do we cleanly stop the KRE tests? Observations by killing it via Ctrl-c would result to locked up Reports directory and sometimes tests are still running (by looking at resource monitor).

Thanks!

A Test Suite Collection forks multiple OS sub-processes
Test Suite Collection
#2

If I were you, I would stop using Test Suite Collection. Test Suite Collection provides almost no feature to manage the sub-processes (100 Test Suites) forked from the parent process (Test Suite Collection).

I would use Jenkins Pipeline to execute processes of 100 Test Suites in a managed way.

image
image1322×499

With Jenkins Pipeline you can program how you want to run and manage 100 sub-process (KRE with Test Suites).

#3

Thank you for the information. Unfortunately, we don’t have the luxury to use Jenkins. We just need a good way of aborting tests. We have a work around to use task manager and resource monitor to kill “ghost” processes and “hanging” handles.

#4

I thought that you are not satisfied with Task Manager and Resource Monitor.

Let me add a word, Jenkins is not luxurious at all, I think. Jenkins is an open source project, free to use. You can run a Jenkins on your Windows PC locally. It just requires intensive learning; but will give you full control over subprocesses.

#5

Thank you. Luxury (no time and resource to add that extra infrastructure). I’m not contented with the workaround. That is why asking here if they addressed this already or someone complained already.

#6

As far as I (3 years in this forum) am aware, no they didn’t, I was wrong. See the following post by @duyluong.

As far as I am aware, nobody.

I suppose that you are an exceptional user who has this big Test Suite Collection.

#7

The test suite collection is a “workaround” to handle bigger data file. This will allow us to get reports right away (after a test suite run) and not lose reports if ever KS decide to stop.

#8

@Russ_Thomas

I think it would make sense to move this topic to the “Feature Request” category.

The OP requires

  • KRE should provide a feature to gracefully stop an executing Test Suite Collection which comprises many Test Suites (e.g, 100). This would possibly be implented some command line like
$ katalonc stop -testSuiteCollectionPath=<path>
  • even stopped, KRE should compile a report of the Test Suite Collection; KRE should not lose the report when the TSC is stopped.
1 Like
#9

I know nothing about TestOps. So I would like to ask others.

Does anybody know if Katalon’s TestOps provides favourable solution to @david.casiano?

Does TestOps provide a feature to gracefully stop an executing Test Suite Collection?

Does TestOps compile a report of the stopped Test Suite Collection?

#10

We will not be able to use test ops unless it supports air gapped (not connected to the world) environment.

#11

OK. I understand it.

#12

Hi @david.casiano,

KRE v8.1.0 and onward provides a feature call Terminate Execution Conditionally. KRE will automatically stop if the total failed tests reach the given limit number. You can give a try with it.

#13

@duyluong Thank you! That is a feature that I’m not aware and that seems to be really useful. However, there will be cases that we need to stop the current execution. I don’t know if there is a signaling method that you can introduce to safely terminate the execution of the tests.

#14

@duyluong Question on this feature, is the test failure count for the whole test collection or is it per test suite in the collection?

#15

Pressing Ctrl-C will send the SIGINT (interrupt from keyboard) signal to the process.
You can try to use CTRL - \ instead, which should send SIGQUIT (I don’t know exactly how it may behave)

Also, you can try to issue from another terminal a standard kill (that will send a SIGTERM which is similar to SIGINT somehow) vs kill -9 (SIGKILL).

I don’t know if will help you, in theory for SIGINT and SIGTERM the shutdown hooks should be executed by JVM, SIGKILL should do a ‘force stop’, but based on the behavior it may give some clues to the katalon devs what and how to implement/improve (catch the right signal and execute a graceful shutdown from the java process)

ref1: https://man7.org/linux/man-pages/man7/signal.7.html
ref2: https://docs.oracle.com/javase/9/troubleshoot/handle-signals-and-exceptions.htm#JSTGD356
ref3: https://docs.oracle.com/javase/6/docs/api/java/lang/Runtime.html#addShutdownHook(java.lang.Thread)

hints: https://stackoverflow.com/questions/2541597/how-to-gracefully-handle-the-sigkill-signal-in-java

#16

Just FYI

2 Likes