Retry failed test case immediately

Hi! Is it possible to retry failed test case within test suite immediately after fail? I know there are -retry=1 -retryFailedTestCases=true parameters, but failed tests are retried after complete execution of current test suite, not immediately.

I’d like to achieve following:

I have tests 1 2 and 3.
If 2 fails, test exec order is 1 2 3 2. But I need 1 2 2 3. Is it possible?

Thank you!

Good point, this is useful workaround which can solve my issues. Thank you Vinh.

Hi Vinh,

yes I know it’s not usual behavior. But I’d like to prevent some random errors during test execution.

Example: I have add, modify and delete test. Modify and delete are dependent on add test (they share same object). Add test fails because of some non-functional error (server can be slow and some timeout in test expires) and then all other tests fail (because object was not created).

I don’t want to apply this retry model everywhere, but we have few critical tests, which are worth to be retried immediately. If there is no possibility to do it, I’d add verify/waitfor methods before each step, but it’s not ideal solution.

Unfortunately, it’s not provided in Katalon Studio.

However, there is an alternate way to do this by using Test Suite Collection. You know that Test Suite has the ability to re-run, right? How about adding your pre-condition test cases in a single Test Suite called 'TS_Pre-conditions' and add in all of your ‘add’ test cases into it. If any of them is failed, then it will be retried for only those ‘add’ test cases for that specific test suite

Other test cases will be added in other test suites and will be executed after that’TS_Pre-Conditions’ is executed successfully.

This is just another way to resolve your issues. You can organize it better based on your test scenarios. Hope it help you resolve this case

Hi there,

Failed test cases are re-run after test suite execution is ended. If it’s retried immediately after failed without waiting for test suite execution to end, it will likely break the usual testing flow.

I think re-run failed test case after test suite execution is ended is the same with your ideal. Why would you want to re-run like that?

1 Like

Interesting, Thanks for sharing your views…

Software Testing

Vinh Nguyen said:

Unfortunately, it’s not provided in Katalon Studio.

However, there is an alternate way to do this by using Test Suite Collection. You know that Test Suite has the ability to re-run, right? How about adding your pre-condition test cases in a single Test Suite called 'TS_Pre-conditions' and add in all of your ‘add’ test cases into it. If any of them is failed, then it will be retried for only those ‘add’ test cases for that specific test suite

Other test cases will be added in other test suites and will be executed after that’TS_Pre-Conditions’ is executed successfully.

http://prnt.sc/eq7t58

This is just another way to resolve your issues. You can organize it better based on your test scenarios. Hope it help you resolve this case

HI Vinh,

I also want this to done but the workaround you have given will not work for me.

My situation is that I have 30 test cases in a test suite, sometimes one or two fail ( not sure why as those are not failing always) ,
now after they fail, i want to rerun them immediately. I know katalon has ability to rerun the test cases but that is after completing the suite. now my test suit has login test case as well so it is like

1. login test case
2. do something
3. do something
4 do something
5 close browser

Now if let say test case 3 is failed , then i can be in 2 below scenarios

1. i click rerun for failed test cases , in this case login would not be called ( as it passes ) and so anyways test case 3 will not run because login is must for any further test case to run.

2. If I rerun all test cases, it will take lot of time because i am going to have 300 test cases in this suit , so rerunning the suite on failure of one test case does not make any sense .

Could you please suggest me any solution or workaround please ?

Regards!
Uma

1 Like

Testing creates a testing-failed.xml in the output folder after a run, which you can run after running the testing.xml. So you have the 10 testcases in testing.xml and run them and whatever fails will come in the testing-failed.xml which you can run then.

Where is this testing.xml file created?

While I Agree its against the normal of flow testing. It sometimes is a more viable option for more delicate flows.
Having the option in the Test Suite Collection’s Run Configuration would be a flexible way to implement Marek’s idea
Also in the run configuration, it would be a good idea to include.
Retry if Failed -> Use next data set.

Hi all,

Katalon Studio has improved the Retry Failed Executions Immediately feature and introduced the consolidated execution reports to address test flakiness in version 8.1.0. Please see the Release note for more details.

Thank you for your attention!

Nam Nguyen.