Retry failed test cases only: issue with running multiple scenarios based on Test Data Iteration

Bug Report
,
1

Hello Support,

**OS **(Windows 8.1 64bit):

Katalon Studio Version (Katalon Studio Version: 5.3.0 Build: 1)

**## Katalon Studio logs: **

See attached

## Environment (for Web testing)

Browser not relevant (Chrome 63.0.3239.132 (Official Build) (64-bit))

**## Steps to reproduce -
**Having a Test Suite to execute, use the Retry (once in this case) feature and only for the failed test cases only.
The particularity of this test suite is that the test case is using a Test Data file that contains multiple URLs for the test case to create test scenarios from.
Please see the bellow setup as a printscreen:


## Expected Behavior
If one of the 20 URL scenario execution fails, I would expect that only that URL/test data record is retried.

**## Actual Behavior -
**The second Test case is being retried in execution, with all 20 different URLs/scenarios that the Test Data is providing as starting point.
I understand that this might be a not-implemented-yet behavior. But I am also having this expectation based on the fact that, during execution, the logger shows that 15 out of 21 test cases have passed. So, in the reporting, Katalon does not see only 2 test cases being executed but 1 + 20 different instances of the Test Data iteration.

Thank you and keep up the good work.
Cornel Neacsu

.log

Katalon Studio - Retry Failed Test Cases 2018-02-02 17.13.53.png

1 Like
2

Hello Support,

Any chance to get an answer on this question, please? How can we see if the request was seen or planned for handle?
Is it a desired behavior that is not functioning, is it a behavior that is worth discussing or is it something that is not the purpose for Katalon test suite execution?

Thank you,
Cornel Neacsu

1 Like
3

Hello…? Katalon?

A simple “good idea” / “bad idea” reply would suffice.
Or if this discussion is covered in another post, I am interested in reading it.

Thank you,
Cornel Neacsu

1 Like
4

Hi Support,

I am still interested in this feature or, at least, an answer about the status/viewpoint of Katalon team on this.

Thank you,
Cornel Neacsu

2 Likes
5

I would like to get an answer to this question as well

6

I too would like to know an answer on this.

1 Like
7

Hi support team
I would like to get an answer to this question as well.

thank team

1 Like
8

@devalex88, @duyluong or @Zarashima
Please help us with a feedback on this, it seems it has multiple interested users.
Even if it is not feasible to implement, an answer would be nice after so long time.

Thank you,
Cornel

9

Another useful idea for me would be to randomize test data on suite execution here:

image
image500Ă—200 8.58 KB

In this way, the execution will take very short time (because only one scenario is taken from the test data pool) but the coverage would be great because it takes a different(ish) scenario on each execution.
This would exponentially enhance the test data testing in Katalon.

I know it can be done programmatic also, but this would be very useful for non technical testers.

Thank you,
Cornel

10

Hi @Neacsu_Cornel

Just want to let you know that the development team is made aware of this issue and we’ll keep you updated.

Cheers !

1 Like
11

Hi @ThanhTo ! Any idea in what version will be implemented this feature? Cause we don’t have information about it since 5.3.0 Build:1 and nowadays katalon studio is in 6.3.0(beta).

Do you have any plans for the future?

Best regards!

12

@Neacsu_Cornel did you solve the problem?