Consider a case where user has 2 test case inside a testsuite and first test case is failing and we put the retry count to 2. So first test case rerun 2 times after failing and system will create a new report for each new rerun instance plus the first report. In case i want to present a consolidated report i can’t present as i don’t have a consolidated report where i can tell retry count for this test case is 2 and test case failed even after rerun. We should have some capability to create such consolidated report
rahul khurana said:
Consider a case where user has 2 test case inside a testsuite and first test case is failing and we put the retry count to 2. So first test case rerun 2 times after failing and system will create a new report for each new rerun instance plus the first report. In case i want to present a consolidated report i can’t present as i don’t have a consolidated report where i can tell retry count for this test case is 2 and test case failed even after rerun. We should have some capability to create such consolidated report
Hi Rahul,
Thanks for the suggestion. I will share it with the team. However, you can leverage Katalon Analytics for advanced test execution reports including charts, graphs, execution history, and detect flaky tests.
Give it a try! Let us know if you have any further concerns or questions,
Thanks again for using Katalon Studio,
Liam
Hi @716-Liam The current behavior where it creates multiple report folders is not good and will be difficult to manage in the pipeline runs. As Rahul suggested we need to have a consolidated report instead of multiple report folders for each retry.
Hi @716-Liam The current behavior where it creates multiple report folders is not good and will be difficult to manage in the pipeline runs. As Rahul suggested we need to have a consolidated report instead of multiple report folders for each retry.
It is equally complicated to do an analysis of results when obtaining several reports of execution of a suite, as seen in the image