Failed to get Report folder for every suite in collection

We ran a Test Suite Collection that contains nine (9) suites, but only got three report folders with the following names:

20181226_132644
20181226_132645
20181226_132646

I am suspecting that, since the folder names are unique only to the second (not millisecond), that the report folders are being overwritten by the next one until the second changes. Just a guess.

2 Likes

I am suspecting that, since the folder names are unique only to the second (not millisecond), that the report folders are being overwritten by the next one until the second changes. Just a guess.

Can you confirm this based on the information in each report (i.e. do you expect each report to contain results for a different subset of Test Cases)?

We expect each report folder to contain the HTML report (and JUnit XML, images, etc.) for one suite. Since the collection has 9 suites, we expect 9 report folders. Instead we get three report folders whose folder names differ by exactly one second: 44, 45, and 46. There are no reports for 6 of the nine suites. Each of the three folders contains a report for one suite, say, the third, sixth, and ninth suite.

This sample from the build log shows that multiple suites are sharing the same output folders.

1 Like

Thank you for the idea. We’ll check and if it’s a bug it’s gonna be fixed soon.

1 Like

Thank you. This is not super critical to us now, as we have separate jobs that run each suite. However, over time, after our Katalon project becomes more mature, we will rely mainly on the collection of suites to perform our regression tests. And we’d like to be able to see all of the reports for all the suites.

Hi,
We have exactly the same issue by using a test suite collection.

We have by pass the issue by sync the content of the “Reports” folder but the “reportFolder” option is not working with test suite collection and many test inside it.

Thank in advance to solve this issue.

1 Like

Thank you for fixing this. We have upgraded.

1 Like

We are still seeing the bug, multiple suite reports written to same folder for the collection.

Thank you for letting us know. The version in the report was 6.0.6, so I just want to confirm if you were using 6.0.6 or 6.1.0 beta.

See my attached imaging showing 6.0.6.4.

Could you please try again with 6.1.0. Version 6.0.6 does not contain this fix.

Yes, this appears to be fixed in 6.1.0. We get five report folders for a collection that has five suites.

image

1 Like

Would be nice if there was a roll-up index.html report file at the same level that had links to the reports showing the suite names. As it stands, the tester has to open each folder and open each report file to find a failure. JUnit reports the failures, but there is no easy way to get to the specific report.

We are hand-jamming a powershell script to make the roll-up index.html.

1 Like

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.