Hi,
Is there a way that I can create a test suite and add test cases to it through command?
What i want is that whatever test cases are failed in my CI-CD pipeline, I want to re-execute them (only failed tests). I want to create a test suite, add those failed TCs in it and re-execute the stage. All in CI-CD so Im looking for such command.
Regards
Soumya
I am sure there is no such build-in automatic functionality in Katalon Studio.
In a definition of a Test Suite, each Test Case is marked with attribute “Run” with value On/Off. You can manually select which test case to run.
In a Katalon project there are files <projectDir>/Test Suites/folder name/<test case name>.ts which has such code snipet:
<testCaseLink>¶
<guid>cbec7f36-1c4f-4ede-ae10-35dbfc07d946</guid>¶
<isReuseDriver>false</isReuseDriver>¶
<isRun>true</isRun>¶
<testCaseId>Test Cases/CURA/restorePreviousScreenshots</testCaseId>¶
</testCaseLink>¶
I believe that the <isRun> here denotes On/Off value you check for the Run attribute in the GUI.
If you want to, you can write and run a script that modifies the <isRun> value in the *.ts file,
How do you determin which test case to turn On/Off for <isRun>?
In the <projectDir>/Reports/yyyyMMdd_hhmmss/folders/<test suite name>/yyyyMMdd_hhmmss/JUnit_Report.xml. In the file you can find such code fragments:
<testsuites name="chronos_capture" time="51" tests="2" failures="1" errors="0">¶
<testsuite name="chronos_capture" tests="2" failures="1" errors="0" time="51" timestamp="2019-10-10 22:43:08" id="Test Suites/CURA/chronos_capture">¶
....
<testcase name="Test Cases/CURA/restorePreviousScreenshots" status="FAILED">¶
If you develop a script to read the JUnit_Report.xml file, then you can find which test cases of which test case failed so that you want to run again later.
Good answer, Kaz.
This requirement has been mentioned before - build a test suite programmatically (with and easy way to take fails form one suite and pass them to a retest-suite).
‘Love’ this solution.
With the amendment … this will only confuse the main scope of automated tests, relevancy.
Unless fuzz testing is desired.
I agree with @bionel.
I would not mind repeating all test cases, regardless passed or failed in the previous run, every time I run a test suite. Especially in a CI/CD process, I would not mind the burden of redoing already-passed tests because my machine takes it and I can be away from the redo.