# How to continue Test Suites after fail in Jenkins?

**URL:** <https://forum.katalon.com/t/how-to-continue-test-suites-after-fail-in-jenkins/8739>\
**Category:** Integrations\
**Tags:** jenkins, integrations, katalon-studio\
**Created:** [January 5, 2018, 11:11am UTC](https://forum.katalon.com/t/how-to-continue-test-suites-after-fail-in-jenkins/8739 "2018-01-05T11:11:01Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![Mariusz](https://dub1.discourse-cdn.com/katalon/user_avatar/forum.katalon.com/mariusz/32/386_2.png) [@Mariusz](https://forum.katalon.com/u/Mariusz)\
**Post date:** [January 5, 2018, 11:11am UTC](https://forum.katalon.com/t/how-to-continue-test-suites-after-fail-in-jenkins/8739/1 "2018-01-05T11:11:01Z")

</div>

Hello,  
Let’s say that I have a test (Test1). And in this test I want to verify if something isn’t present.  
If it’s not visible, throw new exception and mark test as failed.  
If it’s visible - do nothing.

```auto
'Check if object isn\'t present.'
if (WebUI.verifyElementNotPresent(findTestObject('Table/NameOfObject', [('name') : 'AutoTestLB']), 0)) {
    'Throw an error if object isn\'t visible.'
    throw new com.kms.katalon.core.exception.StepErrorException('Object isn\'t visible')
}

```

So, in this case, if something won’t be visible, test suite will be marked as FAILED.

Now, in Jenkins, if I have more Test Suites, other tests after that above aren’t anymore executed.

```auto
cd /home/dev/Documents/Katalon/
./katalon -runMode=console -projectPath="Project.prj" -reportFolder="Reports" -reportFileName="report1" -retry=0 -testSuitePath="Test Suites/Test1" -browserType="Chrome"
./katalon -runMode=console -projectPath="Project.prj" -reportFolder="Reports" -reportFileName="report2" -retry=0 -testSuitePath="Test Suites/Test2" -browserType="Chrome"
./katalon -runMode=console -projectPath="Project.prj" -reportFolder="Reports" -reportFileName="report3" -retry=0 -testSuitePath="Test Suites/Test3" -browserType="Chrome"

```

What I want to do is to “Start test, if TestSuite is failed, mark that and continue tests in another TestSuite”.

What should be changed?

---

<div class="post-metadata">

**Author:** ![Trong\_Bui](https://dub1.discourse-cdn.com/katalon/user_avatar/forum.katalon.com/trong_bui/32/374_2.png) [@Trong\_Bui](https://forum.katalon.com/u/Trong_Bui)\
**Post date:** [March 15, 2018, 7:37am UTC](https://forum.katalon.com/t/how-to-continue-test-suites-after-fail-in-jenkins/8739/2 "2018-03-15T07:37:21Z")

</div>

This topic is moved to New Feature/Suggestion box for further discussion.

---

<div class="post-metadata">

**Author:** ![Zarashima](https://dub1.discourse-cdn.com/katalon/user_avatar/forum.katalon.com/zarashima/32/928_2.png) [@Zarashima](https://forum.katalon.com/u/Zarashima)\
**Post date:** [March 15, 2018, 9:55am UTC](https://forum.katalon.com/t/how-to-continue-test-suites-after-fail-in-jenkins/8739/3 "2018-03-15T09:55:40Z")

</div>

> [Mariusz](https://forum.katalon.com/profile/2862/Mariusz) said:
> 
> Hello,  
> Let’s say that I have a test (Test1). And in this test I want to verify if something isn’t present.  
> If it’s not visible, throw new exception and mark test as failed.  
> If it’s visible - do nothing.
> 
> ```auto
> 'Check if object isn\'t present.'
> 
> ```

if (WebUI.verifyElementNotPresent(findTestObject(‘Table/NameOfObject’, [(‘name’) : ‘AutoTestLB’]), 0)) {  
‘Throw an error if object isn't visible.’  
throw new com.kms.katalon.core.exception.StepErrorException(‘Object isn't visible’)  
}

> ```auto
> 
> So, in this case, if something won't be visible, test suite will be marked as FAILED.  
>   
> Now, in Jenkins, if I have more Test Suites, other tests after that above aren't anymore executed.
> 
> ```
> 
> cd /home/dev/Documents/Katalon/  
> ./katalon -runMode=console -projectPath=“Project.prj” -reportFolder=“Reports” -reportFileName=“report1” -retry=0 -testSuitePath=“Test Suites/Test1” -browserType=“Chrome”  
> ./katalon -runMode=console -projectPath=“Project.prj” -reportFolder=“Reports” -reportFileName=“report2” -retry=0 -testSuitePath=“Test Suites/Test2” -browserType=“Chrome”  
> ./katalon -runMode=console -projectPath=“Project.prj” -reportFolder=“Reports” -reportFileName=“report3” -retry=0 -testSuitePath=“Test Suites/Test3” -browserType=“Chrome”

> ```auto
> 
>   
> What I want to do is to "Start test, if TestSuite is failed, mark that and continue tests in another TestSuite".  
>   
> What should be changed?  
> 
>   
> 
> ```

A Test Suite should be run completely (all test cases within it). I believe this is a normal behavior and it should not be changed accordingly if one of the executed test cases are failed.

---

<div class="post-metadata">

**Author:** ![Mariusz](https://dub1.discourse-cdn.com/katalon/user_avatar/forum.katalon.com/mariusz/32/386_2.png) [@Mariusz](https://forum.katalon.com/u/Mariusz)\
**Post date:** [March 21, 2018, 7:55am UTC](https://forum.katalon.com/t/how-to-continue-test-suites-after-fail-in-jenkins/8739/4 "2018-03-21T07:55:34Z")

</div>

I am not entirely convinced of this.  
So let us assume one more thing. Three successful tests should give us 100% efficacy (~33% for each test). If one test is not performed, we should get a score of ~66% (assuming that the other tests are successful), and in this case we really have 0%, because Jenkins (or Katalon?) did not continue further testing.  
It seems that some extra option would be useful to continue all the TCs, independently, until the end. However, I am not sure on which side this should be implemented (Katalon or some Jenkins modification).  
Or maybe instead of doing it this way, I should switch to Test Suite Collection?

---

<div class="post-metadata">

**Author:** ![Mariusz](https://dub1.discourse-cdn.com/katalon/user_avatar/forum.katalon.com/mariusz/32/386_2.png) [@Mariusz](https://forum.katalon.com/u/Mariusz)\
**Post date:** [April 13, 2018, 8:22am UTC](https://forum.katalon.com/t/how-to-continue-test-suites-after-fail-in-jenkins/8739/5 "2018-04-13T08:22:10Z")

</div>

Finally, I found a solution.

```auto
$KATALON -reportFileName="API" -retry=0 -testSuiteCollectionPath="Test Suites/API/API" -browserType="Chrome" || error=true
$KATALON -reportFileName="Dashboard" -retry=0 -testSuitePath="Test Suites/Dashboard" -browserType="Chrome" || error=true

Fail the build if there was an error
if [$error]
then 
    exit -1
fi

```

In this case, if one test fail, it will be marked, but all other test will be continued.  
In last step, we will check if there were any errors. If yes, Jenkins will mark whole job as FAILED (or not).

---

<div class="post-metadata">

**Author:** ![agarcia](https://dub1.discourse-cdn.com/katalon/user_avatar/forum.katalon.com/agarcia/32/16104_2.png) [@agarcia](https://forum.katalon.com/u/agarcia)\
**Post date:** [June 24, 2019, 1:23pm UTC](https://forum.katalon.com/t/how-to-continue-test-suites-after-fail-in-jenkins/8739/6 "2019-06-24T13:23:55Z")

</div>

Hi, what about this? I have a Test Suite with 5 test cases. Let’s say Test Case 1 fails, how can I have it proceed to Test Case 2 and so on?

This is what I am getting when a test case fails in a Test Suite, it was like it keeps on retrying the failed step on that test case even the retry is equal to 0:

 ![image](https://europe1.discourse-cdn.com/katalon/original/2X/5/5e3bd672f3b0351aa8a54571d197e2bb90831ff8.png)

---

<div class="post-metadata">

**Author:** ![Mariusz](https://dub1.discourse-cdn.com/katalon/user_avatar/forum.katalon.com/mariusz/32/386_2.png) [@Mariusz](https://forum.katalon.com/u/Mariusz)\
**Post date:** [June 25, 2019, 6:21am UTC](https://forum.katalon.com/t/how-to-continue-test-suites-after-fail-in-jenkins/8739/7 "2019-06-25T06:21:25Z")

</div>

Could you show the structure of your TS and TCs?

---

<div class="post-metadata">

**Author:** ![agarcia](https://dub1.discourse-cdn.com/katalon/user_avatar/forum.katalon.com/agarcia/32/16104_2.png) [@agarcia](https://forum.katalon.com/u/agarcia)\
**Post date:** [June 25, 2019, 6:48am UTC](https://forum.katalon.com/t/how-to-continue-test-suites-after-fail-in-jenkins/8739/8 "2019-06-25T06:48:28Z")

</div>

Hi @Mariusz,

Here is my sample Test Suite:

 ![image](https://europe1.discourse-cdn.com/katalon/original/2X/8/883166d5e147a9b7e6c5b77ccd63e8dd860ef405.png)

Here are the test cases:

 ![image](https://europe1.discourse-cdn.com/katalon/original/2X/a/abf0840775e4aa9958454f961673d4e648f5acb7.png) ![image](https://europe1.discourse-cdn.com/katalon/original/2X/1/12d63400abb825d8749e2bf1b7dbd7ff35c73f4f.png)

Also, I set the default failure handling of the project to **CONTINUE\_ON\_FAILURE**.

Here’s the job:

 ![image](https://europe1.discourse-cdn.com/katalon/original/2X/8/84307475d8198dcbead06ca9589c2c1d88dd8a77.png)

---

<div class="post-metadata">

**Author:** ![Mariusz](https://dub1.discourse-cdn.com/katalon/user_avatar/forum.katalon.com/mariusz/32/386_2.png) [@Mariusz](https://forum.katalon.com/u/Mariusz)\
**Post date:** [June 25, 2019, 9:34am UTC](https://forum.katalon.com/t/how-to-continue-test-suites-after-fail-in-jenkins/8739/9 "2019-06-25T09:34:59Z")

</div>

Add missing conditions from my Jenkins code above and check.

---

<div class="post-metadata">

**Author:** ![agarcia](https://dub1.discourse-cdn.com/katalon/user_avatar/forum.katalon.com/agarcia/32/16104_2.png) [@agarcia](https://forum.katalon.com/u/agarcia)\
**Post date:** [June 25, 2019, 10:24am UTC](https://forum.katalon.com/t/how-to-continue-test-suites-after-fail-in-jenkins/8739/10 "2019-06-25T10:24:13Z")

</div>

Hi @Mariusz,

Sorry but is this the condition that you want me to add to my build?

> $KATALON -reportFileName=“API” -retry=0 -testSuiteCollectionPath=“Test Suites/API/API” -browserType=“Chrome” || error=true  
> $KATALON -reportFileName=“Dashboard” -retry=0 -testSuitePath=“Test Suites/Dashboard” -browserType=“Chrome” || error=true
> 
> Fail the build if there was an error  
> if [$error]  
> then  
> exit -1  
> fi

May I ask for the exact condition that I should copy? Please bear with me as I’m just new to using Katalon and Jenkins 😞

---

<div class="post-metadata">

**Author:** ![Mariusz](https://dub1.discourse-cdn.com/katalon/user_avatar/forum.katalon.com/mariusz/32/386_2.png) [@Mariusz](https://forum.katalon.com/u/Mariusz)\
**Post date:** [June 25, 2019, 10:42am UTC](https://forum.katalon.com/t/how-to-continue-test-suites-after-fail-in-jenkins/8739/11 "2019-06-25T10:42:54Z")

</div>

At the end of the command you need to set `|| error=true`  
And then

> ```
> if [$error]
> then
> exit -1
> fi
> 
> ```

---

<div class="post-metadata">

**Author:** ![agarcia](https://dub1.discourse-cdn.com/katalon/user_avatar/forum.katalon.com/agarcia/32/16104_2.png) [@agarcia](https://forum.katalon.com/u/agarcia)\
**Post date:** [June 25, 2019, 11:03am UTC](https://forum.katalon.com/t/how-to-continue-test-suites-after-fail-in-jenkins/8739/12 "2019-06-25T11:03:40Z")

</div>

I added it:

> -projectPath=“C:\Users\Admin\git\pimmdesktoptestscripts\PIMM Desktop CCM.prj” -retry=0 -testSuitePath=“Test Suites/Test\_TS” -executionProfile=“default” -browserType=“Chrome (headless)” -apiKey=5beb57ba-6f7a-4897-bae6-970f4313cc14 --config -webui.autoUpdateDrivers=true || error=true
> 
> if [$error]  
> then  
> exit -1  
> fi

But still getting the same issue:

 ![image](https://europe1.discourse-cdn.com/katalon/original/2X/c/c3e7c83545461f95ffd578a9a0689f85f3c28cc0.png)