How can I force test failures when @SetUp fails?

Product Forums Katalon Studio
,
1

Hello,
The basic issue I am trying to solve is the test suite’s @SetUp step failed and as such none of the tests in the suite ran. This is a problem because the pipeline is passing showing 0 tests ran out of 0 possible tests. To fix this, I am trying to catch any errors occurring in the @SetUp step and throw an exception in the individual tests of that test suite, but I haven’t been able to do this yet.

I tried the following in the test suite:

import com.kms.katalon.core.util.KeywordUtil
import groovy.transform.Field
…

@Field setUpFailed = false

@SetUp(skipped = false)
def setUp() {
try {
// if the code in here fails, every test in the suite should fail
}
catch (Exception e) {
setUpFailed = true
}
}

…
@SetupTestCase(skipped = false)
def setupTestCase() {
if (setUpFailed) {
KeywordUtil.markFailed(“Fail the test case”)
}
}

However, this doesn’t appear to do what I was hoping. Any suggestions are much appreciated. Thank you.

2

Hi @james.milos,
We are considering to support marking a test suite execution as failure in case its @SetUp step fails. We will inform you should there be any updates. Please stay tuned.

2 Likes
3

Hi @james.milos, any updates here?

4

How about

  1. in the default Execution Profile, create a GlobalVariable TS1_SETUP_SUCCESSFUL of Boolean type with initial value true.

  2. create Test Suites/TS1 of which script looks:

import com.kms.katalon.core.annotation.SetUp

import internal.GlobalVariable

/**
 * Setup test suite environment.
 */
@SetUp(skipped = false) // Please change skipped to be false to activate this method.
def setUp() {
	GlobalVariable.TS1_SETUP_SUCCESSFUL = false
}
  1. create Test Cases/TC1 which looks:
import com.kms.katalon.core.util.KeywordUtil
import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI

import internal.GlobalVariable

if (GlobalVariable.TS1_SETUP_SUCCESSFUL) {
	WebUI.comment(">>>TS1 @SetUp was successful")
} else {
	KeywordUtil.markFailed(">>>TS1 @SetUp failed")	
}

  1. include TC1 into TS1

  2. execute TS1

When I ran this, I got the following output:

2021-05-06 00:00:17.489 INFO  c.k.katalon.core.main.TestSuiteExecutor  - START Test Suites/TS1
2021-05-06 00:00:17.551 INFO  c.k.katalon.core.main.TestSuiteExecutor  - hostName = kazuakiurayama - kazuakinoair.airport
2021-05-06 00:00:17.553 INFO  c.k.katalon.core.main.TestSuiteExecutor  - os = Mac OS X 64bit
2021-05-06 00:00:17.554 INFO  c.k.katalon.core.main.TestSuiteExecutor  - hostAddress = 192.168.0.9
2021-05-06 00:00:17.555 INFO  c.k.katalon.core.main.TestSuiteExecutor  - katalonVersion = 7.9.1.208
2021-05-06 00:00:19.032 INFO  c.k.katalon.core.main.TestCaseExecutor   - --------------------
2021-05-06 00:00:19.033 INFO  c.k.katalon.core.main.TestCaseExecutor   - START Test Cases/TC1
2021-05-06 00:00:19.239 DEBUG testcase.TC1                             - 1: if (TS1_SETUP_SUCCESSFUL)
2021-05-06 00:00:19.241 DEBUG testcase.TC1                             - 2: else
2021-05-06 00:00:19.257 DEBUG testcase.TC1                             - 1: markFailed(">>>TS1 @SetUp failed")
2021-05-06 00:00:19.330 ERROR com.kms.katalon.core.util.KeywordUtil    - ❌ >>>TS1 @SetUp failed
2021-05-06 00:00:19.341 ERROR c.k.katalon.core.main.TestCaseExecutor   - ❌ Test Cases/TC1 FAILED.
Reason:
com.kms.katalon.core.exception.StepFailedException: >>>TS1 @SetUp failed
	at com.kms.katalon.core.util.KeywordUtil.markFailed(KeywordUtil.java:19)
	at com.kms.katalon.core.util.KeywordUtil$markFailed.call(Unknown Source)
	at TC1.run(TC1:9)
        ...

This is what you want, isn’t it?

1 Like