At the test suite level, we have the script mode rite
Please open any test suite and click on Script mode, where you see setupTestCase() , tearDownTestCase() methods no ?
it runs before and after of every testcase
can we know the corresponding test case status here ?
I found a Test Suite has the following code in the script mode tab.
import ...
@SetUp(skipped = true) // Please change skipped to be false to activate this method.
def setUp() {
// Put your code here.
}
@TearDown(skipped = true) // Please change skipped to be false to activate this method.
def tearDown() {
// Put your code here.
}
@SetupTestCase(skipped = true) // Please change skipped to be false to activate this method.
def setupTestCase() {
// Put your code here.
}
@TearDownTestCase(skipped = true) // Please change skipped to be false to activate this method.
def tearDownTestCase() {
// Put your code here.
}
You wrote:
Then, do you want to rewrite the above script to be the following ? (please find TestCaseContext object as arguments):
import ...
@SetUp(skipped = true) // Please change skipped to be false to activate this method.
def setUp(TestSuiteContext testSuiteContext) {
// Put your code here.
}
@TearDown(skipped = true) // Please change skipped to be false to activate this method.
def tearDown(TestSuiteContext testSuiteContext) {
// Put your code here.
}
@SetupTestCase(skipped = true) // Please change skipped to be false to activate this method.
def setupTestCase(TestCaseContext testCaseContext) {
// Put your code here.
}
@TearDownTestCase(skipped = true) // Please change skipped to be false to activate this method.
def tearDownTestCase(TestCaseContext testCaseContext) {
// Put your code here.
}
But in a TestListener, you can implement what you want.
You wrote:
if a test case has failed i will need to do some test cleanup before the next test case gets triggered.
Then why not you do:
import com.kms.katalon.core.annotation.AfterTestCase
import com.kms.katalon.core.context.TestCaseContext
class MusaffirTestListener {
/**
* Executes after every test case ends.
* @param testCaseContext related information of the executed test case.
*/
@AfterTestCase
def sampleAfterTestCase(TestCaseContext testCaseContext) {
if (testCaseContext.getTestCaseStatus() == 'FAILED') {
// do whatever clean ups
}
}
}
in that case, the test listener will be applicable to all test suites rite.
for each test suite i will have specific needs. i will have specific things need to do when a testcase is failed.
So doing this at test listener looks not feasible to me
Do you think we can do something here ?
class MusaffirTestListener {
@AfterTestSuite
def afterTestSuite(TestSuiteContext testSuiteContext) {
switch (testSuiteContext.getTestSuiteName()) {
case 'Orange':
// do something specific for Orange
break
case 'Apple':
// do something specific for Apple
break
...
}
}
}
Test Listeners (Test Hooks)
Introducing Test Hooks feature in Katalon Studio version 5.2 to support users extend testing flows. Test Hooks are events binding that represent automation logic and will be executed once the condition is matched.
Prior to v5.2 there wasn’t TestListener. All we had was @SetUp/@TearDown-annotated method of each indivisula TestCases and TestSuites. Why Katalon Team added TestListener? Anything unsatisfactory with @SetUp/@TearDown-annotated methods? — Yes, there was. The API of @SetUp/@TearDown-annotated methods did not have the context arguments — TestCaseContext and TestSuiteContext. Therefore, I guess, Katalon Team decided to introduce a new concept of TestListener.
I personally always use TestListeners; I never use @SetUp/@TearDown-annotated method of each indivisula TestCases and TestSuites
Thank you so much @kazurayam for taking time to look at it and for writing these.
I get this point , how ever i still feel like some thing is lagging at the tool end on this perspective.
TestListener would be something I like not to touch often, i.e if we go with the above approach, when ever we have a new suite, the TestListener class would need to be modified to handle the clean up script which required for the respective suite, as TestListener is applicable to all the suites in the project.
If I could write an abstract method in TestListener and if I have the flexibility of completing that abstract method in my Test suite, I would have been more happier here. So it becomes responsibility of individual suite developer to handle such clean up scripts as well and do not have to touch the TestListener.
Your thoughts ?
Let me show you one of my “TestListener” as an example:
As you can see, this VTListener applies the Composittion pattern. It has an instance of other class VisualTestingListenerImpl which implements some lenghy business logic. An instance of the VTListener class delegates the VisualTestingListernerImpl object to do every meaningful stuff. The code of VTListener class is very short. Once I wrote the VTListener class, I have rarely touched it and will not in future.
It is completely up to you how to design a TestListener implementation. It is just a plain old Groovy class with some annotations (@BeforeTestSuite, @AfterTestSuite, @BeforeTestCase, @AfterTestSuite). You have 100% of freedom how to design it.
For example, if I want to, I can move my com.kazurayam.visualtesting.VisualTestingListenerImpl class out of Katalon Studio project, and host it in a seperated Eclipse/Gradle project. There I would further develop the code, create a jar file, and bring the jar file into the Drivers folder of multiple Katalon Studo projects. I mean, I can share a single implementation of TestListener in multiple Katalon Studio projects. Flexible enough, isn’t it?
Finally I understand what you want. Provided that your Katalon Studio project is shared by your team of developers, you want to make development of clean up scripts to be the responsibility of individual suite developers, am I right?
I think it is an unique idea. I have never thought of it.
With this interest in mind, the centralized approach of TestListener would not fit to your idea. I suppose that Katalon team does not share your idea. You should raise a feature request and wait for their response.
I am however upvoting for @Katalon_team to contain the possibility to access current test case instance in the TestSuite dedicated scripting area also.
I believe this is the main reason why you should use dedicated test suite scripting for such events, to access the specific suite and test cases context.
The other test listeners are for a generic project approach.
Of course I may be biased and seeing only a small portion of the entire picture here.
