In my project, I am calling multiple test cases (say 10 test cases) as a flow and executing it. Is it best practice or shall I create custom keywords of 10 and use them as a flow for execution?
I wanted to know which approach is the best one?
A composite of Test Cases tends to be (far) slower than a Test Case that calles Keywords due to the verbosity of step-execution logs emitted by Katalon Studio.
Please have a look at my previous report about this point.
This report shows a comparison of 5 min 37 secs with verbose logging versus 0 min 25 secs without logging, both carry out just the same processing. I believe that this difference is significant.
On the other hand, you should note that:
- Katalon Studio emits verbose logs of executing steps in Test Case scripts.
- Katalon Studio emits no step execution log for Keyword classes.
Therefore, âComposite Test Casesâ tends to be slower than âa Test Case that calls Keywordsâ if you do not make effort to minimize the logging verbosity.
Another aspect.
It is easier to develop & test Test Case scripts in Katalon Studio. You can run a Test Case by single button click, Katalon Studio emits verbose execution log, you can interactively see how your Test Case ran.
On the other hand, it is difficult to develop quaility Groovy classes as custom Keywords in Katalon Studio. Katalon Studio does not offer any built-in support of unit-testing of custom Keywords.
I once developed junit4ks to perform unit-test over my custom classes.
Often I develop custom classes to use in a Katalon project OUTSIDE Katalon Studio. I use IntelliJ + Gradle where I carry out enough unit-testing using JUnit4. Once developed, I will create a jar, import the jar into the Katalon project, use the classes in the jar from Test Cases.
A practical suggestion from me for easier path could be:
- use a composite of multiple Test Cases, rather than Keyword classes
- try to minimize the verbosity of step-execution logs emitted by Katalon Studio. See the above-mentioned post how to.
Thanks @kazurayam But here I see a problem when I use Custom Keywords. The problem is, if I wanted to know where all this keyword is used, there is no way. Can you suggest any way to do this.
My use case is: If I want to update any parameterize values for a particular key word, how can I verify where this keyword is used and make changes in the test cases.
You are right. I think there is no way to do that.
Katalon Studio provides very little support for ârefactoring Test Casesâ.
In this aspect, you would find that the 2 patterns (âComposite Test Casesâ and âa Test Case calling Keywordsâ) are no different. In both cases, all you can do is to repeat running all Test Case + find errors + correct the code + retry.
We have this Advance Search which Katalon provides, I think we can search for the keyword which is used in the test cases and correct or update it.
What do you think about this?
But we need to run the test case to ensure it took the updated or corrected value.
I didnât know it.
This is a fairly new feature since v8.2.5 (jan 2022). I havenât used it. I have no opinion about it.
If you find it good enough, then fine. You wouldnât need my opinion.
The Advanced Search seems to be a sort of âgrepâ over multiple test case scripts in a project. If so, it will apply similarly to the 2 approaches (âComposite Test Casesâ and âa Test Case calls Keywordsâ). Therefore the Advanced Search feature would not be a key to answer to your original question: which approach is better.
Hi, dont know if this helps but if you right-click on the Class or Package shown in the Tests Explorer view (in Keywords folder) and click "Show Referencesâ you will at least see the TCs that use it, along with the specific method.
I didnât know this. Thank you for letting me know.
But, when I tried this in my project, âShow Referencesâ showed 0 result, where I expected to see more than 1.
I studied âShow Referencesâ a bit and found it is buggy. The result is unreliable. I believe, Katalon developers havenât tested this feature thoroughly. You shouldnât expect it to work correctly. If the result is not 100% correct, the feature is useless.
How is it buggy? ---- well, I am lasy and would not report my findings. Even if I report it, I am sure, they will just put it into the backlog, and nobody would mind it.
Same here.
A cross reference from TestCases to a Class is too coase grained. It is not useful. We would need a cross reference from TestCases to a Method of a Class.
I looked at the editor pane of a Keyword class source, chose a method, right-clicked for drop-down menu. There I found âReferenceâ menu.
This âReferenceâ feature seemed to report a cross reference from Test Cases to a Method of a Class. If it work, I would love it.
I tried âReferenceâ menu and unfortunately I found it is buggy as well. So I would not used this.
I guess that this âReferenceâ menu is automatically generated by the underlying Eclipse Platfrom, but Katalon team hasnât devoted themselves to make the âReferenceâ menu useful.
@kazurayam - here is how I use it (and bearing in mind I am unable to do so at method level, only class):
right-click on Class (eg âRandomString.groovyâ):
and then click âShow Referencesâ. From here , I get results as follows. You can see that the full package>class>method path is visible and double-clicking one of these results opens the script at the relevant point. I think perhaps not what you were looking for but it is possible to see methods in the results returned - although you will see ALL methods from the class of course
Thank you for your response.
This screenshot reveals that your Test Case Script Common/CreateXXXXX
is written as follows:
AreaInputTest = CustomKeywords.'randomString.RandomString.randomStringDate'()
Let me call this style as viaCustomeKeywords.
An alternative style is possible:
import static randomString.RandomString.randomStringDate;
...
AreaInputTest = randomStringDate()
I would call this style as Plain Old Java Style (POJaS).
I prefer POJaS to viaCustomKeywords. The viaCustomKeywords style looks odd to me. I donât like it. I never write Test Cases in the viaCustomKeywords style.
I do not need the viaCustomKeywords style at all. Because I always use the Script mode to write Test Cases. I never use the Manual mode.
I found that the âShow Referencesâ can find only the Test Cases in viaCustomKeywords style. It fails to find Test Cases in the POJaS. Therefore I wrote âShow Referencesâ is buggy.
You may say this is by design, not a bug. â Fair enough. I would accept that.
Then I would say; the âShow Refereceâ feature is designed useless for me.