Which is the better approach? Calling test cases multiple times or calling custom keywords intead?

In my project, I am calling multiple test cases (say 10 test cases) as a flow and executing it. Is it best practice or shall I create custom keywords of 10 and use them as a flow for execution?
I wanted to know which approach is the best one?

A composite of Test Cases tends to be (far) slower than a Test Case that calles Keywords due to the verbosity of step-execution logs emitted by Katalon Studio.

Please have a look at my previous report about this point.

This report shows a comparison of 5 min 37 secs with verbose logging versus 0 min 25 secs without logging, both carry out just the same processing. I believe that this difference is significant.

On the other hand, you should note that:

  1. Katalon Studio emits verbose logs of executing steps in Test Case scripts.
  2. Katalon Studio emits no step execution log for Keyword classes.

Therefore, “Composite Test Cases” tends to be slower than “a Test Case that calls Keywords” if you do not make effort to minimize the logging verbosity.

Another aspect.

It is easier to develop & test Test Case scripts in Katalon Studio. You can run a Test Case by single button click, Katalon Studio emits verbose execution log, you can interactively see how your Test Case ran.

On the other hand, it is difficult to develop quaility Groovy classes as custom Keywords in Katalon Studio. Katalon Studio does not offer any built-in support of unit-testing of custom Keywords.

I once developed junit4ks to perform unit-test over my custom classes.

Often I develop custom classes to use in a Katalon project OUTSIDE Katalon Studio. I use IntelliJ + Gradle where I carry out enough unit-testing using JUnit4. Once developed, I will create a jar, import the jar into the Katalon project, use the classes in the jar from Test Cases.

A practical suggestion from me for easier path could be:

  1. use a composite of multiple Test Cases, rather than Keyword classes
  2. try to minimize the verbosity of step-execution logs emitted by Katalon Studio. See the above-mentioned post how to.
1 Like

Thanks @kazurayam But here I see a problem when I use Custom Keywords. The problem is, if I wanted to know where all this keyword is used, there is no way. Can you suggest any way to do this.

My use case is: If I want to update any parameterize values for a particular key word, how can I verify where this keyword is used and make changes in the test cases.

You are right. I think there is no way to do that.

Katalon Studio provides very little support for “refactoring Test Cases”.

In this aspect, you would find that the 2 patterns (“Composite Test Cases” and “a Test Case calling Keywords”) are no different. In both cases, all you can do is to repeat running all Test Case + find errors + correct the code + retry.

We have this Advance Search which Katalon provides, I think we can search for the keyword which is used in the test cases and correct or update it.

What do you think about this?

But we need to run the test case to ensure it took the updated or corrected value.

I didn’t know it.

https://docs.katalon.com/docs/katalon-studio-enterprise/create-tests-and-projects/manage-projects/search-test-cases#advanced-search

This is a fairly new feature since v8.2.5 (jan 2022). I haven’t used it. I have no opinion about it.

If you find it good enough, then fine. You wouldn’t need my opinion.


The Advanced Search seems to be a sort of “grep” over multiple test case scripts in a project. If so, it will apply similarly to the 2 approaches (“Composite Test Cases” and “a Test Case calls Keywords”). Therefore the Advanced Search feature would not be a key to answer to your original question: which approach is better.

Just use search?

1 Like

Thanks @Russ_Thomas , it served the purpose.

1 Like

Hi, dont know if this helps but if you right-click on the Class or Package shown in the Tests Explorer view (in Keywords folder) and click "Show References’ you will at least see the TCs that use it, along with the specific method.

1 Like

I didn’t know this. Thank you for letting me know.

But, when I tried this in my project, “Show References” showed 0 result, where I expected to see more than 1.

I studied “Show References” a bit and found it is buggy. The result is unreliable. I believe, Katalon developers haven’t tested this feature thoroughly. You shouldn’t expect it to work correctly. If the result is not 100% correct, the feature is useless.

How is it buggy? ---- well, I am lasy and would not report my findings. Even if I report it, I am sure, they will just put it into the backlog, and nobody would mind it.

Same here.

A cross reference from TestCases to a Class is too coase grained. It is not useful. We would need a cross reference from TestCases to a Method of a Class.

I looked at the editor pane of a Keyword class source, chose a method, right-clicked for drop-down menu. There I found “Reference” menu.

This “Reference” feature seemed to report a cross reference from Test Cases to a Method of a Class. If it work, I would love it.

I tried “Reference” menu and unfortunately I found it is buggy as well. So I would not used this.

I guess that this “Reference” menu is automatically generated by the underlying Eclipse Platfrom, but Katalon team hasn’t devoted themselves to make the “Reference” menu useful.

@kazurayam - here is how I use it (and bearing in mind I am unable to do so at method level, only class):
right-click on Class (eg ‘RandomString.groovy’):
image

and then click ‘Show References’. From here , I get results as follows. You can see that the full package>class>method path is visible and double-clicking one of these results opens the script at the relevant point. I think perhaps not what you were looking for but it is possible to see methods in the results returned - although you will see ALL methods from the class of course

@Dan_Bown

Thank you for your response.

This screenshot reveals that your Test Case Script Common/CreateXXXXX is written as follows:

        AreaInputTest = CustomKeywords.'randomString.RandomString.randomStringDate'()

Let me call this style as viaCustomeKeywords.

An alternative style is possible:

import static randomString.RandomString.randomStringDate;
...
        AreaInputTest = randomStringDate()

I would call this style as Plain Old Java Style (POJaS).

I prefer POJaS to viaCustomKeywords. The viaCustomKeywords style looks odd to me. I don’t like it. I never write Test Cases in the viaCustomKeywords style.

I do not need the viaCustomKeywords style at all. Because I always use the Script mode to write Test Cases. I never use the Manual mode.

@Dan_Bown

I found that the “Show References” can find only the Test Cases in viaCustomKeywords style. It fails to find Test Cases in the POJaS. Therefore I wrote “Show References” is buggy.

You may say this is by design, not a bug. — Fair enough. I would accept that.

Then I would say; the “Show Referece” feature is designed useless for me.

1 Like