Logging the steps of a custom keyword method

Product Forums Katalon Studio
,
1

This question originally posted on StackOverflow.

Context

We run across the same widgets on multiple pages, and oftentimes cover multiple scenarios for a given page on the site under test. This and other concerns has motivated me to move the page action concerns to GeneralWebUIUtils and page objects, respectively, in the Keywords.

Example

For example, we have this class GeneralWebUIUtils, defined to be:

package com.xxx.utils

import com.kms.katalon.core.testobject.TestObject
import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI

public final class GeneralWebUIUtils {
	//...

	public static void HandleAutoComplete(TestObject textField, String input, TestObject loader, TestObject firstDropdownOption) {
		WebUI.click(textField)

		WebUI.setText(textField, input)

		WebUI.waitForElementVisible(loader, 3)

		WebUI.waitForElementNotVisible(loader, 3)

		WebUI.waitForElementNotPresent(firstDropdownOption, 3)

		WebUI.waitForElementVisible(firstDropdownOption, 3)

		WebUI.click(firstDropdownOption)
	}
}

In one test case in particular, I have to handle autocomplete, for which I’ll use GeneralWebUIUtils.HandleAutoComplete from this class.

OK, now what’s the issue?

I am now facing performance issue from that method, and before that, a stale element exception. Typical test case issues, just moved to a package. Whenever I check out the log viewer, it doesn’t show the steps of that method like it would if that method existed on the test case itself:

enter image description here

This is problematic because it means you can’t check out the logs to find out what is going on, let alone what is creating the bottleneck. You are therefore forced to use the debug mode, which is prone to problems of its own (think forced waiting, and b/c of that, flaky debug sessions).

How can we tell Katalon to log the different WebUI keywords like it does in the local-to-test-case version of this method?

2

Do you want to see in the LogViewer the step execution of the lines inside the HandleAutoComplete?

Then, you should not make it a Custom Keyword; because, by design, a Custom Keyword is out of the scope of LogViewer.

You should make HandleAutoComplete a Test Case. And test case calls it using the following:

https://docs.katalon.com/katalon-studio/docs/common-call-test-case.html

3

Yea that’s a code smell.

1 Like
4

No, it is not possible.

Katalon Studio assumes that all keywords are already well-tested somewhere outside KS. Therefore Katalon’s LogViewer is not designed to monitor the step execution of keywords.

I am afraid, you code is still to be debugged, not ready to form a Keyword.

In other words, if you want to develop your own Custom Keywords, you have to make sure your Custom Keywords are somehow well-tested and bug-free. But I am afraid that Katalon Studio does not help unit-testing your Custom Keywords at all. There is no documentation titled “How to unit-test your Custom Keywords in Katalon Studio” in the Katalon’s official docs. This proves, I think, that Katalon Team is not concerned about the quality assurance of user’s custom keywords.

I guess, the Katalon Team develops the built-in Keywords outside Katalon Studio product (possibly in Eclipse), and does enough unit-tests there. So they are contented. They do not imagine how much users would struggle to debug their own Custom Keywords. ==> @Jass any comment?

So, @mwarren04011990, I would advise you, it is a difficult task to develop well-tested Custom Keywords. You had better start “modularizing test cases” with callTestCase(), rather than jumping into Custom Keywords.

5

With callTestCase() you would see in the Log Viewer display every step execution log of your common routines as a Test Case.

Please be warned, however, executing common routines as Test Case will be slower than executing common routines as Keyword, because step execution logging slows down Test Cases(Groovy scripts) significantly. Keywords (Groovy class) runs far faster, as no step execution log is taken. In future, when you get worried about the speed, you may consider migrating the common routine from a Test Case to a Keyword.

6

There could be 2 types of modules in Katalon project:

  • one type is dependent on Katalon Studio’s API like WebUI.openBrowser(),
  • another type is independent on Katalon Studio’s API.

When I write my test codes I always try to distinguish these 2 types.

I would try to implement the first type of module (Katalon API dependent) as Test Cases to be called by WebUI.callTestCase(), not as Keywords. I can not develop this type of module outside Katalon Studio, as the Katalon API is not distributed as importable jar files outside the Katalon Studio product.

I would implement the second type of modules(Katalon-independent one) as Groovy/Java classes OUTSIDE Katalon Studio. I prefer IntelliJ IDEA, as IDEA is far easier than Katalon Studio to develop Groovy/Java classes. I will code, test, and build a jar file which I will import into the Drivers folder of a Katalon project. An example is here.

However, sometimes, I need to develop 3rd type of module in a Katalon Studio project. That is, a Custom Keyword that depends on Katalon Studio’s API. In order to perform unit-tests for the 3rd type Custom Keywords inside Katalon Studio, I invented a way using JUnit in KS.

7

Thank you for your detailed response. I’m sorry, but I cannot accept this “solution” of turning it into test cases.

Making a general Web UI custom utils class, whose job includes handling an auto-complete widget, and may have custom methods that work properly and solve other generic concerns (e.g. handling a date widget), its own set of test cases, is a bullshit design. They are general actions to be performed on the pages, NOT assertions.

This is especially unacceptable given that I am working on page objects. The whole point of those is to separate general page action details from the tests themselves, and thus from the verifications.

The bugs I was facing, are the bugs that pop up in general with developing any test case. Stuff like a step executing on a test object before it’s ready in the DOM, causing a StaleElementException, as well as the performance issue, which ended up being waiting for something to de-spawn, at the wrong time.

I’m sorry, maybe I don’t know as much about unit-testing as I thought I did, but how is unit-testing going to find and fix these sort of test-case-and-AUT race conditions?

8

UPDATE: I see that I can use KeywordUtils.logInfo() to log the info, in a custom keyword. This can help me find the bottleneck.

I found it and fixed it btw.