Hi,
I created few custom Keywords, but I am not using them directly in my TestCase, but rather in some other methods, because I am using PageObjectModel.
So for example I am using my custom keywords let’s say SetDate() and CheckMessages() in let’s say Page_Register.RegisterNewUser() method and from withing my TestCase, I call only this registerNewUser() method.
But there are 2 problems:
In LogViewer, I see only that registerNewUser() step, but I would like to also see those substeps SetDate() and CheckMessages() under that step, and those substeps should display if I expand that step using that expanding arrow. Is there any way, how can I achieve this?
In that step, I can’ call CustomKeywords.something, because it’s not possible and shows an error. I can call my keywords directly like a method, but I think Katalon will then see it only as normal function and not as keyword, beacuse when some of these substeps fails (either using KeywordUtil.markFailed or by throwing an exception), step is still shown as passed instead of failed.
So is it possible to implement this Step-Substeps pattern, or it’s just intended to use custom Keywords directly in TestCase?
Yes, of course, you can do it.(edited 13/Jul) Could you show us your code of your custom keyword? I do not see how the 3 names (registerNewUser, SetDate and CheckMessages) relates each other.
By this line, the Test Case Executer (a part of Katalon Studio) is informed that the registerNewUser method of POMClass was executed. It will display in the LogViewer a single line of step execution saying “Test Case Executor executed the registerNewUser() method of the POMClass object”.
The registerNewUser method may call other methods in the POMClass : foo() and bar(). But the method implementation hidden inside the POMClass class. This detail is not exposed to the Test Case Executor. The Test Case Executor does not know anything about foo() and bar(). Therefore in the LogView we will not see the Step Execution log of foo and bar at all.
Ok, I don’t need to see substeps in LogViewer, but it would be awesome, if I could.
But what is absolutely crucial, is to be able to have that step failed, when some of those substeps failed.
If it is possible to do it, would you please show me?
I tried both KeywordUtil.markFailed() method and throwing Exception in those substeps, but my main step still shows as passed, which is extremely strange, because Katalon Studio absolutely should mark step as failed, when there is uncaught exception anywhere in execution. Alsowherever I write KeywordUtil.markFailed() (in any of substeps or even substeps of substeps), it should mean, that the most parent Step is failed.
I also tried to mark the most parent Step as Keyword, and all Substeps are also marked as keyword, but still not working…
I started to use Katalon Studio like a month ago and I am still just trying to solve many problems it has I wonder, how is it possible, that fundamental problems like I have are still not solved
Do you mean, you use “KeywordUtil.markFailed” inside your Custom Keyword’s implementation?
Probably it would not work (or work strange), as “KeywordUtil” is supposed to be used in Test Case scripts, not inside Custom Keywords. Though I have never tried it personally — as I doubt it. (edit 13/Jul) I was wrong. “KeywordUtil” is supposed to be used in classes as Custom Keywords. And I do use it as well. Calling KeywordUtil.markFailed() throws a StepFailedException.
You can make it simple. Why not you write your Custom keyword class to throw some Exception? such as IllegalArgumentException, IllegalStateException, IOException, MyOwnCustomException, etc.
Assuming your Custom Keyword is designed to throw some Exception, you want to write your Test Case like this:
try {
CustomKeywords."my.brilliant.POMClass.RegisterNewUser"("something")
} catch (Exception e) {
KeywordUtil.markFailedAndStop("the end of the world")
}
Yes, I used it inside Custom Keyword’s implementation, because that’s what docs says:
(“Mark a keyword to be failed and continue execution”) https://docs.katalon.com/javadoc/com/kms/katalon/core/util/KeywordUtil.html
So I am now very confused, when you said, it is supposed to be used inside TestCase. If you are right, then the documentation is dumb. even the name KeywordUtil totally suggest, that this class should be used inside Custom Keyword’s implementation
Are you sure in what you are saying?
I tried to throw exception, and it marks whole TestCase as failed, but marks my Step (my custom Keyword) as passed, even if it throws exception.
To be honest, I sometimes wonder, what were the thoughts of Katalon Developers, when they were implementing this. There is no way how you can do this Steps-Substeps pattern, which is absolutely logical way of doing it, even they implemented it in their custom keywords, that’s why sometimes, you can expand step in LogViewer and see substeps, but it’s not possible to do it in my own keywords?
And how come throwing an exception won’t mark that step as failed? That’s totally weird and I read many other posts, where people had a problem with this, that Step, or even whole TestCase is marked as passed, even though it definitely should be marked as failed. Even writing “assert something” won’t make TestCase failed…
Like almost nothing seems to be working in a good way here in Katalon Studio, I start to be demotivated…
When you want to use data binding, it’s binding everything as a String, so I had to create my own workaround to be able to use something else, than just a String…
How come you guys use Katalon Studio, when it has so many problems?
Sorry, still I do not understand what you call “Steps-Substeps pattern”. Could you make a small runnable project which demonstrates your arguments, and disclose it here by attaching the zip of the project? Also could you give us descriptions how weird you think the demo project is, with reference to the concrete code?
Both @kazurayam and I are experienced KS users but I think it’s fair to say, we tend to approach Katalon differently and, therefore, sometimes differ on how a particular API is intended to be used.
For me, markFailed is most definitely of use inside Keyword classes/methods. I did a search in my classes and found it’s being used 104 times.
Even so, that does not mean I can promise you an outcome you would like. My testing philosophy and approach are likely very different to yours - my POM takes advantage of my own test case executor developed over a number of years.
That said, here’s how my Test Case philosophy works from a high level overview:
I don’t care about passes. I don’t log them, I don’t keep them, I don’t add them to reports. PASSES == NOISE.
I write my own messages to the log using WARNINGs – KeywordUtil.markWarning(). These messages are “steps” passed to the Report subsystem.
For failures I also throw a StepErrorException – this is the point at which my code exits to the native Katalon executor.
You can see some of the internals here in this screenshot:
I repeat - I don’t expect you to glean much from this. I certainly could not throw a few lines of code at you and suddenly you have things working just as you would like. But what I am saying is, you can develop a satisfactory system if you are prepared to put the work in. It’s just “java and groovy” and, as @kazurayam would attest, if you don’t like how Katalon does something, you can hijack the internals and make it work your way, if that’s what you want.
You mentioned few things that are very important for me:
You created your own TestExecutor - can you please tell me, how I can also implement my own TestExecutor? Or maybe even share your TestExecutor implementation with me?
“You can hijack the internals” - Can you please show me how can I work with Katalon internals?
But still, how you hijack those internals? How to make Katalon Studio work with any of my modifications?
How you made your TestCaseExecutor work with Katalon Studio?
Do you create a plugin to use your own TestCaseExecutor?
You have a misunderstanding. It is @Russ_Thomas who wrote “you can hijack the internals”, not me @kazurayam. I just pointed you to the URL of Katalon source codes.
But I believe that the com.kms.katalon.core.main.TestCaseExecutor in the ver 8.1.0 would stay the same as the Dec 2020 version. So practically the Dec 2020 source code would be good enough to learn the internal of Katalon Studio.
I used the Groovy’s metaprogramming techinque in my Keyword class implementation. The class is invoked by the TestExecutor built-in Katalon Studio. Here comes a timing issue. Since the built-in TestExecutor object is already instantiated and in action, any Metaprogramming done by a TestCase (+custom keyword class) in action will never be able to modify the TestExecutor instance.
So, I believe, my article is NOT informative for you @peter.hevesi
It is totally informative, I can’ thank you enough
There is no doubt we should call you Katalon King
Aha, you just wrote, that I can’t replace that instance
Nevermind, maybe it won’t help me in what I am trying to achieve, but still I am very glad that you sent me that project
But actually, what you are saying is, that I can’t replace TestExecutor from TestCase, because it is already created and running, but I probably should replace it from TestListener?
All I want to do is to modify the behavior of “logging and reporting” so to say and I want to do it gloabaly for each TestCase, so I actually think I may be able to do it
I wonder, how is it possible, that fundamental problems like I have are still not solved 
