Manipulate the progress bar on test case action?

mwarren04011990 · July 1, 2023, 9:17pm

Right now, I am running a test case, that calls a child test case 10 times…

As we can see in this screen shot, the progress bar is either all-or-nothing for this use case:

The child test case has already ran successfully some n < 10 times…

Is there any way that we can add to the number of progress bar on child test case pass?

kazurayam · July 1, 2023, 10:05pm

I tried to reproduce what you wrote.

Let me assume you have 2 Test Cases: “Test Case/main” and “Test Case/sub”. The “main” has the following code:

import static com.kms.katalon.core.testcase.TestCaseFactory.findTestCase

import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI

for (int i = 0; i < 10; i++) {
	int j = i + 1
	WebUI.callTestCase(findTestCase('Test Cases/sub'), ['seq': j])
}

and the “sub” has the following code:

import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI

WebUI.comment("seq=" + seq)
WebUI.delay(1)

when I ran the “Test Case/main”, I saw the following output in the console:

2023-07-02 06:54:57.781 INFO  c.k.katalon.core.main.TestCaseExecutor   - --------------------
2023-07-02 06:54:57.785 INFO  c.k.katalon.core.main.TestCaseExecutor   - START Test Cases/main
2023-07-02 06:54:58.559 INFO  c.k.katalon.core.main.TestCaseExecutor   - --------------------
2023-07-02 06:54:58.562 INFO  c.k.katalon.core.main.TestCaseExecutor   - CALL Test Cases/sub
2023-07-02 06:54:58.683 INFO  c.k.k.c.keyword.builtin.CommentKeyword   - seq=1
2023-07-02 06:54:59.728 INFO  c.k.katalon.core.main.TestCaseExecutor   - END CALL Test Cases/sub
2023-07-02 06:54:59.728 INFO  c.k.katalon.core.main.TestCaseExecutor   - --------------------
2023-07-02 06:54:59.818 INFO  c.k.katalon.core.main.TestCaseExecutor   - --------------------
2023-07-02 06:54:59.818 INFO  c.k.katalon.core.main.TestCaseExecutor   - CALL Test Cases/sub
2023-07-02 06:54:59.837 INFO  c.k.k.c.keyword.builtin.CommentKeyword   - seq=2
2023-07-02 06:55:00.858 INFO  c.k.katalon.core.main.TestCaseExecutor   - END CALL Test Cases/sub
2023-07-02 06:55:00.859 INFO  c.k.katalon.core.main.TestCaseExecutor   - --------------------
2023-07-02 06:55:00.931 INFO  c.k.katalon.core.main.TestCaseExecutor   - --------------------
2023-07-02 06:55:00.931 INFO  c.k.katalon.core.main.TestCaseExecutor   - CALL Test Cases/sub
2023-07-02 06:55:00.943 INFO  c.k.k.c.keyword.builtin.CommentKeyword   - seq=3
2023-07-02 06:55:01.957 INFO  c.k.katalon.core.main.TestCaseExecutor   - END CALL Test Cases/sub
2023-07-02 06:55:01.957 INFO  c.k.katalon.core.main.TestCaseExecutor   - --------------------
2023-07-02 06:55:02.032 INFO  c.k.katalon.core.main.TestCaseExecutor   - --------------------
2023-07-02 06:55:02.032 INFO  c.k.katalon.core.main.TestCaseExecutor   - CALL Test Cases/sub
2023-07-02 06:55:02.045 INFO  c.k.k.c.keyword.builtin.CommentKeyword   - seq=4
2023-07-02 06:55:03.058 INFO  c.k.katalon.core.main.TestCaseExecutor   - END CALL Test Cases/sub
2023-07-02 06:55:03.058 INFO  c.k.katalon.core.main.TestCaseExecutor   - --------------------
2023-07-02 06:55:03.133 INFO  c.k.katalon.core.main.TestCaseExecutor   - --------------------
2023-07-02 06:55:03.133 INFO  c.k.katalon.core.main.TestCaseExecutor   - CALL Test Cases/sub
2023-07-02 06:55:03.143 INFO  c.k.k.c.keyword.builtin.CommentKeyword   - seq=5
2023-07-02 06:55:04.163 INFO  c.k.katalon.core.main.TestCaseExecutor   - END CALL Test Cases/sub
2023-07-02 06:55:04.163 INFO  c.k.katalon.core.main.TestCaseExecutor   - --------------------
2023-07-02 06:55:04.238 INFO  c.k.katalon.core.main.TestCaseExecutor   - --------------------
2023-07-02 06:55:04.238 INFO  c.k.katalon.core.main.TestCaseExecutor   - CALL Test Cases/sub
2023-07-02 06:55:04.249 INFO  c.k.k.c.keyword.builtin.CommentKeyword   - seq=6
2023-07-02 06:55:05.260 INFO  c.k.katalon.core.main.TestCaseExecutor   - END CALL Test Cases/sub
2023-07-02 06:55:05.261 INFO  c.k.katalon.core.main.TestCaseExecutor   - --------------------
2023-07-02 06:55:05.336 INFO  c.k.katalon.core.main.TestCaseExecutor   - --------------------
2023-07-02 06:55:05.336 INFO  c.k.katalon.core.main.TestCaseExecutor   - CALL Test Cases/sub
2023-07-02 06:55:05.370 INFO  c.k.k.c.keyword.builtin.CommentKeyword   - seq=7
2023-07-02 06:55:06.384 INFO  c.k.katalon.core.main.TestCaseExecutor   - END CALL Test Cases/sub
2023-07-02 06:55:06.384 INFO  c.k.katalon.core.main.TestCaseExecutor   - --------------------
2023-07-02 06:55:06.453 INFO  c.k.katalon.core.main.TestCaseExecutor   - --------------------
2023-07-02 06:55:06.453 INFO  c.k.katalon.core.main.TestCaseExecutor   - CALL Test Cases/sub
2023-07-02 06:55:06.463 INFO  c.k.k.c.keyword.builtin.CommentKeyword   - seq=8
2023-07-02 06:55:07.475 INFO  c.k.katalon.core.main.TestCaseExecutor   - END CALL Test Cases/sub
2023-07-02 06:55:07.475 INFO  c.k.katalon.core.main.TestCaseExecutor   - --------------------
2023-07-02 06:55:07.547 INFO  c.k.katalon.core.main.TestCaseExecutor   - --------------------
2023-07-02 06:55:07.547 INFO  c.k.katalon.core.main.TestCaseExecutor   - CALL Test Cases/sub
2023-07-02 06:55:07.558 INFO  c.k.k.c.keyword.builtin.CommentKeyword   - seq=9
2023-07-02 06:55:08.568 INFO  c.k.katalon.core.main.TestCaseExecutor   - END CALL Test Cases/sub
2023-07-02 06:55:08.569 INFO  c.k.katalon.core.main.TestCaseExecutor   - --------------------
2023-07-02 06:55:08.638 INFO  c.k.katalon.core.main.TestCaseExecutor   - --------------------
2023-07-02 06:55:08.638 INFO  c.k.katalon.core.main.TestCaseExecutor   - CALL Test Cases/sub
2023-07-02 06:55:08.648 INFO  c.k.k.c.keyword.builtin.CommentKeyword   - seq=10
2023-07-02 06:55:09.661 INFO  c.k.katalon.core.main.TestCaseExecutor   - END CALL Test Cases/sub
2023-07-02 06:55:09.662 INFO  c.k.katalon.core.main.TestCaseExecutor   - --------------------
2023-07-02 06:55:09.678 INFO  c.k.katalon.core.main.TestCaseExecutor   - END Test Cases/main

In the GUI, the Job Progress showed 2 views: at first when it started:

then 10 seconds later, it chainged to

The Job Progress showed no progress; it started and finished. That’s all.

I could reproduce what @mwarren04011990 reported.

kazurayam · July 1, 2023, 10:06pm

No, there is no way.

I will tell you why as follows.

kazurayam · July 1, 2023, 10:16pm

You can find the source code of CallTestCase keyword of an old version Katalon Studio here:

github.com

katalon-studio/katalon-studio-testing-framework/blob/master/Include/scripts/groovy/com/kms/katalon/core/keyword/builtin/CallTestCaseKeyword.groovy

package com.kms.katalon.core.keyword.builtin

import groovy.transform.CompileStatic

import java.text.MessageFormat

import org.apache.commons.lang.math.NumberUtils

import com.kms.katalon.core.annotation.Keyword
import com.kms.katalon.core.annotation.internal.Action
import com.kms.katalon.core.constants.StringConstants
import com.kms.katalon.core.exception.StepErrorException
import com.kms.katalon.core.exception.StepFailedException
import com.kms.katalon.core.helper.KeywordHelper
import com.kms.katalon.core.keyword.internal.AbstractKeyword
import com.kms.katalon.core.keyword.internal.KeywordExecutor
import com.kms.katalon.core.keyword.internal.KeywordMain
import com.kms.katalon.core.keyword.internal.SupportLevel
import com.kms.katalon.core.logging.ErrorCollector
import com.kms.katalon.core.logging.KeywordLogger

This file has been truncated. show original

You can find the source of related classes including TestCaseExecutor here:

github.com

katalon-studio/katalon-studio-testing-framework/blob/master/Include/scripts/groovy/com/kms/katalon/core/main/TestCaseExecutor.java

package com.kms.katalon.core.main;

import static com.kms.katalon.core.constants.StringConstants.DF_CHARSET;

import java.io.File;
import java.io.IOException;
import java.security.AccessController;
import java.text.MessageFormat;
import java.util.Collections;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.Stack;

import org.apache.commons.io.FileUtils;
import org.apache.commons.lang.StringUtils;
import org.codehaus.groovy.ast.MethodNode;
import org.codehaus.groovy.control.CompilationFailedException;
import org.codehaus.groovy.control.MultipleCompilationErrorsException;

This file has been truncated. show original

Please read the method public TestResult execute(FailureHandling flowControl) of TestCaseExecutor. You can find the following fragment:

            if (testCaseContext.isMainTestCase()) {
                eventManager.publicEvent(ExecutionListenerEvent.BEFORE_TEST_CASE, new Object[] { testCaseContext });
            }

and

            if (testCaseContext.isMainTestCase()) {
                eventManager.publicEvent(ExecutionListenerEvent.AFTER_TEST_CASE, new Object[] { testCaseContext });
            }

I believe that Test Case/main will return true for isMainTestCase() but Test Case/sub returns false. So the eventManager will be informed of the progress of the main only; it won’t be informed of the progress of the sub at all. Therefore the Job Progress window displays the progress of the Test Case/main only; it does not display anything about the Test Case/sub.

Why is the TestCaseExecutor designed as such? — I don’t know. Ask Katalon. I personally have no preference about the Job Progress window.

kazurayam · July 2, 2023, 12:09am

I can guess why the Job Progress monitor is designed to show only the Main test case, why it ignores the Sub test case.

The Katalon Studio GUI needs to know, prior to starting the job, how many Test Cases are there in total to monitor. For example, let me assume I have a Test Suite X contains Test Cases A, B, C; then the number of Test Cases to monitor is obviously 3. Provided that the total number of Test Cases to monitor is known, the GUI can draw a progress bar (0/3, 1/3, 2/3, and 3/3; Done). Without the total number of Test Cases to monitor is givien, GUI can not draw a progress bar.

Then how many Test Cases Katalon Studio GUI will find when I executed the Test Case/main ? ---- only 1.

Why the GUI does not respect the number of times where the Test Case/sub is called? — because the number will be determined only after the Main test case has been started. The main will determine how many times it will call the sub (may be only once, 10 times, or 1000 times). The main will never inform Katalon GUI of the count. Therefore Katalon GUI will never be able to render the Job Progress with the calls to the Sub test cases appropriately.

So, I guess, Katalon developers designed the Job Progress to disregard the Sub test cases at all.

Topic		Replies	Views
Delay between actions not working at test case level? Archive	2	1057	November 14, 2018
How can i make a test case repeat X number of times Katalon Studio katalon-studio , web-testing	15	19346	December 16, 2021
Test case Execution in Katalon Studio Katalon Studio katalon-studio , web-testing	11	1035	May 14, 2021
How to calculate test case run time Tips & Tricks katalon-studio , tips-tricks	1	1157	August 8, 2022
Increment Test Case Counter through Groovy Script Katalon Studio katalon-studio , api-testing	0	744	October 31, 2019

Manipulate the progress bar on test case action?

Related Topics