Execution that's been handled is still considered failed?

Hi.
I just want to ask. I check the “Take Screenshot when execution failed” setting.
So, if i handled it with FailureHandling, is it still considered failed ?

I just find it weird. Is there any alternatives besides FailureHandling ?

verifyResult = WebUI.verifyElementNotPresent(findTestObject(‘Login/text_errorLogin’), 1, FailureHandling.OPTIONAL)

while (verifyResult == false) {
WebUI.click(findTestObject(‘Login/button_login’))
}

Thats my script to handling “unexpected error” when login. My PDF Report is doing screenshot without takeScreenshot scripts. I guess its because of the setting 'Take Screenshot when execution failed".
Thank you.

1 Like

I don’t understad this sentence. It doesn’t make sense to me.

1 Like

When a Keyword invokation threw a StepFailedException (it means, the keyword execution failed), and in case that you had chosen the setting “Take Screenshot when execution failed” to be ON, then a screenshot will be taken regardless which option of FailureHandling you specified in the script.

Only when a keyword invokation threw a StepFailedException, the FailureHandling option (OPTIONAL, CONTINUE_ON_FAILURE, STOP_ON_FAILURE) will be referred to in order to determine how Katalon Studio’s Test Case Executor class should behave:

  • whether the Executor should stop the execution of the script immediately, or continue?
  • whether the Executor should memorize the Stacktrace of the Exception thrown by the keyword, or not?.

When a keyword invokation threw no StepFailedException (it means, the keyword passed), the FailureHandling option will be just ignored.

Let me repeat it: the “Take Screenshot when execution failed” setting and the FailureHandling option — these 2 spells have nothing to do with each other.

1 Like

kindly clarify your question

1 Like

I suppose that @wiky.hendra wants to make sure that the HTML element selected by findTestObject(‘Login/text_errorLogin’) to disappear in a timeout period; and when the HTML element is still present after the timeout expired, he wants no screenshot to be taken.

Then, @wiky.hendra should use WebUI.waitForElementNotPresent instead of WebUI.verifyElementNotPresent.

The WebUI.waitFor* keyword never throw a StepFailedException (= never fails). Therefore no screenshot will be taken even if the “Take Screenshot when execution failed” is set to be ON.

1 Like

Hi @wiky.hendra,
You are correct. In Katalon Studio, if the “Take Screenshot when execution failed” setting is checked, screenshots will be taken for steps with a failed, error, or warning status.

1 Like

Hi. Sorry for late replying. I was questioning how FailureHandling works, since the name is reffered to handling a failure (just by its name).

So, my assumption is if one line is Failed with FailureHandling, so its been handled, right. Is it handling the “Failed” one with OPTIONAL / CONTINUE_ON_FAILURE to become some flag (such as Warning). Based on your answer in other comment,
" When a Keyword invokation threw a StepFailedException (it means, the keyword execution failed), and in case that you had chosen the setting “Take Screenshot when execution failed” to be ON, then a screenshot will be taken regardless which option of FailureHandling you specified in the script.

Only when a keyword invokation threw a StepFailedException, the FailureHandling option (OPTIONAL, CONTINUE_ON_FAILURE, STOP_ON_FAILURE) will be referred to in order to determine how Katalon Studio’s Test Case Executor class should behave:

  • whether the Executor should stop the execution of the script immediately, or continue?
  • whether the Executor should memorize the Stacktrace of the Exception thrown by the keyword, or not?.

When a keyword invokation threw no StepFailedException (it means, the keyword passed), the FailureHandling option will be just ignored."

Now i understand that FailureHandling is handling about execution, not about Failed/Success/Warn. Thank you very much. This detail makes me understand that every scripts cant be FailureHandling since its not really effective.

Let me explain about condition in this website.
So, this website have a login function. When you login with true password, sometimes it occurs error. This is an object of an error.
findTestObject(‘Login/text_errorLogin’)

If the error occurs, i want to click button login until its success.
WebUI.click(findTestObject(‘Login/button_login’))

This condition leads me to making a while loop with verifyElementNotPresent expecting the element is not present most of the time (return false in verifyElementNotPresent).
But, when its present, it gives me screenshot, which i dont need it.
For now, i use a bit different code

while (!(WebUI.verifyElementNotPresent(findTestObject('Login/text_errorLogin'), 1, FailureHandling.OPTIONAL))) {
	WebUI.click(findTestObject('Login/button_login'))
}

just the sake of cutting lines.

Ah alright. Can i ask about the setting ?
I made a code for renaming the screenshot file. Can i rename it with picture of “Take Screenshot when execution failed” setting ?

No, you can’t.

A screenshot taken on a WebUI.* keyword failure — how is it named? For example, the following screenshot shows, a sample project on my machine created a screenshot named 1733040516240.png, 1722040522047.png and others.

What is "1733040516240"?

I can easily guess that the file name was determined as follows:

    private String newFileName() {
        return System.currentTimeMillis() + "." + SCREENSHOT_EXT;
    }

You can read the source codes of Katalon Studio. You can find it in the folder

  • <Katalon installation directory>/Contents/Eclipse/configuration/resources/source

Let me trace a bit.

When a keyword ran and failed, the stepFiled() method of com.kms.katalon.core.webui.keyword.internal.WebUIKeywordMain class will be called, which is as follows:

package com.kms.katalon.core.webui.keyword.internal;
...
public class WebUIKeywordMain {
    ...
    public static stepFailed(String message, FailureHandling flHandling, Throwable t, 
                             boolean takeScreenShot)
        throws StepFailedException {
            KeywordMain.stepFailed(message, flHandling, new StepFailedException(message, t), 
                new WebUIScreenCaptor().takeScreenshotAndGetAttributes(takeScreenShot));
    }

Here you can see that new WebUIScreenshot().takeScreenshotAndGetAttribute(boolean) is called. This call actually takes a screenshot and save an PNG image into the Reports folder.

You yourself should be able to trace the code further. You would find the ScreenCaptor class:

package com.kms.katalon.core.helper.screenshot;
...
public abstract class ScreenCaptor {
    ...
    protected static final String SCREENSHOT_EXT = "png";

    protected final KeywordLogger logger = KeywordLogger.getInstance(this.getClass());

    /**
     * System will take browser screenshot when the executed keyword decides it can do
     * <code> takeScreenShot = true </code> and screenshot capture options is enabled by users
     *
     * @param takeScreenShot
     *            true if keyword can take screenshot
     * @return Map of attributes contains attachment XML log
     */
    public final Map<String, String> takeScreenshotAndGetAttributes(boolean takeScreenShot) {
        if (!takeScreenShot || !isScreenCaptureEnabled()) {
            return Collections.emptyMap();
        }

        Map<String, String> attributes = new HashMap<>();
        try {
            File newScreenshot = getNewFile(logger.getLogFolderPath());

            take(newScreenshot);

            attributes.put(StringConstants.XML_LOG_ATTACHMENT_PROPERTY, newScreenshot.getName());
        } catch (ScreenCaptureException e) {
            logger.logWarning(
                    MessageFormat.format(StringConstants.KW_LOG_WARNING_CANNOT_TAKE_SCREENSHOT,
                            ExceptionsUtil.getMessageForThrowable(e)), null, e);
        }

        return attributes;
    }

    protected abstract void take(File newFile) throws ScreenCaptureException;

    private String newFileName() {
        return System.currentTimeMillis() + "." + SCREENSHOT_EXT;
    }

    private File getNewFile(String parentFolder) {
        File newFile = null;
        while (newFile == null || newFile.exists()) {
            newFile = new File(parentFolder, newFileName());
        }
        return newFile;
    }

    private final boolean isScreenCaptureEnabled() {
        @SuppressWarnings("unchecked")
        Boolean screenCaptureOption = (Boolean) ((Map<String, Object>) RunConfiguration
                .getExecutionGeneralProperties()
                                                 .get(StringConstants.CONF_PROPERTY_REPORT))
                .get(StringConstants.CONF_PROPERTY_SCREEN_CAPTURE_OPTION);

        return screenCaptureOption.booleanValue();
    }
}

So, the name of screenshot is determined by the system clock.

1 Like

@wiky.hendra

I think you do not need to rename the screenshot png files at all.

In the Reports folder, you can find a file named execution0.log. It is a XML file. It contains all log records of a Test Suite execution, from which HTML/PDF reports are compiled. In the execution0.log file you can find a <record> element which contains the name of the png file (e.g, 1733040516240.png), and the detail information associated (the timestamp, the exception stacktrace, etc). See the following example fragment.

...
<record>
  <date>2024-12-01T08:08:37.694520Z</date>
  <millis>1733040517694</millis>
  <nanos>520000</nanos>
  <sequence>53</sequence>
  <level>WARNING</level>
  <class>com.kms.katalon.core.logging.XmlKeywordLogger</class>
  <method>logMessage</method>
  <thread>1</thread>
  <message>Web element with id: &amp;apos;Make Appointment button&amp;apos; located by &amp;apos;By.xpath: //a[@id=&amp;apos;btn-make-appointment&amp;apos;]&amp;apos; is present after &amp;apos;3&amp;apos; second(s) (Root cause: com.kms.katalon.core.exception.StepFailedException: Web element with id: &amp;apos;Make Appointment button&amp;apos; located by &amp;apos;By.xpath: //a[@id=&amp;apos;btn-make-appointment&amp;apos;]&amp;apos; is present after &amp;apos;3&amp;apos; second(s)
	at com.kms.katalon.core.webui.keyword.internal.WebUIKeywordMain.stepFailed(WebUIKeywordMain.groovy:64)
	at com.kms.katalon.core.webui.keyword.internal.WebUIKeywordMain$stepFailed$0.call(Unknown Source)
	at com.kms.katalon.core.webui.keyword.builtin.VerifyElementNotPresentKeyword$_verifyElementNotPresent_closure1.doCall(VerifyElementNotPresentKeyword.groovy:113)
	at com.kms.katalon.core.webui.keyword.builtin.VerifyElementNotPresentKeyword$_verifyElementNotPresent_closure1.doCall(VerifyElementNotPresentKeyword.groovy)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at com.kms.katalon.core.webui.keyword.internal.WebUIKeywordMain.runKeyword(WebUIKeywordMain.groovy:20)
	at com.kms.katalon.core.webui.keyword.internal.WebUIKeywordMain$runKeyword.call(Unknown Source)
	at com.kms.katalon.core.webui.keyword.builtin.VerifyElementNotPresentKeyword.verifyElementNotPresent(VerifyElementNotPresentKeyword.groovy:122)
	at com.kms.katalon.core.webui.keyword.builtin.VerifyElementNotPresentKeyword.execute(VerifyElementNotPresentKeyword.groovy:68)
	at com.kms.katalon.core.keyword.internal.KeywordExecutor.executeKeywordForPlatform(KeywordExecutor.groovy:74)
	at com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords.verifyElementNotPresent(WebUiBuiltInKeywords.groovy:1481)
	at TC1.run(TC1:27)
        ...
   </message>
  <nestedLevel>1</nestedLevel>
  <escapedJava>false</escapedJava>
  <property name="failed.exception.class">com.kms.katalon.core.exception.StepFailedException</property>
  <property name="attachment">1733040516240.png</property>
    ...

If I want to find out what the 1733040516240.png is, I would look into the execution0.log file and find the answer.

Honestly, the needs are preparing screenshot for manual tester documentation. I can deal if its my code with following code.
‘’’
‘Take screenshot for SIT’
WebUI.takeScreenshot((((GV.screenshotFolderPath + GV.additionalFolderPath) + 'Screenshot SIT Documentation - ‘) + (GV.incScreenshot)++) +
‘.jpg’)
‘’’
Simply put the screenshot file into Test Suite projects and incrementing it.

If it can change it, it will be beautiful in reports, and user will know the flow since its incrementing.
If it can’t, its fine. Thanks for the knowledge, appreciate it. I will open that source for my depth understanding.

You seem to be prepared to call WebUI.takeScreenshot keyword explicitly. It seems you won’t rely on the “takes screenshot on failure” feature any more. Then, fine. I think you can specify whatever file path you want for the screenshots.

1 Like

yes, sir. Once again, thanks for the support.