Verify image and take screen shot keyword doesn't work?

So I read one post that this function does not work properly…but there was no solution or end result for this problem. I wonder if anyone figured it out… so I’m posting this here to check before I log a formal ticket for this.

I need to verify an image in a test and after playing around with it… that method does not work. Only thing I can do is verify the img src which is dumb because it wouldn’t verify a broken image or an incorrect img.

So what is the point of verify image feature?


and why is it the obj spy fails to take screenshot? I’m assuming taking a SS adds that image to the image selection method?

1 Like

Hi,

I will take a look and find a workaround. back to you soon

1 Like

Here I would show some sample codes and explain my thoughts.

You can see the URLs as test target in here

“Test Cases/test_page1” passed

Please read the source code of “Test Cases/test_page1”:

import static com.kms.katalon.core.testobject.ObjectRepository.findTestObject

import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI

WebUI.openBrowser("https://kazurayam.github.io/ks_verifyImagePresent/page1.html")
WebUI.verifyImagePresent(findTestObject("apple"))
WebUI.closeBrowser()

source

This script queries the remote web page at https://kazuraaym.github.io/ks_verifyImagePresent/page1.html which looks like

page1

The test_page1 script tries to verify if an image of apple is displayed in it. The Test Object “apple” points the element <img id="apple"> as follows:

test object apple

When I ran this, it just passes as I expected.

“Test Cases/test_page2” failed

Please read the source code of “Test Cases/test_page2”:

import static com.kms.katalon.core.testobject.ObjectRepository.findTestObject

import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI

WebUI.openBrowser("https://kazurayam.github.io/ks_verifyImagePresent/page2.html")
WebUI.verifyImagePresent(findTestObject("apple"))
WebUI.closeBrowser()

source

The only difference from the test_page1 script is the URL to query. page1.htmlpage2.html. The page looks like

page1

When I ran the test_page2, it failed with the following message:

2023-11-30 12:52:57.648 DEBUG testcase.test_page2                      - 2: verifyImagePresent(findTestObject("apple"))
2023-11-30 12:52:58.477 ERROR c.k.k.core.keyword.internal.KeywordMain  - ❌ Unable to verify image present (Root cause: com.kms.katalon.core.exception.StepFailedException: Unable to verify image present
	at com.kms.katalon.core.webui.keyword.internal.WebUIKeywordMain.stepFailed(WebUIKeywordMain.groovy:64)
	at com.kms.katalon.core.webui.keyword.internal.WebUIKeywordMain.runKeyword(WebUIKeywordMain.groovy:26)
	at com.kms.katalon.core.webui.keyword.builtin.VerifyImagePresentKeyword.verifyImagePresent(VerifyImagePresentKeyword.groovy:94)
	at com.kms.katalon.core.webui.keyword.builtin.VerifyImagePresentKeyword.execute(VerifyImagePresentKeyword.groovy:67)
	at com.kms.katalon.core.keyword.internal.KeywordExecutor.executeKeywordForPlatform(KeywordExecutor.groovy:74)
	at com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords.verifyImagePresent(WebUiBuiltInKeywords.groovy:2794)
	at com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords$verifyImagePresent$0.call(Unknown Source)
	at test_page2.run(test_page2:6)
...

The message tell just the keyword WebUI.verifyImagePresent failed without any diagnostics.

But I can tell you the reason. The apple image was resized by CSS width property. You can find the following line in the page2.html:

    <img id="apple" src="./a-bite-in-the-apple.png" alt="as_is" style="width: 100px">

The original image in the a-bite-in-the-apple.png file has the width = 200px. But the page2.html specifies resizing the image by style="width: 100px". So, in the browser’s view port, the image looks different from the original PNG file. Therefore the WebUI.verifyImagePresent keyword failed to find the apple. How poor the keyword is!

“Test Cases/test_page3” failed

Please read the source code of “Test Cases/test_page3”:

import static com.kms.katalon.core.testobject.ObjectRepository.findTestObject

import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI

WebUI.openBrowser("https://kazurayam.github.io/ks_verifyImagePresent/page3.html")
WebUI.verifyImagePresent(findTestObject("apple"))
WebUI.closeBrowser()

source

The only difference from the test_page1 script is the URL to query. page1.htmlpage3.html. The page looks like

page3

When I ran the test_page3, it failed with messages just like the test_page2, which tell the keyword WebUI.verifyImagePresent failed without any detail diagnostics.

But I can tell you the reason. The page2.html contains a lot of long paragraphs before the <img> element so that the apple image is positioned out of the browser’s view port when the page is loaded.

    <p>Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. At tellus at urna condimentum mattis. Cras adipiscing enim eu turpis egestas. Lectus proin 
    ....
    <img id="apple" src="./a-bite-in-the-apple.png" alt="as_is">

The WebUI.verifyImagePresent keyword tries to find the target image in the screenshot of the current desktop. It does not automatically scroll the browser’s view port to the target <img> element. The WebUI.verifyImagePresent keyword was unabled to find the apple image out of the view port. How poor the keyword is!

Why the message from the keyword is so poor?

As the above samples show, the error message emitted by the WebUI.verifyImagePresent keyword is poor. It tells nothing about why the keyword failed to find the image you specified. Therefore when the keyword failed, almost always, the users will loose their way to find out what’s wrong. I think that this poorness of diagnostics from the keyword is the most serious defect of the keyword. The verifyImagePresent keyword is too difficult to use rightly.

Why the message is so poor?

You can study the source code and find the reason. You can find the source code of WebUI.verifyImagePresent keyword at the following:

The VerifyImagePresentKeyword class delegates the task of “verifying if the image is present in the web page” to the Sikuli project’s library. (The version of Sikuli’s bundled in Katalon Studio seem to be very old. I could not find an appropriate URL to refer to for the further information). Ultimately the following class does the task.

import org.sikuli.api.DesktopScreenRegion;
import org.sikuli.api.ImageTarget;
import org.sikuli.api.ScreenRegion;
import org.sikuli.api.Target;

public class ScreenUtil {

    private ScreenRegion mainScreen;
    
    private double similarity = 0.75; // Default value

    public ScreenUtil() {
        mainScreen = new DesktopScreenRegion();
    }
    ....
    private ScreenRegion findImage(String imagePath) throws Exception {
        File imgFile = new File(imagePath);
        if (imgFile.exists()) {
            Target target = new ImageTarget(imgFile);
            target.setMinScore(this.similarity);
            ScreenRegion reg = this.mainScreen.find(target);
            return reg;
        } else {
            throw new Exception(StringConstants.COMM_EXC_IMG_FILE_DOES_NOT_EXIST);
        }
    }

The point is this single line:

            ScreenRegion reg = this.mainScreen.find(target);

Katalon’s WebUI.verifyImagePresent keyword is a thin wrapper of the org.sikuli.api.DefaultScreenRegion.find(ScreenRegion). The DefaultScreenRegion.find() method just returns null when it found no “apple” image in the desktop screenshot. It does not give any diagnostics. The WebUI.verifyImagePresent keyword checks if the result is null or not. When null, the keyword fails. It does no more than that.

My Conclusion

The WebUI.verifyImagePresent keywords quite often fails due to many causes. I would enumerate just some:

  1. Your test code does not wait the page to load completely
  2. The <img> element can be resized by CSS, therefore the image is displayed diffent from the image file
  3. The <img> element may be located outside the view port of the browser. You may need to scroll the view port to the target <img> element explicitly.

When WebUI.verifyImagePresent keyword failed, Katalon Studio won’t given any diagnostics why it failed. Katalon Studio can give you no clue to fix the failure.

Is it possible for Katalon to improve the diagnostics from the WebUI.verifyImagePresent keyword in future? — I don’t think it is possible. The first reason is that Katalon would not be able to change the Sikuli library. The second reason is that even Sikuli libary would NOT be able to tell you why it found no “apple” in the target desktop screenshot. How can Sikuli library detect the true causes = CSS witdth etc? Only God knows.

I think that the idea of trying to find an image in the desktop screenshot is a poor idea; it will never be productive. Don’t take it.

I personally will never use the WebUI.verifyImagePresent keyword.

When I want to assert an image is displayed in an HTML page, I would take other approach. I would verify the presence of <img src="..."> element. It would be almost enough. If I want to do further and make sure the image is displayed, then I would get the src="url" value and make an HTTP request to the URL and see if it returns HTTP Status 200.

The thing is img src is not reliable… because it never verifies the actual img… i’ve tried that. i’ve done bunch of testing around an img and at no point any of the test method actually verifies/compares the img. The reason <img src=> is not reliable is because if an image renders blurry or broken in the UI, the test will not fail since it’s only checking the img src.
On your apple logo page, can you do a transform: rotateY(180deg); and see if your test passes or fails - i’m thinking it will pass cuz it might only be verifying the resolution/size of the img, not the actual img.

I have never been asked to do such test scenario. Honestly I can not imagin why you need to do it.

Katalon Studio provides a keyword that takes a screenshot of HTML elements, like <img>, and save it into a PNG file.

Once you get a screenshot PNG file of <img> element and the base image file as base, you should be able to write a simple class that compares 2 images files to make a diff object.

I use Ashot to take diff of 2 images

I blame it on the FE… but to answer, our app does have several areas where we have “image preview” of UI overlays… think of it like skins or stencils that gets dropped on top of the UI. there’s been bugs with the img preview where it renders blurry, zoomed out, or doesn’t match the actual “Stencil”…and I wanted to capture those edge cases.

I can certainly live without it…really not high priority but it’s always nice to cover or catch these types of issues to promote the maturity of test automation.

I suppose that WebUI.verifyImagePresent keyword would not help you.

@ashraf.madina

Please have a look if you are still interested