Visual Testing in Katalon Studio

Hi,

For this use case you can try to search for Sikuli library which provides you with a set of APIs to find an image within another image.

Cheers !

1 Like

Hey Thanh, Sikuli was actually what I used back then and it wouldn’t work with finding say an element button in a full screen image.

1 Like

Hi.

What was the difficulties ? I tried Sikuli before and I think it works rather nicely.

1 Like

It wouldn’t find the image I will try again maybe it’s fixed.

1 Like

My VisualTestingInKatalonStudio project uses the aShot library.

The aShot library supports taking an image of a specific Web Element in a page, as well as taking a full page screenshot.

WebElement view

aShot takes a screenshot in three simple steps:

  • Capture a screenshot of the entire page
  • Find the element’s size and position
  • Crop the original screenshot

As a result, aShot provides an image of the WebElement

If you are interested in a specific Web Element in a page and if you can locate that element using Katalon’s Web Object (using XPath expression or CSS selector), then I suppose that aShot might be useful for you.

The example VisualTestingInKatalonStudio project demonstrates how to compare pairs of entire page screenshots. But you can compare images of specific Web Elements in pages as well. Full page or not — it doesn’t really matter.

1 Like

Hi Kazurayam,

What If I create the base assertion screenshots to be compared manually, I’m assuming it would still work correct?

Say I manually capture an element save it locally somewhere, then pass it on to find in the full page screenshot?

1 Like

Unfortunately not.

1 Like

Hi @ThanhTo

Do you have a code sample for Sikuli?

1 Like

@kazurayam If I use the framework to capture the screenshots for the elements can the scenario work?

Basically to test the same environment like someone mentioned.

  1. Create baseline element image through framework (never update)
  2. Create 2nd element image to compare through framework.
1 Like

@kazurayam

Worked like a charm thanks for this amazing library was pulling hair with Sikuli again.

Basically I run the script once to take screenshot of Element into Baseline Folder (Will Change As UI changes, rarely)
Then change the script to take screenshot of Element into the Current Folder
Then run the comparison

Thanks Again!!! Saved me countless of hours

2 Likes

@ealiaj,

I am not sure if you have already solved your problem, or still asking me some question. If you have any question, could you please reword it a bit more clearly?

1 Like

solved thanks

1 Like

The following post will show you a simpler way to compare images of a specific web element in a page.

1 Like

I have upgraded the VisualTestingInKatalonStudio project to v1.12.0.

As of v1.12.0, the Modal dialog presents 2 panes in a carousel format. The first pane shows Expected image on the left, Actual image on the right. concise_parallel And another pane shows the Diff image. concise_diff You can switch between the 2 panes by a single mouse click on button_right_next. This format of index.html makes it easy to locate and see the visual differences in the Expected-Actual images.

See
http://vtprojects.kazurayam.com/VisualTestingInKatalonStudio/Materials/index.html
for the new format in action.

4 Likes

I wrote an article in Qiita in Japanese:

あなたのWebサイトをVisual Testingする方法

there I described how to enable a Katalon Studio project to perform what I call “Visual Testing”. A few steps of preparation makes a Katalon project capable of capturing anonymous Web site you are interested in, analyzing the screenshots and compiling reports.

I wrote this because I wanted to introduce my product to my colleagues in Tokyo. It must be written in Japanese.:wink:

3 Likes

:+1:

It seems I’m able to read some Japanese. I understood “Web”, “Visual Testing” and “Katalon Studio” from your article. :wink:

1 Like

I have released the version 1.14.0 of VisualTestingInKatalonStudio. Wtih this version, the index.html displays the name of Execution Profiles as the following example shows. Displaying Execution Profile names like “Production” and “Development” makes the report easier to understand.

If you want to enable your own Katalon Studio project to carry out the Visual Testing approach as I propose, please refer to

1 Like

I have updated the VisualTestingInKatalonStudio project to the version 1.17.3.. In this version I fixed a OutOfMemoryError: Java Heap Space problem as described at https://github.com/kazurayam/Materials/issues/17

If you have any project using the VisualTestingInKatalonStudio as framework, you can easily update the framework to the latest version. In your project, you have build.gradle file, right? You change the vt.version as follows:

plugins {
  id "com.github.kazurayam.visualtestinginks" version "0.1.21"
}
vt.version = '1.17.3'

And in the commandline, you want to do:

$ cd $yourVisualTesingProject
$ ./gradlew enableVisualTesting

then, the new version of “VisualTestingInKatalonStudio” artifacts will be automatically downloaded and located in you project. The updates to the 1.17.3 requires no changes to your test case scripts.

In the $KatalonStudioInstalledDirectory/katalon.ini file, you specify the maximum heap space allocated to Java VM of Katalon Studio. I would recommend you to write in the katalon.ini

-Xms512m
-Xmx4096m
1 Like

I have updated the VisualTestingInKatalonStudio project to the version 1.17.4.

This version added information how many screenshots were taken (PNG:x) and how long each Test Suite execution took in seconds (TIME:t). See the following screenshots for example:

When I tried to test over 300 URLs it took approximately 30minutes. I wanted to know exact figures how many URLs, how long it took. So I modified the framework.

If you have any project using the VisualTestingInKatalonStudio as framework, you can easily update the framework to the latest version. In your project, you have build.gradle file, right? You change the vt.version as follows:

plugins {
  id "com.github.kazurayam.visualtestinginks" version "0.1.21"
}
vt.version = '1.17.4'

And in the commandline, you want to do:

$ cd $yourVisualTesingProject
$ ./gradlew enableVisualTesting

then, the new version of “VisualTestingInKatalonStudio” artifacts will be automatically downloaded and located in you project. The updates to the 1.17.4 requires no changes to your test case scripts.

tag 1.23.0 implemented this.

In the Test Cases/CURA/visiteSite script, now write code that calls MaterialRepository#resolve*Path* method with argument of type MaterialDescription . You are supposed to write any description what that screenshot is about. For example, a screenshot of http://demoaut.katalon.com can have a description ホームページ in my native language. The code will look like this.

// create one more screenshot file with name in Japanese
Path png2 = mr.resolveMaterialPath(
					GlobalVariable[MGV.CURRENT_TESTCASE_ID.getName()],
					'screenshots',
					"ホーム.png",
					new MaterialDescription("1", "ホームページ"))

And the Test Suite Collection Test Suite/CURA/Execute_chronos and Test Suite/CURA/Execute_twins will produce Materials/index.html where all “diff images” will be annotated with the description which you added in the Test Cases/CURA/visitSite script.

In short, the Materials/index.html page shows descriptions of each URL’s screenshots (you assign them in your native language, in my case 日本語で). The page looks like this. Much easier to understand, isn’t it?