Hello, I have the following setup for window size
but this is ignored when I am running a test suite collection and my screenshots are 3570x10083 (although I can see that the components layout shows as for the 1440px wide screen)
This did not happen before.
I even added a ‘Set View Port Size’ on the test case level:
In addition to that I noticed that some pages are being captured twice on the same screenshot (scrolling the screenshot I see the page continues again from the top to bottom)
You specified 1440 pixel as window width, but the screenshot image has a width of 3570 pixel; you wondered about these 2 values.
As a starting point, you need to know a term “Device Pixel Ratio” (a.k.a DPR). Read the following article:
What’s the Device Pixcel Ratio shortly? See the following figure quoted from this article:
You would want to know what value you have on the machine where you execute your test. Make the following script as a Test Case and run it on the environment you ran your test in question:
import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI
String js = '''
const scale = window.devicePixelRatio;
console.log(scale)
return scale
'''
WebUI.openBrowser('')
WebUI.navigateToUrl('http://demoaut.katalon.com')
Object result = WebUI.executeJavaScript(js, null)
Double devicePixelRatio = (Double)result
println "devicePixelRatio=" + devicePixelRatio
WebUI.closeBrowser()
On my machine, I got the following output in the console:
2023-06-15 10:06:43.260 INFO c.k.katalon.core.main.TestCaseExecutor - --------------------
2023-06-15 10:06:43.263 INFO c.k.katalon.core.main.TestCaseExecutor - START Test Cases/retrieveDevicePixelRatio
2023-06-15 10:06:44.140 INFO c.k.k.core.webui.driver.DriverFactory - Starting 'Chrome' driver
6 15, 2023 10:06:44 午前 org.openqa.selenium.remote.DesiredCapabilities chrome
情報: Using `new ChromeOptions()` is preferred to `DesiredCapabilities.chrome()`
2023-06-15 10:06:44.208 INFO c.k.k.core.webui.driver.DriverFactory - Action delay is set to 0 milliseconds
Starting ChromeDriver 113.0.5672.24 (65f30d4e8051264233c679c7cd3743679f15339d-refs/branch-heads/5672@{#243}) on port 20451
Only local connections are allowed.
Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.
ChromeDriver was started successfully.
[1686791208.950][WARNING]: This version of ChromeDriver has not been tested with Chrome version 114.
6 15, 2023 10:06:49 午前 org.openqa.selenium.remote.ProtocolHandshake createSession
情報: Detected dialect: W3C
2023-06-15 10:06:49.230 INFO c.k.k.core.webui.driver.DriverFactory - sessionId = b84ce7115bbf7cf4883f69b36deeaf50
2023-06-15 10:06:49.278 INFO c.k.k.core.webui.driver.DriverFactory - browser = Chrome 114.0.0.0
2023-06-15 10:06:49.286 INFO c.k.k.core.webui.driver.DriverFactory - platform = Mac OS X
2023-06-15 10:06:49.306 INFO c.k.k.core.webui.driver.DriverFactory - seleniumVersion = 3.141.59
2023-06-15 10:06:49.344 INFO c.k.k.core.webui.driver.DriverFactory - proxyInformation = ProxyInformation { proxyOption=NO_PROXY, proxyServerType=HTTP, username=, password=********, proxyServerAddress=, proxyServerPort=0, executionList="", isApplyToDesiredCapabilities=true }
devicePixelRatio=2.0
2023-06-15 10:06:52.177 INFO c.k.katalon.core.main.TestCaseExecutor - END Test Cases/retrieveDevicePixelRatio
The DPR on my machine (Mac Book Air, Retina Display) was 2.0
Then what value do you see on your machine?
You can read the source code of WebUI.takeFullPageScreenshot keyword:
If you read the source of the keyword and related classes, you would find the implementation doesn’t take the DPR into account.
Your case seems to be complicated. You case might have some factors that make the keyword to perform wrong. I don’t know what. I don’t have any more clues to resolve your case. You need to find them out on your machine for yourself.
Thank you very much!
This is what I got:
So the devicePixelRation=2.5 on my machine.
How can I update it to 1?
Your CSS pixels : 1440
Your screen device pixels : 3570
Your DPR : 2.5
1440 * 2.5 makes 3600, which is nearly equal to 3570. This makes sense.
3570 = 3600 - 30; possibly 30 pixel is occupied by a vertical scroll bar of the browser window, so the WebUI.takeFullPageScreenshot keyword intentionally chomped the 30 pixels off the image.
As long as you run your test on a single machine, you can not update the DPR by software as it is a property of the hardware.
If you run your test on another machine, the DPR could be different as the machine may be newer or older and has higher or lower display resolution.
Thanks for the answer. I will then have to accept them all as “pass” and so have a new baseline images.
Can we force Katalon to compare snapshots by using the file name only and ignore width and height of the screenshot images?
I do not know the TestOps Visual Testing feature at all.
Ask @vu.tran, or somebody who knows it well.
I would show what I made as an experiment.
My Test Case:
import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI
import java.awt.image.BufferedImage
import javax.imageio.ImageIO
WebUI.openBrowser('')
int cssWidth = 1400
int cssHeight = 500
WebUI.setViewPortSize(cssWidth, cssHeight)
println "css width: " + cssWidth
println "css height: " + cssHeight
String url = 'http://demoaut.katalon.com/'
println url
WebUI.navigateToUrl(url)
// print the Device Pixel Ratio of the runtime environment
String js = '''
const scale = window.devicePixelRatio;
console.log(scale)
return scale
'''
Object result = WebUI.executeJavaScript(js, null)
Double devicePixelRatio = (Double)result
println "devicePixelRatio=" + devicePixelRatio
// take a screenshot
WebUI.takeFullPageScreenshot("screenshot.png")
WebUI.closeBrowser()
// print the screen width and height of the PNG image on disk
File img = new File("screenshot.png")
BufferedImage bi = ImageIO.read(img)
println "screen width: " + bi.getWidth()
println "screen height: " + bi.getHeight()
When I ran this, I saw:
2023-06-16 07:06:47.796 INFO c.k.katalon.core.main.TestCaseExecutor - --------------------
2023-06-16 07:06:47.806 INFO c.k.katalon.core.main.TestCaseExecutor - START Test Cases/New Test Case
2023-06-16 07:06:49.512 INFO c.k.k.core.webui.driver.DriverFactory - Starting 'Chrome' driver
...
2023-06-16 07:07:05.087 INFO c.k.k.core.webui.driver.DriverFactory - sessionId = c344a3f61fc4344195b491e8f1c1131f
2023-06-16 07:07:05.182 INFO c.k.k.core.webui.driver.DriverFactory - browser = Chrome 114.0.0.0
2023-06-16 07:07:05.185 INFO c.k.k.core.webui.driver.DriverFactory - platform = Mac OS X
2023-06-16 07:07:05.209 INFO c.k.k.core.webui.driver.DriverFactory - seleniumVersion = 3.141.59
2023-06-16 07:07:05.219 INFO c.k.k.core.webui.driver.DriverFactory - proxyInformation = ProxyInformation { proxyOption=NO_PROXY, proxyServerType=HTTP, username=, password=********, proxyServerAddress=, proxyServerPort=0, executionList="", isApplyToDesiredCapabilities=true }
css width: 1400
css height: 500
http://demoaut.katalon.com/
devicePixelRatio=2.0
2023-06-16 07:07:14.906 INFO c.kms.katalon.core.webui.util.FileUtil - 0 hidden object(s)
screen width: 2800
screen height: 1946
2023-06-16 07:07:19.076 INFO c.k.katalon.core.main.TestCaseExecutor - END Test Cases/New Test Case
Please note that I specified the cssWidth to be 1400 pixels and the created PNG file had width of 2800; the DPR of my machine is 2.0.
cssWidth (1400) * DPR (2.0) makes the width of PNG (2800). This looks fine to me.
@nghi.hua can you help?
Can you give more details about this by giving an example based on your assumption?
You wrote you are using WebUI.takeFullPageScreenshotAsCheckpoint keyword, which will produce a “full height” screenshot image.
Previously you posted another post: How can I ignore page height for the visual testing to match with existing baseline
where you asked a question
When running my visual testing all my screenshots go to U (Unresolved) because of different page height.
To me, it seems that your target web page changes its full height dynamically. At one time the screenshot image has the height of, for instance, 10082px; and at another time the height changes to 9882px; and at another timing the height could be 10180px. Is it the case?
If 2 PNG files with different height are given, any image comparison library would determine that the 2 files are different. — Are you unhappy with this behavior? What do you what them to behave otherwise?
Are you aware what is the reason why your target web page dynamically changes its height? There might be some HTML element(s) with movable height, which result the full-page height to change. Are you aware which element(s) are the bad factors? I think, you should analyse your target web page in more detail. If you have found out the reason, then you might realize some way to workaround your difficulty for yourself.
I guess that the Katalon’s Visual Testing feature (of Full Page Screenshots) implicitly assumes that the target web page has static width and static height; only the content pixels are to change.
If you can not stop the target web page to change its full-page height dynamically, I think that you should not apply the “Visual Testing” facility to it at all.
If you use WebUI.takeScreenshotAsCheckpoint keyword or WebUI.takeElementSreenshotAsCheckpoint keyword instead of WebUI.takeFullPageScreenshotAsCheckpoit keyword, then you can effectively ignore the moving height of the full page screenshot as you desired.
Hi, the pages can have different height depending on the content changes (adding a new component or if component was removed, or even when the text within an existing component was updated which expands/shrinks the component height). I would like to be able to still compare the two images instead of having them treated as “Missing” (for baseline image) and “New” (for the screenshot captured during execution).
This is the same page and the only difference is the height of the image (height of the page)
@kazurayam FYI
It seems to me that you are unhappy about the Katalon’s “Visual Testing” because it categorise your target web page, which tends to change its height, as U (unresolved). I don’t have any opinion about this design.
Do you want the Katalon’s “Visual Testing” to identify a pair of images just by id (name) while ignoring their width and height, and just compare the 2?
When 2 PNG images have different height, the comparison library will tell “these are different”. You would accept that.
Katalon’s “Visual Testing” refuses comparing the 2 images with the same id but with different height. You don’t like it. … Am I right?
If I am right, I think it is a matter of design. Katalon team took the current design where the software expects the width&height of the 2 images should be the same; it refuses coparing 2 images with different width&height. — I personally think that this strictness is unnecessary; if I were to design it, I would not take that approach.
Yes, you are right! this is exactly my point! Thanks for the details and getting interested in my topic.
I wish this restriction can be omitted. This would solve the issue for me.
For example if anything changes in the footer (impact for all pages) while the footer container becomes longer/shorter, then I will have to review manually myself x2 screenshots! (I have >200 pages right now) so it will be >400 pages to review (comparing manually myself “baseline-missing no image” and “snapshot from the new execution”
I don’t know the Katalon’s Visual Testing feature. I have never used it. I just guess what it is from your descriptions above and a glance over the doc.
You wrote, you have more than 200 pages to apply the Katalon’s “Visual Testing”. Do you have to do operation to register the baseline images one by one in GUI, more than 200 times? In other words, the “baseline” is defined by the unit of individual image file?
If I am right, I think that Katalon’s “Visual Testing” doesn’t scale. You can apply it to a handful web pages (e.g, less than 5 pages). But applying Katalon’s “Visual Testing” to 200 pages will be a never-ending nightmare.
I guess you have just a few number of test cases, for instance 5 test cases, which produces 200 image files as sum. A single Test Case produces 40 PNG files as average.
I suppose that Katalon team had an alternative design option. They could have designed the product to allow switching sets of 40 baseline images by a single Test Case at a time. In other words, the “baseline” could have been defined as a group of image files, grouped by the timing of screenshooting action. With this design is given, you are supposed to do “switch baselines” operation only 5 times. With this design is given and if you could manage to group the 200 images as one, then you would be able to switch the baseline by a singe operation. … But actually Katalon is not designed as such.
Any comment? If I wrote any misunderstandings about the product, please correct me.
Please advise @anna.kotliarevskiy if she could expect some tips&tricks from Katalon team so that Anna can continue using the Katalon’s Visual Testing.
I have no more idea about this. Let’s Katalon team to think about it. → @vu.tran
Let me suspend judgement how significant the burden of required operations (200, 400 times) is.
I think that users should be able to renew the baselines as soon as possible when the targeted system has been changed. He/she should be able to repeat the renewal as often as required. Therefore the software should make it very easy to renew the baselines (200 png images). The baseline images should not be a historical archive. The baseline images should be fresh enough to check the current system against. If you don’t renew the baselines appropriately and timely enough, any visual comparison tool will become useless.
If you can renew the baselines for all 200 pages by just a few operation (hopefully zero, at most 1 time), I suppose you would say “yes, i can renew the baselines frequently”, wouldn’t you?
There are two stages in Katalon Visual Testing:
How does Katalon Visual Testing work?
When the “Take Full Page Screenshot as Checkpoint” keyword is used, the returned resolution of one captured image could be different among each test case execution because of different heights. And when the resolution is different, Katalon Visual Testing cannot identify the new version of the image and trigger the comparing process with the its baseline / original version.
To summarize, Katalon Visual Testing is impossible to compare images by using only their names and ignoring the resolutions (W x H). And my recommendation is to use the “Take Screenshot as Checkpoint” instead of using the “Take Full Screenshot as Checkpoint” keyword.