Full screenshots usually have big size and you are having more than 200 screenshots, so it takes time to analyze each pair of images out of over 200 images.
Could you please briefly share with me the purpose of the presentation and the way you are preparing for it? With this insight, I might think of solution or recommendation to help you.
Thank you for your suggestion to add a “Search Images” function to Katalon Visual Testing. While we don’t currently have plans to implement this feature, we appreciate your feedback and will consider it for future updates.
Hello,
so the purpose of the presentation is to share with the global team how I am using Katalon as I have build a combination of the functional smoke tests and visual tests in one for all pages which I am using for the pre-release smoke tests.
Now to give you some background, previously I had another machine and it was more or less smooth… (same Width, not always same height for all my screenshots)
After switching to a new machine with a different DPR I had to re-accept all the screenshots (different DPR = different height and width)… and here is why I initiated this current post and as you can see I am not able to now get all my screenshots analyzed anymore…
I also have a new observation about some screenshots showing the top of the page at the bottom of it.
Here is an example:
Top so the screens and the entire page is looking good until you scroll to the bottom of it, where the top component reappears again
I have this happening for a few pages every run and cannot accept those screenshots to save in my baseline. the live page looks good.
Ahhh, you surprised me. You switched the machine …
Now, you have 2 options:
- you want to continue working on the new machine
- you want to go back to the previous machine
If you want to continue working on the new machine, you have to throw the current baseline images away because your new machine has a different Device Pixel Ratio. You have to rebuild the set of baseline images from scratch.
If you do not like to go through the burden of rebuilding the baseline images 200 times, then you should go back to the previous machine. On that machine, you would not have different image width, do you?
I do not think you can abuse Katalon’s Visual Testing : it refuses comparing 2 images with different pixel width because it assumes that DPR stays the same.
I just feel sorry for @anna.kotliarevskiy because Katalon’s Visual Testing requires her to go though the tedious operations of re-rebuilding the baseline images of 200.
As for the issue “some screenshots showing the top of the page at the bottom of it”, I made a Test Case on my machine to reproduce your case.
import java.awt.image.BufferedImage
import javax.imageio.ImageIO
import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI
String url = 'https://www.adm.com/en-us/insights-and-innovation/formulation-challenges/sugar-reduction/'
String fileName = "sugarReduction.png"
File png = new File("./" + fileName)
WebUI.openBrowser('')
WebUI.setViewPortSize(1200, 800)
WebUI.navigateToUrl(url)
WebUI.takeFullPageScreenshot(png.toString())
WebUI.closeBrowser()
BufferedImage bi = ImageIO.read(png)
println "png width=" + bi.getWidth()
println "png height=" + bi.getHeight()
This script created a PNG image:
This PNG image has width=2400, height=15482 pixel. Its size = 16.7mega bytes (how huge!).
The DPR of my machine is 2.0.
I find nothing like what you reported in this PNG image. So I could not reproduce your case. I have no more idea about your case.
so my original expectation was that if the browser window setup (1440px) is the same it should not matter what machine it is. But you explained to me about the DPR and I got this and I did “throw” away all my 1424px wide screenshots from the baseline. My new baseline images are now 3570px wide…
But as I said above, my images are now stuck in “analyzing” status
I see your image is looking good. i am not sure why mine has an issue. If you accept the cookie will it be the same?
Can you please do 1440px wide, as per your test it is a different layout (1200px) - top component has a different layout
My advice is, you should take out from yor demo the Visual testing part and focus on any other functionalities.
As proven, this feature is prone to error, if i will be in your shoes i will just skip it for a presentation
I created another demo project for investigation, which gave me an interesting insight.
I created a Test Case “TC1” as follows:
import java.awt.image.BufferedImage
import javax.imageio.ImageIO
import org.openqa.selenium.WebDriver
import com.kazurayam.ashotwrapper.AShotWrapper
import com.kms.katalon.core.webui.driver.DriverFactory
import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI
String url = 'https://www.adm.com/en-us/insights-and-innovation/formulation-challenges/sugar-reduction/'
WebUI.openBrowser('')
WebUI.setViewPortSize(1440, 800)
WebUI.navigateToUrl(url)
WebDriver driver = DriverFactory.getWebDriver()
File png1 = new File("./sugarReduction_using_builtin_keyword.png")
WebUI.takeFullPageScreenshot(png1.toString())
BufferedImage bi1 = ImageIO.read(png1)
println "png1: width=" + bi1.getWidth() + ", height=" + bi1.getHeight() + ", size=" + png1.length()
File png2 = new File("./sugarReduction_by_ashot.png")
AShotWrapper.saveEntirePageImage(driver, png2);
BufferedImage bi2 = ImageIO.read(png2)
println "png2: width=" + bi2.getWidth() + ", height=" + bi2.getHeight() + ", size=" + png2.length()
File jpg3 = new File("./sugarReduction_by_ashot_as_jpeg.jpg")
AShotWrapper.saveEntirePageImageAsJpeg(driver, jpg3, 0.8f);
BufferedImage bi3 = ImageIO.read(jpg3)
println "jpg3: width=" + bi3.getWidth() + ", height=" + bi3.getHeight() + ", size=" + jpg3.length()
WebUI.closeBrowser()
This code requires 2 new jar files downloaded into the Drivers folder of the project.
You can find the documentation of the AShotWrapper library (I am the author) here:
When I ran the above Test Case, I got the following message in the console:
png1: width=2880, height=13772, size=11933726
png2: width=1440, height=6886, size=3750810
jpg3: width=1440, height=6886, size=896438
I specified the width of the view port to be 1440 pixels.
The Device Pixel Ration of my machine (Mac Book Air, 2019) is 2.0f.
This tells a fact:
- The Katalon built-in keyword
takeFullPageScreenshot()created a PNG file of length=11.93MB - My custom class
AShotWrapper.saveEntirePageImage()created a PNG file of length=3.75MB - My custom class
AShotWrapper.saveEntirePageImageAsJpeg()created a JPEG file of length=0.89MB
Please find here the differences of file length.
I attached the 3 images above. You would find that all 3 images look just similar. To human eyes, they are the same. But to the software, these 3 files are completely different.
The larger is an input image, it requires more of computing resources (CPU, memory, time). Especially, comparing 2 images by pixel-by-pixel basis consumes a lot of memory. The comparison logic creates a in-memory buffer of different pixels. The larger the input image pixels is and the greater the difference of 2 images are, the larger the buffer becomes.
I guess, that TestOps got some unrecoverable error on the server-side when it tried to process the images you posted. Most probably due to “OutOfMemoryError”. The error messages should be recorded in the log file of TestOps server. Unfortunately TestOps provides no way for users to look into the logs. We can not investigate more.
To sum up, I think that Katalon’s Visual Testing feature is not likely to satisfy you immediately. There seems to be no way for you to overcome the difficulties you are facing at the moment.
I have been here in the Katalon’s User Forum for years. I am sure that you, @anna.kotliarevskiy, are the very first person who posted a discussion about the TestOps Visual Testing feature with an actual usage experience. Does anybody else use it? I don’t know. The feature shines in their marketing brochures but, I think, it seems to have a lot of rooms to be improved with customers’ feedbacks including @anna.kotliarevskiy.
I guess, this might be the reason why Katalon’s Visual Testing is designed to be strict about the width and the height of 2 images. If either of width and height is different, then it is highly likely the comparison would result a huge set of different pixels, which might cause java.lang.OutOfMemoryError. The OutOfMeroryError is unable for an application code to catch and deal with gently. You can not ignore the OutOfMemoryError. So I guess that the Katalon team wanted to avoid the risk of OOME, so that they designed the product to refuse comparing 2 images with different width & height.
I do not know if it is explained in the document (the width and the height of PNG images must be the same). If not, the doc must be improved.
→ @vu.tran
I think that Katalon team should consider an alternative design. WebUI.takeFullPageScreenshotAsCheckpoint produces a PNG file which is too large to post-process. Katalon should provide another keyword that creates a full page screenshot in JPEG format which would be far smaller. Also TestOps should be able to compare 2 JPEG files, not only PNG.
Visual was main reason for me when I chose to try Katalon. We have other tools for functional testing, I need to talk about visual testing. I hope this feature will be improved…It needs to be improved.
I will try this. Thank you very much
What are you going to try? I think there is no more you can do on the TestOps Visual Testing.
yes, I got it, I hope Katalon team will improve the image sizing on saving to resolve java.lang.OutOfMemoryError. so i can use visual testing.
You can use my AShotWrapper.saveEntirePageImage(driver, png2);. This will create a smaller PNG than WebUI.takeFullPageScreenshotAsCheckpoint().
However, TestOps will never pickup the PNG file created by AShotWrapper. My AShotWrapper has nothing to do with the TestOps.
I don’t think they can do it by your presentation scheduled at June 27.