Why katalon is not showing a failed test case when screenshot results are different?

Product Forums Katalon TestOps
, ,
1

In the below code, i took a screenshot of a pop up that appears when a button is clicked.
This is my code:

import static com.kms.katalon.core.checkpoint.CheckpointFactory.findCheckpoint

import static com.kms.katalon.core.testcase.TestCaseFactory.findTestCase

import static com.kms.katalon.core.testdata.TestDataFactory.findTestData

import static com.kms.katalon.core.testobject.ObjectRepository.findTestObject

import static com.kms.katalon.core.testobject.ObjectRepository.findWindowsObject

import com.kms.katalon.core.checkpoint.Checkpoint as Checkpoint

import com.kms.katalon.core.cucumber.keyword.CucumberBuiltinKeywords as CucumberKW

import com.kms.katalon.core.mobile.keyword.MobileBuiltInKeywords as Mobile

import com.kms.katalon.core.model.FailureHandling as FailureHandling

import com.kms.katalon.core.testcase.TestCase as TestCase

import com.kms.katalon.core.testdata.TestData as TestData

import com.kms.katalon.core.testng.keyword.TestNGBuiltinKeywords as TestNGKW

import com.kms.katalon.core.testobject.TestObject as TestObject

import com.kms.katalon.core.webservice.keyword.WSBuiltInKeywords as WS

import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI

import com.kms.katalon.core.windows.keyword.WindowsBuiltinKeywords as Windows

import internal.GlobalVariable as GlobalVariable

import org.openqa.selenium.Keys as Keys

WebUI.openBrowser('')

WebUI.setViewPortSize(800,600)

WebUI.navigateToUrl('xxxx')

WebUI.setText(findTestObject('Object Repository/Page_Sign in/input_Email Address_Email Address'), 'x@gmail.com')

WebUI.setEncryptedText(findTestObject('Object Repository/Page_Sign in/input_Forgot your password_Password'), 'xxx')

WebUI.sendKeys(findTestObject('Object Repository/Page_Sign in/input_Forgot your password_Password'), Keys.chord(Keys.ENTER))

WebUI.click(findTestObject('Object Repository/Page_xLicense Details/button_'))

WebUI.click(findTestObject('Object Repository/Page_xAI License Details/button_Buy More Seats'))

WebUI.takeScreenshotAsCheckpoint('new_screenshot')

The first time this test case was executed it gave pop up A.
The next time it was executed it gave another pop up B - as i changed the name and email address that leads to a different pop up in the same test case.

On katalon testops , i can see the two different screenshots on different test runs.
But , why is it marking the second test run as passed , although it’s different?

image
image2420×1002 230 KB

image
image2398×488 36.8 KB

1 Like
2

Hi there,

Thank you very much for your topic. Please note that it may take a little while before a member of our community or from Katalon team responds to you.

Thanks!

3

@nghi.hua @kazurayam if you guys can help please

4

I don’t know TestOps at all. I have no idea.

1 Like
5

@emine_akturk @kevin.mcandrew1 if you guys can help please

6

@productionamena could you click on the ID (12) in the Visual Testing page (the second screenshot in this post) to drill down into the visual comparisons, and show us one of the comparisons please (you may have to change the Status dropdown from ‘Unresolved’ to ‘Passed’, since no visual differences appear to have been picked up)

7

image
image2326×1190 146 KB

I read in the documentation that the checkpoints concept is not available for the free license. Possible this is the reason why screenshots are not showing here or something?
image
image1738×546 54.9 KB

I can see the screenshots taken in the test case only from here:
image
image2392×1208 341 KB

Would be thankful if you could explain how should I apply comparison if this is not possible as a free license user…

8

I’m unsure about the licensing aspect, but I do believe that you will need to run a test suite (that includes your test case) before images are created and compared? That would be my next suggestion, as from memory, I think that just running a test case on its own may not generate the images?

9

Yess I did so the above results are from running the test suite

10

In your first screenshot above, you haven’t changed the Status dropdown from Unresolved to Passed, but I’m assuming both checkpoint images are the same otherwise they wouldn’t have matched (but I’m guessing that seeing this visually confirmed could help?)

I don’t know about the checkpoints concept being unavailable for the free license - you probably need someone from Katalon to comment on this.