Looking for duplicate links on a page where values change daily


I am currently trying to figure out how script the process of finding duplicate links on a webpage. I started with using suggestions from Looking for duplicate links in a page - #4 by B_L, but I’ve run into some difficulties. Primarily, because the page links change daily, I can’t just do a search for the same text or URL every time.

The code I have so far is:

import static com.kms.katalon.core.checkpoint.CheckpointFactory.findCheckpoint
import static com.kms.katalon.core.testcase.TestCaseFactory.findTestCase
import static com.kms.katalon.core.testdata.TestDataFactory.findTestData
import static com.kms.katalon.core.testobject.ObjectRepository.findTestObject
import com.kms.katalon.core.checkpoint.Checkpoint as Checkpoint
import com.kms.katalon.core.cucumber.keyword.CucumberBuiltinKeywords as CucumberKW
import com.kms.katalon.core.mobile.keyword.MobileBuiltInKeywords as Mobile
import com.kms.katalon.core.model.FailureHandling as FailureHandling
import com.kms.katalon.core.testcase.TestCase as TestCase
import com.kms.katalon.core.testdata.TestData as TestData
import com.kms.katalon.core.testobject.TestObject as TestObject
import com.kms.katalon.core.webservice.keyword.WSBuiltInKeywords as WS
import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI
import com.kms.katalon.core.logging.KeywordLogger as KeywordLogger
import internal.GlobalVariable as GlobalVariable
import org.openqa.selenium.WebElement as WebElement
import com.kms.katalon.core.webui.common.WebUiCommonHelper as WebUiCommonHelper



WebUI.click(findTestObject(‘links object folder/linknamegoeshere’))


WebUI.click(findTestObject(‘links object folder/pageobjectgoeshere’), FailureHandling.CONTINUE_ON_FAILURE)

List links = WebUiCommonHelper.findWebElements(findTestObject(‘links’), 30)

int linkCount = 0

int comparisonCount = 0

KeywordLogger log = new KeywordLogger()

for (WebElement link : links) {
String text = link.getText()

for (WebElement k : links) {
    if (k != link) {
        if (k.getText().equals(text)) {




assert linkCount <= 1


Some of the issues I’m running into:
-the code compares the links to all the other links on the page, but won’t show where the duplicates occur.
-the list of links wasn’t created as a table but rather as a stack of divs, I tried looking at some documentation on counting values in a table but it wasn’t effective.

Is there anything else I can do? The page is pulling links in from another source and I’m trying to ensure that it isn’t accidentally pulling data twice (which would cause a link to display twice in this case).

Thank you!

How about a little Java magic?

List<WebElement> links = WebUiCommonHelper.findWebElements(findTestObject('links'), 30);
List<String> linkTexts = new ArrayList<String>();
for(WebElement link : links) {
Set<String> uniqueLinkTexts = new HashSet<String>(linkTexts);
if(linkTexts.size() != uniqueLinkTexts.size()) {
    // mark as failure, print out list and set to check diffs, etc.

The idea here is that while a List of objects may contain duplicates, a Set cannot contain duplicates by definition. So by dumping the List of link texts into a Set, then comparing the size of the List against the size of the Set, you can easily assert whether or not there were duplicates.


I tried that and it appears that the script isn’t capturing any of the code I have on the page. Could I have my test object set up incorrectly? I have the object’s properties set up as “xpath equals //a”.

Can you share a screenshot of your Test Object configuration, as well as the error you are getting when you run it?

The error I’m getting is that the test is passing when there are still duplicate links on the page.

Then what was this bit about?

I’m just curious - what does “duplicate link” mean for you? Is it the same element text? Because - IMO - you should compare href attributes.


replace with


This is the way how to identify duplicates:

List<WebElement> links = WebUiCommonHelper.findWebElements(whatever, 30)
List<String> linkTexts = new ArrayList<String>()
for(WebElement link : links) {

Set uniqueLinkTexts = new HashSet()
boolean isUnique
for(String s in linkTexts) {
    // Set.add method returns false if the element is already present in the set
	isUnique = uniqueLinkTexts.add(s)
	if(!isUnique) {
		KeywordUtil.markWarning("Duplicate found. Link: " + s)