WebCrawler to Capture Objects

Curious of Colorado here… Is there a way to automatically capture every object in a website and place it into the Object Repository?

I’ve created a script which crawls and captures all the objects and outputs them to CSV file (unique object names and short xpaths), but can’t find a way of importing it into the OR.

I tried creating a .rs file in the Object Repository folder, but wasn’t sure what to put in the elementGuidId tag of the XML. I tried leaving it blank, and then (after saving the file) refreshed the Tests Explorer window. While the object appeared, it wouldn’t open, and furthermore it then prevented me from opening other objects in the OR, so I concluded this was not the direction I should be heading!

I then wondered if the Object Spy had a way of at least capturing all the objects on a single page, but it looks like it’s only possible to add them one by one.

Is there another solution to doing this that I’ve not thought of perhaps?
Thanks!

2 Likes

What is GUID(=UUID)? See Wikipedia UUID at DuckDuckGo

How to create a GUID by code? See https://www.baeldung.com/java-uuid

Any example of creating a file in the “Object Repository” folder with a valid GUID? — See the following Test Case script:

import java.nio.file.Files
import java.nio.file.Path
import java.nio.file.Paths

import com.kms.katalon.core.configuration.RunConfiguration

Path projectDir = Paths.get(RunConfiguration.getProjectDir())
Path mockFile = projectDir.resolve('Object Repository').resolve('Web Browser/btn_MakeAppointoment_mock.rs')
Files.createDirectories(mockFile.getParent())

// You can generate a UUID (= GUID: Global Unique ID) easily
def guid = UUID.randomUUID().toString()

println "guid=${guid}"

def btn_MakeAppointment_mock_template = """<?xml version="1.0" encoding="UTF-8"?>
<WebElementEntity>
   <description>kazurayam created this by code</description>
   <name>btn_MakeAppointmen_mock</name>
   <tag></tag>
   <elementGuidId>${guid}</elementGuidId>
   <useRalativeImagePath>false</useRalativeImagePath>
   <webElementProperties>
      <isSelected>true</isSelected>
      <matchCondition>equals</matchCondition>
      <name>id</name>
      <type>Main</type>
      <value>btn-make-appointment</value>
   </webElementProperties>
</WebElementEntity>
"""
	
mockFile.toFile().text = btn_MakeAppointment_mock_template.toString()

assert Files.exists(mockFile)

When I ran this code, it could make a file. I stopped Katalon Studio and restarted it so that it recognizes the newly created file.

I could see the icon of btn_MakeAppointmen_mock which was newly created

But Katalon Studio GUI did not like it. In the Test Explorer pane, I double-clicked the btn_MakeAppointment_mock icon, Katalon Studio did not open it. I am sure that Katalon Studio GUI dislikes the GUID value I randomly generated.

What sort of GUID value Katalon Studio likes? — The Katalon Studio’s source code of that part is not disclosed. So no one but the Katalon developer team can know it.

The Spy and Recorder tools do not let you generate entries in the Object Repository programatically.

So, as @gengland concluded, it is impossible to create an entity inside the “Object Repository” folder on disk by your code.

1 Like

Don’t you know, you can create an instance of com.kms.katalon.core.testobject.TestObject runtime in memory by code and use it? See

Using xpath,

static TestObject makeTestObject(String id, String xpath) {
    TestObject to = new TestObject(id)
    to.addProperty("xpath", ConditionType.EQUALS, xpath)
    return to
}

If you use this technique, you no longer need to retrieve entities from the “Object Repository” managed by Katalon Studio. Your CSV file can be a custom testobject repository for you. You can instantiate TestObjects as defined in your CSV file directly.

Now you have to invent a set of Groovy codes that forms your custom testobject repository on top of your CSV file. That could be a fun and easy for you @gengland.

1 Like

Thanks @kazurayam , I did think about this as a solution, but it’s to augment existing projects with ORs, so it might get confusing to have two different approaches, plus since our developers have only just gotten familiar with Katalon and the OR, I’d rather not make such a change at this time.

I appreciate your comments though, thanks!

@gengland

Are your a paying Katalon user? If so, through the official support channel, why not you ask Katalon to disclose what sort of GUID value Katalon Studio likes?

As https://www.baeldung.com/java-uuid#uuid-class-java shows, java.util.UUID class has a few variations of constructor. For example,

UUID uuid = new UUID(mostSignificant64Bits, leastSignificant64Bits);
UUID uuid = UUID.nameUUIDFromBytes(bytes);
UUID uuid = UUID.fromString(uuidHexDigitString);
UUID uuid = UUID.randomUUID();

Which constructor should you use? That would be the question from you to Katalon ---- simple enough, easy to reply.

If they agree to disclose that little point, you would be able to make full use of your CSV database.

1 Like

Great info, thanks @kazurayam . We are using full licenses, so I’ll raise a ticket for this next week!

@gengland

Please ask Katalon if they agree with disclosing their reply — what sort of GUID katalon likes.