WebCrawler to Capture Objects

[gengland]

Curious of Colorado here… Is there a way to automatically capture every object in a website and place it into the Object Repository?

I’ve created a script which crawls and captures all the objects and outputs them to CSV file (unique object names and short xpaths), but can’t find a way of importing it into the OR.

I tried creating a .rs file in the Object Repository folder, but wasn’t sure what to put in the elementGuidId tag of the XML. I tried leaving it blank, and then (after saving the file) refreshed the Tests Explorer window. While the object appeared, it wouldn’t open, and furthermore it then prevented me from opening other objects in the OR, so I concluded this was not the direction I should be heading!

I then wondered if the Object Spy had a way of at least capturing all the objects on a single page, but it looks like it’s only possible to add them one by one.

Is there another solution to doing this that I’ve not thought of perhaps?
Thanks!

[kazurayam]

What is GUID(=UUID)? See Wikipedia UUID at DuckDuckGo

How to create a GUID by code? See https://www.baeldung.com/java-uuid

Any example of creating a file in the “Object Repository” folder with a valid GUID? — See the following Test Case script:

import java.nio.file.Files
import java.nio.file.Path
import java.nio.file.Paths

import com.kms.katalon.core.configuration.RunConfiguration

Path projectDir = Paths.get(RunConfiguration.getProjectDir())
Path mockFile = projectDir.resolve('Object Repository').resolve('Web Browser/btn_MakeAppointoment_mock.rs')
Files.createDirectories(mockFile.getParent())

// You can generate a UUID (= GUID: Global Unique ID) easily
def guid = UUID.randomUUID().toString()

println "guid=${guid}"

def btn_MakeAppointment_mock_template = """<?xml version="1.0" encoding="UTF-8"?>
<WebElementEntity>
   <description>kazurayam created this by code</description>
   <name>btn_MakeAppointmen_mock</name>
   <tag></tag>
   <elementGuidId>${guid}</elementGuidId>
   <useRalativeImagePath>false</useRalativeImagePath>
   <webElementProperties>
      <isSelected>true</isSelected>
      <matchCondition>equals</matchCondition>
      <name>id</name>
      <type>Main</type>
      <value>btn-make-appointment</value>
   </webElementProperties>
</WebElementEntity>
"""
	
mockFile.toFile().text = btn_MakeAppointment_mock_template.toString()

assert Files.exists(mockFile)

When I ran this code, it could make a file. I stopped Katalon Studio and restarted it so that it recognizes the newly created file.

I could see the icon of btn_MakeAppointmen_mock which was newly created

But Katalon Studio GUI did not like it. In the Test Explorer pane, I double-clicked the btn_MakeAppointment_mock icon, Katalon Studio did not open it. I am sure that Katalon Studio GUI dislikes the GUID value I randomly generated.

What sort of GUID value Katalon Studio likes? — The Katalon Studio’s source code of that part is not disclosed. So no one but the Katalon developer team can know it.

The Spy and Recorder tools do not let you generate entries in the Object Repository programatically.

So, as @gengland concluded, it is impossible to create an entity inside the “Object Repository” folder on disk by your code.

[kazurayam]

Don’t you know, you can create an instance of com.kms.katalon.core.testobject.TestObject runtime in memory by code and use it? See

Creation of Test Object in Object Repository in Runtime

Hi Indurajan I use this: /** * Construct a Katalon-compatible TestObject in memory. * @param css (String) The CSS selector used to find the target element. * @return (TestObject) The constructed TestObject. */ static TestObject makeTO(String css) { TestObject to = new TestObject() to.addProperty("css", ConditionType.EQUALS, css) return to }

Using xpath,

static TestObject makeTestObject(String id, String xpath) {
    TestObject to = new TestObject(id)
    to.addProperty("xpath", ConditionType.EQUALS, xpath)
    return to
}

If you use this technique, you no longer need to retrieve entities from the “Object Repository” managed by Katalon Studio. Your CSV file can be a custom testobject repository for you. You can instantiate TestObjects as defined in your CSV file directly.

Now you have to invent a set of Groovy codes that forms your custom testobject repository on top of your CSV file. That could be a fun and easy for you @gengland.

[gengland]

Thanks @kazurayam , I did think about this as a solution, but it’s to augment existing projects with ORs, so it might get confusing to have two different approaches, plus since our developers have only just gotten familiar with Katalon and the OR, I’d rather not make such a change at this time.

I appreciate your comments though, thanks!

[kazurayam]

@kazurayam

What sort of GUID value Katalon Studio likes? — The Katalon Studio’s source code of that part is not disclosed. So no one but the Katalon developer team can know it.

@gengland

Are your a paying Katalon user? If so, through the official support channel, why not you ask Katalon to disclose what sort of GUID value Katalon Studio likes?

As https://www.baeldung.com/java-uuid#uuid-class-java shows, java.util.UUID class has a few variations of constructor. For example,

UUID uuid = new UUID(mostSignificant64Bits, leastSignificant64Bits);

UUID uuid = UUID.nameUUIDFromBytes(bytes);

UUID uuid = UUID.fromString(uuidHexDigitString);

UUID uuid = UUID.randomUUID();

Which constructor should you use? That would be the question from you to Katalon ---- simple enough, easy to reply.

If they agree to disclose that little point, you would be able to make full use of your CSV database.

[gengland]

Great info, thanks @kazurayam . We are using full licenses, so I’ll raise a ticket for this next week!

[kazurayam]

@gengland

Please ask Katalon if they agree with disclosing their reply — what sort of GUID katalon likes.

[gengland]

Sorry it took me a while to get to the bottom of this. Yes, Katalon support has shared the following…

Katalon Studio’s Object’s GuidId have 32-character hexadecimal string, divided into 8-4-4-4-12 format like: 556f85ec-f771-4bed-a36f-f3caac4e36ec

Therefore I suggest to use the below constructor variation for generating the GuidId that can be accepted by Katalon Studio:

UUID uuid = UUID.randomUUID();

[kazurayam]

@gengland

UUID uuid = UUID.randomUUID();

I doubt the reply from Katalon support.

I already tried the “UUID.randomUUID()” which did not work.

@gengland

You should ask them to disclose the program source code of Katalon Studio GUI. You need to read the code which determines the value of GUID of Test Objects.

[gengland]

Ah shoot… I already accepted the response and closed the ticket, sorry! I decided it’s not worth going down this path, considering the risks of going in through Katalon’s back door.

[kazurayam]

@gengland

considering the risks of going in through Katalon’s back door.

??? Are you Ethan Hunt?

No risks you should take. You should create a new ticket saying “the reply from Katalon Support did not work”

[gengland]

LOL, if only I was that young and good looking! Yeah I tried to update the ticket just now, but it says I have to create a new ticket. It’s ok, I’m going to let it go, but for anyone else reading this post, best of luck!

[kazurayam]

@gengland

It’s ok, I’m going to let it go

Have you abandoned your idea? — importing Test Objects from a CSV file into the Object Repository in Katalon Studio?

[gengland]

Probably, but you’ll probably want to play around with the concept yourself, as I’m not sure whether or not I’m going to continue effort on this PoC.

[gengland]

Not sure tbh, I might do a little more on it at some point. Here’s an interesting update though… I used GUID Generator to generate GUIDs, and they seem to work. I created a new rs file, updating the data from an existing object, inserted a GUID from this link, saved and switched back to Katalon. At first it wasn’t visible, but a refresh in the Tests Explorer window made it appear and I could open it.

[kazurayam]

@gengland

Thank you for your information.

I visited GUID Generator. For me, it suggested a GUID value:

7ea58726-b877-49d8-9d1e-3274ac78a18a

So I modified my code as follows:

import java.nio.file.Files
import java.nio.file.Path
import java.nio.file.Paths

import com.kms.katalon.core.configuration.RunConfiguration

Path generateTestObject(String testObjectId, String guid) { 
	Path projectDir = Paths.get(RunConfiguration.getProjectDir())
	Path f = projectDir.resolve('Object Repository').resolve("${testObjectId}.rs")
	Files.createDirectories(f.getParent())
	String name = f.getFileName().toString().replace('.rs', '')
	println "guid=${guid}"
	def template = """<?xml version="1.0" encoding="UTF-8"?>
<WebElementEntity>
   <description>kazurayam created this by code</description>
   <name>${name}</name>
   <tag></tag>
   <elementGuidId>${guid}</elementGuidId>
   <useRalativeImagePath>false</useRalativeImagePath>
   <webElementProperties>
      <isSelected>true</isSelected>
      <matchCondition>equals</matchCondition>
      <name>id</name>
      <type>Main</type>
      <value>btn-make-appointment</value>
   </webElementProperties>
</WebElementEntity>
"""
	f.toFile().text = template.toString()
	return f
}

def mock1 = generateTestObject('Web Browser/btn_MakeAppointment_mock1', UUID.randomUUID().toString())
assert Files.exists(mock1)

def mock2 = generateTestObject('Web Browser/btn_MakeAppointment_mock2', '7ea58726-b877-49d8-9d1e-3274ac78a18a') // https://guidgenerator.app/ gave me this GUID value
assert Files.exists(mock2)

This code generated 2 Test Objects.

I could successfully open both test objects.

I used Katalon Studio v10.0.0.

This experiment proves that the reply from Katalon support was right; UUID.randomUUID() will be fine.

I was wrong in saying:

@kazurayam

But Katalon Studio GUI did not like it. In the Test Explorer pane, I double-clicked the btn_MakeAppointment_mock icon, Katalon Studio did not open it. I am sure that Katalon Studio GUI dislikes the GUID value I randomly generated.

I don’t know how I made a mistake. But I wouldn’t mind it anymore.

[gengland]

This is brilliant, thanks! I’ll probably pick this up again next week, just for giggles and hoots, since we’re so close to a complete solution.