Chrome: saving an xml file instead of displaying it

Hi,

Thanks again to you and to Kazurayam. For the xml, I have realised that if I have a table with rows and cells, and in one of the cells, as Kazurayam said, there is an “a” that contains an href, the simplest way to get all the information I want is to create as many test objects as I now they are (maximum 31, the maximum number of days of a month), and extract the information from each test object, as done for the FIFA project.

This are examples of Test objects I created:

//div[@id=‘resultDiv’]/table/tbody/tr/td - this will contain the date, that I can extract in the code with WebUI.getText

//div[@id=‘resultDiv’]/table/tbody/tr/td[6]/a - this will contain the url to the xml, that I can extract in the script with WebUI.getAttribute(anchorElement, “href”)

etc.

The only thing I don’t know is if I really have to create a test object per row, or this can be somehow parametrized. I mean, that I have to create a test object for tr (row 1), tr[2] (row 2), tr[3] (row 3)…

Thanks again.

Katalon Studio provides a feature called “Parameterized Test Object”. It will certainly help you.

Have a look at the following official document:
https://docs.katalon.com/katalon-studio/docs/parameterize-webmobile-test-object-properties.html

I am afraid this documentation page is poor. It does not tell you how you should write a “parameterized Test Object”. It only provides a sample code of a Test Case which calls a Test Case with param.

I have updated my demo project:

Let me explain a bit about it.

I created a sample target Web Page

This page looks similar (but not identical) to the one which yolanda shared with us. It has a table with a <tr>
element with <a> element to XML.

I made a Test Object which has

  1. Selection method=Attribute
  2. parameterized string as an XPath expression : //div[@id='resultDiv']/table/tbody/tr[${row}]/td[6]/a. Please find a place holder ${row} is used here

I made a Test Case TC2

Please read the following line where the place holder ${row} in the Test Object is interpolated dynamically.

// look up the href attribute of the XML in the 1st row of the table
String href = 
	WebUI.getAttribute(
		findTestObject('Page_Sample/nth_tr_anchor_to_xml', ['row':'1']),   // interpolate tr[${row}] => tr[1]
		'href')

In the Test Case you can write ['row': '2'], ['row': '3'] and so on if the target HTML has more rows. You will apply these key-value pairs to a single Test Object named 'Page_Sample/nth_tr_anchor_to_xml which has a place holder in selector strings(XPath or CSS).

So, you do not have to create many Test Objects for each table rows.

1 Like

Thanks a lot! I set a loop and it works like a charm. I have another question. Sometimes, the month doesn’t have as many files as days, then I put a control: if the file is not there, break the loop. Even when I put CONTINUE_ON_FAILURE, the test result is failed. How can I avoid this?

Please find below the code snippet.

for (int i=1; i<=last_day;i++){

def anchorElement = findTestObject(‘Object Repository/XML_n/XML_file’,[‘row’: i])

def My_id = findTestObject(‘Object Repository/XML_n/My_id’,[‘row’: i])

def My_date = findTestObject(‘Object Repository/XML_n/My_date’,[‘row’:i])

//if the row doesn’t contain a file, that means that there are no more files for the month and you have to stop the loop

boolean fileExists = WebUI.verifyElementPresent(My_date, 10,FailureHandling.CONTINUE_ON_FAILURE)
println “fileExists=${fileExists}”

if (!fileExists){
break;
}

Possibly you should not invoke WebUI.verifyElementPresent(My_date) at all in order to test if the file is there or not.

Instead, how should you do it? — It would depend on how your target HTML is coded. Please have a deep breath, and think about it.

thanks for the hint. Sometimes one doesn’t think about using common tools. I counted the rows of the table

//Count the number of rows in the table. This, functionally, means count the number of xml files in the month

WebDriver driver = DriverFactory.getWebDriver()

‘To locate table’
WebElement Table = driver.findElement(By.xpath("//div[@id=‘resultDiv’]/table/tbody"))

‘To locate rows of table it will Capture all the rows available in the table’
List rows_table = Table.findElements(By.tagName(‘tr’))

‘To calculate no of rows In table’
int rows_count = rows_table.size()

I have a new issue with this code…Exceptionally, it can happen that a row doesn’t contain a link to an xml file, but the row exists.

Exceptionally, the row contains 2 xml files. They are separated by
.
This means that the cell td[6], can exceptionally contain 0 files or 2 files.

How can I handle these cases, taking into account that I loop over all rows in the page?

thanks in advance

I extended my demo project further. See

The Application Under Test https://kazurayam.github.io/SavingAnXmlFileRatherThanDisplayingIt/ now has 3 rows:


Please note: The 1st row contains 1 link to XML. The 2nd row contains no link to XML. The 3rd row contains 2 links to XML files.

I added a Test Case TC3. This test case scans the target HTML for <a> elements to XML file. I took it into account that a <tr> element may contain zero, 1, 2 or more <a> elements to XML.

WebUI.openBrowser('')
WebUI.navigateToUrl('https://kazurayam.github.io/SavingAnXmlFileRatherThanDisplayingIt/')
WebUI.verifyElementPresent(findTestObject('Page_Sample/div_resultDiv'), 10)

// prepare output directory
Path projectdir = Paths.get(RunConfiguration.getProjectDir())
Path outputdir = projectdir.resolve("tmp")

// grasp <tr> elements
List<WebElement> trElements = WebUI.findWebElements(
	new TestObject().addProperty("xpath", ConditionType.EQUALS, 
							"//div[@id='resultDiv']/table/tbody/tr"), 10)

for (int i = 0; i < trElements.size(); i++) {
	// grasp <a> elements to XML
	List<WebElement> anchorElements = WebUI.findWebElements(
		new TestObject().addProperty("xpath", ConditionType.EQUALS,
						"//div[@id='resultDiv']/table/tbody/tr[${i+1}]/td[6]/a"), 10)
	for (int j = 0; j < anchorElements.size(); j++) {
		String href = anchorElements.get(j).getAttribute("href")
		// determin the file location
		Path outfile = outputdir.resolve("sample(${i},${j}).xml")
		WebUI.comment("(${i},${j}) href=${href} outfile=${outfile.toString()}")
		// download XML and save it into file
		WebUI.callTestCase(findTestCase("modules/DownloadXML"),
					["href":href, "outfile": outfile.toString()],
					FailureHandling.OPTIONAL)
	}
}
WebUI.closeBrowser()

The TC3 calls another test case named modules/DownloadXML:

/**
 * @param href (java.lang.String) URL of XML file to download
 * @param outfile (java.lang.String) path of the file into which the XML is saved
 */
Objects.requireNonNull(href)
Objects.requireNonNull(outfile)

// Create a new GET object using builder
def builder = new RestRequestObjectBuilder()
def requestObject = 
    builder.withRestRequestMethod("GET")
        .withRestUrl(href) // here we specify the URL found in the web site
        .build()

// Send a request
def response = WS.sendRequest(requestObject)

// Verify if the response from the URL returns the 200 status code'
WS.verifyResponseStatusCode(response, 200)

// Get the content string
def content = response.getResponseBodyContent()
Path file = Paths.get(outfile)

// prepare output directory
Files.createDirectories(file.getParent())

// save XML into file
file.toFile().write(content))

By executing TC3, I got a few xml files created in the tmp directory under the project dir.

:tmp [master]$ tree
.
├── sample(0,0).xml
├── sample(2,0).xml
└── sample(2,1).xml

0 directories, 3 files

The mission is accomplished, isn’t it?

1 Like

Thanks a lot, Kazurayam.

Up to now, I didn’t create test objects or call test cases. I think this is a use case that should be integrated in the Katalon documentation. You use all kind of best practices.

I still have 1 question for you, if you allow me:

-In another script, I download the xls files. This is a lot easier, but I haven’t find a way to rename them before download them, as for this, the only thing I do in the project is changing the way the browser to download the files automatically instead of opening a file dialog. Knowing that there is no a href in the html code, but a javascript function that extracts data as Excel, would it be possible to rename the files, or should I think about a post treatment?

Thanks again!

Does the javascript function have parameters? Is the output file path included in the parameters? Or, regardless the way how and before invoking the function, can you specify for the function where to save the Excel data ?

If possible, I would pass the output file path to the javascript function call. How to pass value from a Katalon test case to a javascript function in the web page? You can do it with WebUI.executeJavaScript().

If you have no way to specify output file path, I think your only option is to think about a post treatment. Though I wonder if the treatment could be stable enough. When you invoked the function multiple times, how the file name could be? — sample(1).xls, sample(2).xls, sample(3).xls … these automatic naming is nice for human but troublesome for test scripts.

The best stable way would be, if possible, to change the javascript function so that it accepts the output file path as parameter.

Hi,

The executeJavaScript feature looks interesting. I don’t know about the path, but perhaps the name could be set. In fact, it almost has. This is where the function is called:

image

How do I do the call? I tried but it generates exceptions.

Thanks in advance

Please show your code, and the execution log.

exportData('EXCEL','FINAL', '1', '2019', 'DescisionStatements_2019-1');

When you click the img manually on browser screen, you will get an Excel file will be created. Could you show the full path (file name) to us? I want to find the rule how the parameters to exportData() is mapped to the path.

Or, you should be able to read the javascript source code of the exportData function. Please read the source code and find how the file path is deptermined by the parameters to the function call.

Hi kazurayam,

finally, I won’t be able to change the file where to export, so after downloading it I just added a bit of java code to rename and move the file where I was interested to.

Thanks a lot for all your help and your support!

Hello @kazurayam , I’m working to program my katalon studio script to do this exact thing of downloading xml files, I already have the urls, I do not need to scrap for the object.
However it gives me an error:
2021-08-05 13:59:20.035 ERROR c.k.katalon.core.main.TestCaseExecutor - :x: Test Cases/CORDISxml/CORDIS_xmlTest FAILED.
Reason:
java.lang.IllegalArgumentException: Property ‘http://javax.xml.XMLConstants/property/accessExternalDTD’ is not recognized.

The script is the following:

import static com.kms.katalon.core.testobject.ObjectRepository.findTestObject

import java.nio.file.Files
import java.nio.file.Path
import java.nio.file.Paths

import com.kms.katalon.core.configuration.RunConfiguration
import com.kms.katalon.core.model.FailureHandling as FailureHandling
import com.kms.katalon.core.testobject.RestRequestObjectBuilder
import com.kms.katalon.core.webservice.keyword.WSBuiltInKeywords as WS
import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI

def xmlUrl = "https://cordis.europa.eu/search/en?q=%27*%27&p=1&num=100srt=Relevance:decreasing&format=xml"
println (xmlUrl)

// create a new GET object using builder
def builder = new RestRequestObjectBuilder()
def requestObject = builder
		.withRestRequestMethod("GET")
		.withRestUrl(xmlUrl)
		.build()
		
// send a request
def response = WS.sendRequest(requestObject)

// verify status of req
println "status of req == 200" + WS.verifyResponseStatusCode(response, 200)

// get content string
def content = response.getResponseBodyContent();

// prepare output dir
Path projectdir = Paths.get(RunConfiguration.getProjectDir())
Path outputdir = projectdir.resolve("tmp")
Files.createDirectories(outputdir)
Path file = outputdir.resolve("test.xml")

// save XML into file
file.toFile().write(content) 

The path I want to save my xml is within
C:\Users\Ana\Katalon Studio\Katalon Test Case Crearion\Test Cases\CORDISxml

Help would be much appreciated!

Complete stack trace looks like this:

2021-08-05 14:26:05.580 ERROR c.k.katalon.core.main.TestCaseExecutor   - ❌ Test Cases/CORDISxml/CORDIS_xmlTest FAILED.
Reason:
java.lang.IllegalArgumentException: Property 'http://javax.xml.XMLConstants/property/accessExternalDTD' is not recognized.
	at org.apache.xerces.jaxp.DocumentBuilderFactoryImpl.setAttribute(Unknown Source)
	at com.kms.katalon.util.DocumentBuilderProvider.newBuilderInstance(DocumentBuilderProvider.java:16)
	at com.kms.katalon.core.testobject.ResponseObject.getResponseBodyContent(ResponseObject.java:74)
	at com.kms.katalon.core.testobject.ResponseObject$getResponseBodyContent.call(Unknown Source)
	at CORDIS_xmlTest.run(CORDIS_xmlTest:27)
	at com.kms.katalon.core.main.ScriptEngine.run(ScriptEngine.java:194)
	at com.kms.katalon.core.main.ScriptEngine.runScriptAsRawText(ScriptEngine.java:119)
	at com.kms.katalon.core.main.TestCaseExecutor.runScript(TestCaseExecutor.java:369)
	at com.kms.katalon.core.main.TestCaseExecutor.doExecute(TestCaseExecutor.java:360)
	at com.kms.katalon.core.main.TestCaseExecutor.processExecutionPhase(TestCaseExecutor.java:339)
	at com.kms.katalon.core.main.TestCaseExecutor.accessMainPhase(TestCaseExecutor.java:331)
	at com.kms.katalon.core.main.TestCaseExecutor.execute(TestCaseExecutor.java:248)
	at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:142)
	at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:133)
	at com.kms.katalon.core.main.TestCaseMain$runTestCase$0.call(Unknown Source)
	at TempTestCase1628191542258.run(TempTestCase1628191542258.groovy:25)

2021-08-05 14:26:05.600 INFO  c.k.katalon.core.main.TestCaseExecutor   - END Test Cases/CORDISxml/CORDIS_xmlTest

Saw your publication: Simplest SOAP Test doen't work in KS v8.0.x, Property 'http://javax.xml.XMLConstants/property/accessExternalDTD' is not recognized due to "accessExternalDTD and findings about the documentBuilder, so
my question here would be if there’s a way I can not implement that builder while trying to accomplish downloading the xml files?

I’m using Katalon Studio 8.0.5 version

Thanks! @kazurayam

You want to downgrade Katalon Studio to v7.9.1. You will be able to download the xml files with v7.9.1.

Thanks @kazurayam , it worked.

One last question, my XML is superheavy, if I open it on browser I need to wait 15 secs.
The XML file is generated now with the script but due to this, generated file is not complete.

How could I wait 15-20 secs between these steps?

   def response = WS.sendRequest(requestObject)
	
	// Verify if the response from the URL returns the 200 status code
	WS.verifyResponseStatusCode(response, 200)
	
	// Get the content string
	def content = response.getResponseBodyContent()

Here’s a WebServiceRequest I made, it took 20489ms to load completely.
On the test script, sending the request takes aprox. 6-7 ms.
File only reaches first 35 lines of 24967.

@duyluong wrote that your fist problem (“accessExternalDTD”) will be fixed in the version 8.1.0, which is still to come.

Have you tried Project Settings > Execution > Web Service ?