Chrome: saving an xml file instead of displaying it

Hi,

i have to automate a test case where the user goes to a website, clicks right in an xml file, and saves it in a certain folder. This cannot be done with Katalon unless there is a way to download the xml instead of displaying it. Any ideas how to do this?

Any help is welcome…

Hi do you have a screenshot of where the xml file is displayed on the website (If I understand you correctly). Do you have the css/xpath, then I think you can solve it via the actions object.

Hi,

Thank you! If you can explain me how to do it, it would be fantastic! how to save link as? or is something else I should do?

katalon%202

katalon

You can do it by means of the wget
Wget is a small and easy-to-use command-line program used to automate downloads .

some code snippets.
Put this in a custom function (static function is better as a custom function)

WebElement downloadlink = WebUiCommonHelper.findWebElement(to, 30)
//where to is the object (can be id, css, xpath etc, e.g. By.id(“messenger-download”)):wink:

String sourceLocation = downloadlink.getAttribute(“href”);
String wget_command = "cmd /c C:\Wget\wget.exe -P --header=“Accept:text/xml,/” D: --no-check-certificate " + sourceLocation;

    try {
    Process exec = Runtime.getRuntime().exec(wget_command);
    int exitVal = exec.waitFor();
    System.out.println("Exit value: " + exitVal);
    } catch (InterruptedException | IOException ex) {
    System.out.println(ex.toString());
    }
    
    }

Be aware that if you are on linux you need to use another command for wget

I have made a demo project here:


In Katalon Studio, a test case can use 2 types of built-in keywords together: WebUI keywords and WebService keywords. You will use WebUI keyword to scrape a web page to identify the URL of an XML you want. And you can download the XML document using WebService keywords. You do not necessarily use external downloading commands such as wget or curl .

Description of the demo project

My target web site is

The page contains a link with label “FIFA World Cup Russia 2017”. This links to a XML document

I want to download the XML and save it into a local file.

I made a TestObject named a_FIFA World Cup Russia 2018 testObjectThis test object has a xpath

//div[@id='rssTab']//a[text()='FIFA World Cup Russia 2018']

I wrote a test case TC1

import static com.kms.katalon.core.testobject.ObjectRepository.findTestObject

import java.nio.file.Files
import java.nio.file.Path
import java.nio.file.Paths

import com.kms.katalon.core.configuration.RunConfiguration
import com.kms.katalon.core.model.FailureHandling as FailureHandling
import com.kms.katalon.core.testobject.RestRequestObjectBuilder
import com.kms.katalon.core.webservice.keyword.WSBuiltInKeywords as WS
import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI

def anchorElement = findTestObject('Object Repository/Page_RSS Feeds - FIFA.com/a_FIFA World Cup Russia 2018')

// Open site
WebUI.openBrowser('')
WebUI.setViewPortSize(800, 600)
WebUI.navigateToUrl('http://static.fifa.com/rss-feeds/index.html')
WebUI.scrollToElement(anchorElement, 3, FailureHandling.STOP_ON_FAILURE)
WebUI.verifyElementPresent(anchorElement, 10, FailureHandling.STOP_ON_FAILURE)
// identify URL of the XML document
def href = WebUI.getAttribute(anchorElement, "href")
// will see href="http://static.fifa.com/worldcup/news/rss.xml"
println "href=${href}"
WebUI.closeBrowser()

// Create a new GET object using builder
def builder = new RestRequestObjectBuilder()
def requestObject = builder
	.withRestRequestMethod("GET")
	.withRestUrl(href)            // here we specify the URL found in the web site
	.build()

// Send a request'
def response = WS.sendRequest(requestObject)

// Verify if the response from the URL returns the 200 status code'
WS.verifyResponseStatusCode(response, 200)

// Get the content string
def content = response.getResponseBodyContent()

// prepare output directory
Path projectdir = Paths.get(RunConfiguration.getProjectDir())
Path outputdir = projectdir.resolve("tmp")
Files.createDirectories(outputdir)
Path file = outputdir.resolve("FIFA_World_Cup_Russia_2018.rss.xml")

// save XML into file
file.toFile().write(content)

I ran this test case and got a XML file saved in <projectdir>/tmp/FIFA_World_Cup_Russia_2018.rss.xml xml_saved

This worked for me.

2 Likes

Hi Kazurayam, nice suggestion and good example.

But does this also work for documents where you directly get a save as button popup, instead of what I see now you see the raw xml rss when you click on it

1 Like

Well, there would be no single solution that can load silver bullet. Each web page has it’s own design, and we always need to adapt to it.

All I wanted to show in the above post was that WebService keywords can be an alternative to invoking wget from a test case.

1 Like

You are absolute right Kazurayam. I just like to challenge it so we all learn from it eventually. Thank you for your idea.

Respect!:slight_smile: and a good alternative

1 Like

Thank you very much Ralph and Kazurayam!

I only have a problem with both solutions: I don’t know what will be the link to download. There is a table in the page that is populated with a new file every day. Each day is represented as a row in the table. Every day contains 0 or 1 file, and each file, has a new id, but I only know that I want the last file, or a specific file (specific date). How do I handle this issue?

This an example of table with only one row (one day):

Date The XML file another header
28/12/2018 Fichier XML

You want a Test Object which can identify the <a> tag which links to the file you want; the file updated last among the files listed in the <table>. Am I right?

In order to design the Test Object, we need to look into the whole HTML code of your target URL. Screenshot of the page is not enough.

Is your target web site a public URL which we can open in browser? If so please share the URL to us. Otherwise, please save the HTML into file and attach it to your next comment— of course you can erase any sensitive information in it.

Hi kazurayam,

Thanks for the tip. Checking again the code, the only tag that contains the date of the file is contained in a where the file url of the file is not present. From the whole html, only the DIV “resultsDiv” is of interest. In this DIV, there are rows (one for each day of the month). Each row contains cells. The first cell contains the date, the second contains an tag that contains, among other parameters, the date, but the url of the file is located in another cell (td).

Does it mean that it is not possible to automate?

I have attached the DIV html code. Thanks for all your help and support

DIV_results.txt (2.7 KB)

If you need to find the link simplistically do this where you need to parameterize the date 01/01/2019
.//*[text()=“Results from 01/01/2019”]

if you need to click on the img alt pdf then you can do this

.//*[text()=“Results from 01/01/2019”]/ancestor::tbody/descendant::img[contains(@alt,“PDF”)]/… (also parameterize the date)

hope this helps

Hi,

Thanks again to you and to Kazurayam. For the xml, I have realised that if I have a table with rows and cells, and in one of the cells, as Kazurayam said, there is an “a” that contains an href, the simplest way to get all the information I want is to create as many test objects as I now they are (maximum 31, the maximum number of days of a month), and extract the information from each test object, as done for the FIFA project.

This are examples of Test objects I created:

//div[@id=‘resultDiv’]/table/tbody/tr/td - this will contain the date, that I can extract in the code with WebUI.getText

//div[@id=‘resultDiv’]/table/tbody/tr/td[6]/a - this will contain the url to the xml, that I can extract in the script with WebUI.getAttribute(anchorElement, “href”)

etc.

The only thing I don’t know is if I really have to create a test object per row, or this can be somehow parametrized. I mean, that I have to create a test object for tr (row 1), tr[2] (row 2), tr[3] (row 3)…

Thanks again.

Katalon Studio provides a feature called “Parameterized Test Object”. It will certainly help you.

Have a look at the following official document:
https://docs.katalon.com/katalon-studio/docs/parameterize-webmobile-test-object-properties.html

I am afraid this documentation page is poor. It does not tell you how you should write a “parameterized Test Object”. It only provides a sample code of a Test Case which calls a Test Case with param.

I have updated my demo project:

Let me explain a bit about it.

I created a sample target Web Page

This page looks similar (but not identical) to the one which yolanda shared with us. It has a table with a <tr>
element with <a> element to XML.

I made a Test Object which has

  1. Selection method=Attribute
  2. parameterized string as an XPath expression : //div[@id='resultDiv']/table/tbody/tr[${row}]/td[6]/a. Please find a place holder ${row} is used here

I made a Test Case TC2

Please read the following line where the place holder ${row} in the Test Object is interpolated dynamically.

// look up the href attribute of the XML in the 1st row of the table
String href = 
	WebUI.getAttribute(
		findTestObject('Page_Sample/nth_tr_anchor_to_xml', ['row':'1']),   // interpolate tr[${row}] => tr[1]
		'href')

In the Test Case you can write ['row': '2'], ['row': '3'] and so on if the target HTML has more rows. You will apply these key-value pairs to a single Test Object named 'Page_Sample/nth_tr_anchor_to_xml which has a place holder in selector strings(XPath or CSS).

So, you do not have to create many Test Objects for each table rows.

1 Like

Thanks a lot! I set a loop and it works like a charm. I have another question. Sometimes, the month doesn’t have as many files as days, then I put a control: if the file is not there, break the loop. Even when I put CONTINUE_ON_FAILURE, the test result is failed. How can I avoid this?

Please find below the code snippet.

for (int i=1; i<=last_day;i++){

def anchorElement = findTestObject(‘Object Repository/XML_n/XML_file’,[‘row’: i])

def My_id = findTestObject(‘Object Repository/XML_n/My_id’,[‘row’: i])

def My_date = findTestObject(‘Object Repository/XML_n/My_date’,[‘row’:i])

//if the row doesn’t contain a file, that means that there are no more files for the month and you have to stop the loop

boolean fileExists = WebUI.verifyElementPresent(My_date, 10,FailureHandling.CONTINUE_ON_FAILURE)
println “fileExists=${fileExists}”

if (!fileExists){
break;
}

Possibly you should not invoke WebUI.verifyElementPresent(My_date) at all in order to test if the file is there or not.

Instead, how should you do it? — It would depend on how your target HTML is coded. Please have a deep breath, and think about it.

thanks for the hint. Sometimes one doesn’t think about using common tools. I counted the rows of the table

//Count the number of rows in the table. This, functionally, means count the number of xml files in the month

WebDriver driver = DriverFactory.getWebDriver()

‘To locate table’
WebElement Table = driver.findElement(By.xpath("//div[@id=‘resultDiv’]/table/tbody"))

‘To locate rows of table it will Capture all the rows available in the table’
List rows_table = Table.findElements(By.tagName(‘tr’))

‘To calculate no of rows In table’
int rows_count = rows_table.size()

I have a new issue with this code…Exceptionally, it can happen that a row doesn’t contain a link to an xml file, but the row exists.

Exceptionally, the row contains 2 xml files. They are separated by
.
This means that the cell td[6], can exceptionally contain 0 files or 2 files.

How can I handle these cases, taking into account that I loop over all rows in the page?

thanks in advance

I extended my demo project further. See

The Application Under Test Sample now has 3 rows:


Please note: The 1st row contains 1 link to XML. The 2nd row contains no link to XML. The 3rd row contains 2 links to XML files.

I added a Test Case TC3. This test case scans the target HTML for <a> elements to XML file. I took it into account that a <tr> element may contain zero, 1, 2 or more <a> elements to XML.

WebUI.openBrowser('')
WebUI.navigateToUrl('https://kazurayam.github.io/SavingAnXmlFileRatherThanDisplayingIt/')
WebUI.verifyElementPresent(findTestObject('Page_Sample/div_resultDiv'), 10)

// prepare output directory
Path projectdir = Paths.get(RunConfiguration.getProjectDir())
Path outputdir = projectdir.resolve("tmp")

// grasp <tr> elements
List<WebElement> trElements = WebUI.findWebElements(
	new TestObject().addProperty("xpath", ConditionType.EQUALS, 
							"//div[@id='resultDiv']/table/tbody/tr"), 10)

for (int i = 0; i < trElements.size(); i++) {
	// grasp <a> elements to XML
	List<WebElement> anchorElements = WebUI.findWebElements(
		new TestObject().addProperty("xpath", ConditionType.EQUALS,
						"//div[@id='resultDiv']/table/tbody/tr[${i+1}]/td[6]/a"), 10)
	for (int j = 0; j < anchorElements.size(); j++) {
		String href = anchorElements.get(j).getAttribute("href")
		// determin the file location
		Path outfile = outputdir.resolve("sample(${i},${j}).xml")
		WebUI.comment("(${i},${j}) href=${href} outfile=${outfile.toString()}")
		// download XML and save it into file
		WebUI.callTestCase(findTestCase("modules/DownloadXML"),
					["href":href, "outfile": outfile.toString()],
					FailureHandling.OPTIONAL)
	}
}
WebUI.closeBrowser()

The TC3 calls another test case named modules/DownloadXML:

/**
 * @param href (java.lang.String) URL of XML file to download
 * @param outfile (java.lang.String) path of the file into which the XML is saved
 */
Objects.requireNonNull(href)
Objects.requireNonNull(outfile)

// Create a new GET object using builder
def builder = new RestRequestObjectBuilder()
def requestObject = 
    builder.withRestRequestMethod("GET")
        .withRestUrl(href) // here we specify the URL found in the web site
        .build()

// Send a request
def response = WS.sendRequest(requestObject)

// Verify if the response from the URL returns the 200 status code'
WS.verifyResponseStatusCode(response, 200)

// Get the content string
def content = response.getResponseBodyContent()
Path file = Paths.get(outfile)

// prepare output directory
Files.createDirectories(file.getParent())

// save XML into file
file.toFile().write(content))

By executing TC3, I got a few xml files created in the tmp directory under the project dir.

:tmp [master]$ tree
.
├── sample(0,0).xml
├── sample(2,0).xml
└── sample(2,1).xml

0 directories, 3 files

The mission is accomplished, isn’t it?

1 Like