How to crawl multiple browser pages

Product Forums Katalon Studio
1

I have a routine that collects data from a browser table and works great if there is only enough data to fill one browser page. But now I’m getting more data that spans multiple browser pages. On the bottom I have the “Next>” button but can’t get it to work. the code below works great for one page but is there are more I can’t get it to work:

import org.openqa.selenium.By as By
import org.openqa.selenium.WebDriver as WebDriver
import org.openqa.selenium.WebElement as WebElement
import com.kms.katalon.core.webui.driver.DriverFactory as DriverFactory
WebDriver driver = DriverFactory.getWebDriver()
‘Expected value from Table’
‘To locate table’
WebElement Table = driver.findElement(By.xpath(“//table/tbody”))
‘To locate rows of table it will Capture all the rows available in the table’
List rows_table = Table.findElements(By.tagName(‘tr’))
‘To calculate no of rows In table’
int rows_count = rows_table.size()

The above code gives me the number of rows on the first page/table of data but I have over 100 pages to work through.

Any ideas?

2

Can you show the HTML of the “Next>” button so that we may assist you?

1 Like
3

Never mind. I have a way to get around this. I’m able to get the total number of pages associated with this from the first UI page I land on. I also noticed that the URLs are formatted as “https:///tags/index/page:n”. (The “n” is the UI page number.) I created a loop and just update the “n” value with the page number I want and use “WebUI.navigateToUrl(<newly created URL)” to get to it and process the page.

I was just wondering if there was a routine already created in Katalon that would automatically do this step.

1 Like
4

Thanks for posting back your solution.

No, there isn’t. What you’re doing is a workable solution.