Question about Testing Page Titles with Non Printable Characters

@Randy.Ramkissoon

Could you make a zip file of your project and share it to us?
We need to reproduce the problem on our own PC.

On my side using 7.6, this test passes:

Am I doing something differently ?

No, that looks fine to me. My code definitely went bang right where I left the comment. I just reran it and it’s fine.

I’m ducking out now - I’m too confused by this to offer anything but more confusion :blush: :angry:

@kazurayam @ThanhTo @Russ_Thomas

Good Morning All,

Thanks for your feedback. The main item I can see which is missed is:

  • The several examples all use test data directly inputed in the test case (this has always worked).
  • Where we have been failing is when reading the data from the Excel (which I had attached earlier).

Each time we call a compare function to compare:

  • assert WebUI.getWindowTitle() == Test_Page_Title
  • Test_Page_Title = “." + Test_Page_Title + ".
  • assert WebUI.verifyMatch (WebUI.getWindowTitle() “.” + Test_Page_Title + ".", true)
  • assert WebUI.verifyMatch(Test_DOM_Title, Test_Page_Title, false)
  • String Test_DOM_Title = WebUI.executeJavaScript(“return document.title;”, null)
    assert WebUI.verifyMatch(Test_DOM_Title, Test_Page_Title, false)

It seems there is something left in the Page Title which is not matching what is recorded and a failure was reported that the two string did not match.

Error in Console:

Unable to verify match between actual text ‘Residents’ and expected text ‘Resident​​s’ (Root cause: com.kms.katalon.core.exception.StepFailedException: Actual text ‘Residents’ and expected text ‘Resident​​s’ are not matched

The attached project has the data file I used where you will see some web page titles pass(8 of 12). The test data line referencing the web page “Residents” is the one we have all been troubleshooting.

Can’t wait to hear about your observations. Thank you.

torontohousing-ca-forum.zip (2.2 MB)

Hi,

Are you suggesting I convert the Excel to a CSV and then open in Notepad++ to look for hidden characters?

Or am I opening another log file in Katalon?

I copied and pasted from the Katalon visual Console and got this:

Thank you for the zip you provided.

I executed the project you shared in the zip. I got the following output.

2020-08-25 22:56:53.180 DEBUG .2. Crawl Torontohousing Site Map - File - 5: println("This is the Test_DOM_Title value: " + ">>>" + Test_DOM_Title + "<<<")
This is the Test_DOM_Title value: >>>Residents<<<
2020-08-25 22:56:53.186 DEBUG .2. Crawl Torontohousing Site Map - File - 6: println("This is the page title being tested: " + ">>>" + Test_Page_Title + "<<<")
This is the page title being tested: >>>Resident<<<

I noticed a point:

The web page has a title >>>Residents<<< which is plural.

The Excel has a cell with data >>>Resident<<< which is singular.

These two would never match.

You have a code fragment

assert WebUI.verifyMatch(Test_DOM_Title, Test_Page_Title, false)

this is functioning just fine. It reports Residents and Resident does NOT match, which is correct.


I suspect that is an after-the-fact typo - earlier code/examples show Test_Page_Title as “Residents”.


But let’s see what Randy has to say.

I found a few more rows in his Excel causing failure; those have typo as well.

I’m a bit confused here when it comes to the “Resident” instead of “Residents” showing up in your execution. Please check your Data file as I suspect some how it got changed. I opened up the ZIP I sent you and the 'Residents value was correct.

Further, when I wrote my summary it clearly shows a screenshot of my data file (Excel) where the page title is “Residents”.

The data mapping for the file shows this:

It seems my original files are correct. Perhaps if your Test Excel has a missing ‘s’, I would just add it back?

Please share your screenshot where the typos are so I can double check.

@Russ_Thomas @grylion54 @kazurayam @ThanhTo

I think I found a clue on what is causing the issue. My initial guess is it is not the Test Data file, but how Katalon maybe parsing it. To test my theory I decided to export my Excel as a CSV:

Excel Data File in Katalon

CSV Data File in Katalon

The CSV file was generated via Excel Export. You will see that ‘Residents’ is being exported as ‘Resident??s’ .

If Katalon is doing something similar internally when processing\fetching data then this could be why we are having issues with the strings not matching. If my theory is correct there appears to be two bugs off the top of my head:

(1) The error message is misleading on what strings were actually comparing (would need code debug to prove this).

(2) There are compatibility issues with Excel being used as a data file (suspect something you want to nail down for your corporate customers).

I further tested this by cleaning up the ‘Residents’ row in the CSV and the test passed successfully. So Test Logic in the test case is correct.

I’ve included the Data Files for your analysis (created in Excel 2013).

Note:

  • Suspect we have a bug here, so issue is not resolved.

Data Files.zip (14.1 KB)

2 Likes

While this is a fascinating and intriguing topic in the academic sense, it’s a huge time-sink (keep that in mind Randy).

Which is why I refuse to use Excel (or any other “outside” tool) as part of my test environment - too many moving parts, least amount of control. You’ve (we’ve) done a great job of testing Excel but to what end? How does that sit with management?

My advice, FWIW: produce your own data files. Produce your own data-only classes (in Groovy). Throw those at your Tests. If you can’t 100% guarantee what you’re testing with/against, your tests carry at best the same percentage guarantee. :sunglasses:

Caveat: The only time I’d consider Excel (etc) as part of a test is when it is a part of a user story/scenario I’m trying to test.

1 Like

Yes, but be careful. The ?? is how that tool chooses to display “unknown character”. It has found a code-point (in unicode parlance) that it can’t map to a character for printing/display.

<p style="pessimistic">
Really, this is a rat’s nest/time-sink. Somebody has a solution (not me). And when you apply the solution, rest assured you’re just waiting for the next issue (caused by too many moving parts).
</p> :upside_down_face:

That said, praises for finding the culprit, Randy :clap:

Thanks Russ and well the team of Consultants/Developers for the support.

I do agree using Excel for a data file is a slippery slope. Some of you might have been in our industry from 1990s and on and have seen how conversion of files (whether by Excel or other tools\libraries we use in our code have been problematic as new versions of products have been released).

I will leave it to the Katalon Team to research and Test with the project files I have provided. As a QA I am willing to bet that other clients will hit the same issue demonstrated here which just might lose you a sale because many non-technical users trying to use Katalon to start on automation or do their UAT.

I have switched my data file to CSV, and I and my team will be using that as a best practice moving forward.

1 Like

@Randy.Ramkissoon

Thank you for your reprt, I have delegated this issue to our team for reproduction. We will let you know if there’s more information.

I was interested in what is ?? in Resident??s.

I wrote a code to examine it.

import java.nio.file.Files
import java.nio.file.Path
import java.nio.file.Paths

import org.apache.poi.ss.usermodel.Cell
import org.apache.poi.ss.usermodel.Row
import org.apache.poi.ss.usermodel.Sheet
import org.apache.poi.ss.usermodel.Workbook
import org.apache.poi.xssf.usermodel.XSSFWorkbook

import com.kms.katalon.core.configuration.RunConfiguration

Path projectDir = Paths.get(RunConfiguration.getProjectDir())
Path xlsx = projectDir.resolve("Data Files/TorontoHousing-Site-Map-Content.xlsx")
assert Files.exists(xlsx)

FileInputStream file = new FileInputStream(xlsx.toFile())
Workbook workbook = new XSSFWorkbook(file)
Sheet sheet = workbook.getSheet('TCHC_Site_MAP')
assert sheet != null
Row row = sheet.getRow(2)
assert row != null
Cell cell = row.getCell(2)  // Cell 'C3'
assert cell != null

println("'Residents'.length() is ${'Residents'.length()}")

String value = cell.getStringCellValue()
println("cell.value is ${value}")
println("cell.value.length() is ${value.length()}")
for (int i = 0; i < value.length(); i++) {
	println("${i} '${value.charAt(i)}' ${value.codePointAt(i)}")
}

I got the following output

0 R 82
1 e 101
2 s 115
3 i 105
4 d 100
5 e 101
6 n 110
7 t 116
8 ​ 8203
9 ​ 8203
10 s 115

The UNICODE CodePoint 8203 is “Zero-width space”.

By the way I executed the project on Mac, not on Windows. I edited the xlsx file a bit using the Number app instead of Excel. I meant to remove a few rows in the sheet, which run successful. It seems that Resident??s in the cells were silently truncated to Resident by Number while I was not aware of.

Very interesting observation Kazurayam. I’m glad we have the opportunity to look at the intergration and environment difference (this will help the Katalon Test Team). I’m glad we found these different behaviors (as it will also help me troubleshoot issues in the future).

My personal setup is Linux (Manjaro) so if I only used it then I would be making test data files in Libre Office (Application: Calculate).

My employer is Windows 10 with MS Office 2013 (so many of the technical teams using these Microsoft tools as well). In the near future I notice other IT companies are using Google Docs or Office 365. So more slippery slope.

so you have few choices:

  • use windows and MS office to create your data and also to run the tests
  • use csv but take care of data formatting (did you noticed the CR/LF in your previous screenshots?)
  • write a sane data provider for your tests (sql, sqlite, custom classes, whatever)

Thanks for your suggestions bionel. These are good recommendations but misleading when it comes to the real issue in this post. I have to state this because I like many people on the forum look here for supplementary information to the Katalon documentation.

The issue was not the Test Data File for the specific example data ‘Residents’ we were troubleshooting. Instead we observed that something his happening behind the scenes when Katalon is utilizing the Excel Data file. As a work around I switched to a CSV file (which would make it more accessible to everyone who wanted to run the test case).

Note, there was other test data which would not work (because the production website content has since changed). These were not in scope of the discussion.