Can't set the Content-Type

244401716

Is it possible for you to disclose the URL of your application under test to use?

There are many factors that can cause garbled characters. Therefore we require immediate access to the UAT, otherwise it would be hard to figure out the cause.

All URLs with Japanese are scrambled like this https://www.yahoo.co.jp/

@devalex88

Let me tell you what I saw, and my thought.

I opened https://www.yahoo.co.jp/ using Katalon Studio API testing UI. The following screenshot shows that the CJK characters in the response body are rendered garbled.

I opened https://www.yahoo.co.jp/ using Chrome.


The page was encoded in UTF-8, and find the original text is:

日本最大級のポータルサイト。検索、オークション、ニュース、天気、スポーツ、メール、ショッピングなど多数のサービスを展開。あなたの生活をより豊かにする「課題解決エンジン」を目指していきます。

I tried to reproduce gabling the UTF-8 encoded characters.
I used 文字化けさせる奴 - ふくらみ


This interesting tool enabled me to render UTF-8 encoded characters mistakenly as MS932(=SJIS).

The gabled characters reproduced matches exactly with the gabled characters in Katalon Studio.

This proves that, Katalon Studio has a problem rendering the UTF-8 encoded characters. Katalon Studio does not respect the charset expressed by the HTTP Response Header Content-type text/html, charset=UTF-8. It seems rendering the response body with the System’s default charset.

Eclipse=Katalon Studio has preference setting for the text file encoding. I checked it.

The preference is set to be UTF-8. However, I believe KS does not respect this setting as well. KS is rendering the Web API Response body always with Default(MS932). This is a problem to be fixed.

Windows PC in Japan has the default encoding to be MS932. But Mac in Japan has the default encoding to be UTF-8. This difference would be the reason why @244401716 says on Mac OK, on Windows NG.

There are many other posts about the same issue

@devalex88, @YoungNgo, @ThanhTo

Do you recognize this issue as a problem of Katalon Studio?

The issue list of katalon studio Issues · katalon-studio/katalon-studio · GitHub seems to be unaware of this issue.

hello,guys. I had the same problem.Recently I have found a solution to this problem
So, you can try the following solution:
1.Add Windows system environment variables
name:JAVA_TOOL_OPTIONS
path:-Dfile.encoding=UTF-8
2.save it
3.restart your Katalon Studio,

If the above steps are not detailed enough, you can visit my blogtestclass.I wrote the detailed instructions, but I used Chinese to write.
Before configuration modification:


After configuration modification:

u see it works :grinning:

2 Likes

@kazurayam

@devalex88, @YoungNgo, @ThanhTo

Thank you all for this great feedback. This issue will be addressed in KS 7.0.

really helped me

This issue forked into another post:

@kazurayam
@devalex88
@Monster

I also find garbled value in the response,
I had all setup mentioned as above and above example( 'https://www.yahoo.co.jp/`)works fine with it.
In my case charset=Shift_JIS is required, kindly let me know how can I set charset to Shift_JIS.
Unfortunately, I cannot share the URL.
My Request and Response screenshot:

My Configurations

As Katalon.ini should speciify -Dfile.encoding=utf-8 explains, you should try editing C:\Users\myname\Katalon_Studio_Windows_64-x.x.x\katalon.ini so that it includes a line of

-Dfile.encoding=Shift-JIS

You need to stop and restart KS.

Please note, if you change the katalon.ini file this way, it is likely that you would encounter character-garbling when you work on other URLs which have charset=utf-8.

If you want to set `file.encoding=Shift-JIS` just temporarily for a particular URL, then you may try setting Java SystemProperty `file.encoding` dynamically by
def previousCharset = System.getProperty('file.encoding')  // save the previous charset (possibily UTF-8)
System.setProperty('file.encoding', 'Shift-JIS')
... do anything for your target URL
System.setProperty('file.encoding', previousCharset)   // restore the previous charset 

I do not see your test case scripts, so that I have no idea where/how you can do this.

@kazurayam

Here Below I sharing my code,

String URL =‘http://’+serverIP+‘:’+serverPort+‘/’+serverProxyPath+‘/webservice/RbS?command=’+COMMAND+‘&merchantId=’+MERCHANT_ID+‘&sendingBankCode=’+SENDING_BANK_CODE+‘&sendingBranchCode=’+SENDING_BRANCH_CODE+‘&sendingTypeOfDeposit=’+SENDING_TYPE_OF_DEPOSIT+‘&sendingAccountNumber=’+SENDING_ACCOUNT_NUMBER+‘&sendingAccountHolder=’+SENDING_ACCOUNT_HOLDER.replaceAll(“(\p{Z})”, “+”)+‘&hashForFalsificationPrevention=’+this.HASFORFALSIFICATIONPREVENTION
WebUI.comment(URL)
//Set TestObjectProperty
ArrayList headers = new ArrayList()
TestObjectProperty header = new TestObjectProperty(“Content-Type”, ConditionType.EQUALS, “text/csv”)
headers.add(header)
//Set System Properties
def previousCharset = System.getProperty(‘file.encoding’)
WebUI.comment(previousCharset.toString())
System.setProperty(‘file.encoding’, ‘Shift-JIS’)
def currentCharset = System.getProperty(‘file.encoding’)
WebUI.comment(currentCharset.toString())
RequestObject requestObject = new RequestObject(‘’)
requestObject.setServiceType(‘REST’)
requestObject.setRestUrl(URL)
requestObject.setRestRequestMethod(‘GET’)
requestObject.setHttpHeaderProperties(headers)
//Send the request
ResponseObject response = WS.sendRequest(requestObject)
//Verify the request
WS.verifyResponseStatusCode(response, 200)
KeywordUtil.logInfo(response.responseBodyContent)
String responseArray = response.responseBodyContent.split(‘,’)
WebUI.comment('Response Status code: ’ + (responseArray[0]))

I tried your suggestion for dynamic property change, even though it shows the same response,

“繝?せ繝医え繧ァ繧「 繝。繧、繝ウ繧、繝?”

System property configuration output:

image

Sorry, this may not work.

Please try changing the katalon.ini and see how it goes.

I am not sure. Characters-garbling problems are very much application specific, not theoretical, dirty stuff. Try-and-error, that’s only thing you can do. I can not help you very much by chatting remotely.

Or change your Web Server application not to use Shift-JIS, change it to UTF-8. That would definitely make your life easier.

@kazurayam

Please try changing the katalon.ini and see how it goes.

I confirmed by changing various things as below but did not work,

1.Change katalon.ini → -Dfile.encoding=Shift-JIS
2.Update dynamically using System.getProperty
3.Adding JAVA_TOOL_OPTIONS=-Dfile.encoding=Shift-JIS to an environment variable.

I am not sure. Characters-garbling problems are very much application-specific, not theoretical, dirty stuff. Try-and-error, that’s only thing you can do. I can not help you very much by chatting remotely.

そうですね、こちらまで情報いただいてありがとうございました。

Or change your Web Server application not to use Shift-JIS, change it to UTF-8. That would definitely make your life easier.

I will check what can be done.
Thank you

I suppose, all these 3 approaches require Katalon Studio Prefereces> General >Workspace > Text file encoding to be set “Default”, not “Other(selected encoding name)”.

If you choose “Other” then I think Katalon Studio will use the selected encoding rather than the value of System Property file.encoding.

@kazurayam

URL encoding issue has been solved as follows,
We can set the required URL encoding as follows,

Libraries:
import java.nio.charset.Charset as Charset
import java.net.URLEncoder

String random1 = “random word £500 tank \$”
String urlSJIS = “Example Domain” + URLEncoder.encode(random1, Charset.forName(“Shift-JIS”).toString())
WebUI.comment(“SJIS encoded URL”+urlSJIS)
String random2 = “random word £500 tank \$”
String urlUTF = “Example Domain” + URLEncoder.encode(random2, Charset.forName(“UTF-8”).toString())
WebUI.comment(“UTF-8 encoded URL”+urlUTF)

Output:
2021-02-19 10:47:36.259 INFO c.k.k.c.keyword.builtin.CommentKeyword - SJIS encoded URLhttps://example.com?random1=random+word+%81%92500+tank+%24
2021-02-19 10:47:36.261 DEBUG testcase.URLencoding - 9: random2 = “random word £500 tank $”
2021-02-19 10:47:36.263 DEBUG testcase.URLencoding - 10: urlUTF = “Example Domain” + URLEncoder.encode(random2, forName(“UTF-8”).toString())
2021-02-19 10:47:36.270 DEBUG testcase.URLencoding - 11: comment(“UTF-8 encoded URL” + urlUTF)
2021-02-19 10:47:36.272 INFO c.k.k.c.keyword.builtin.CommentKeyword - UTF-8 encoded URLhttps://example.com?random2=random+word+%C2%A3500+tank+%24

Thank you.

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.