URL not built correctly due to variable from data file not correct?

Hi, I’ve had exposure to automation and Katalon for a month now and tried to figure this out, but after numerous tries, I am still struggling and would appreciate your advice regarding reading the content for the endPoint column. I’ve tried escaping the special characters where required, but still struggling. Thank you

The current state is as follows without escaping:
1. Data in my Excel file:

2. Test case Script:
//Get companies
def method1 = “GET”
def endPoint1 = “companies”
def body1 = ‘’
def status1 = 200
def returnString1 = WS.callTestCase(findTestCase(‘Test Cases/1_Reusable_UDF_Scripts/RequestBuilder’),
[body:body1, method:method1, status:status1, endPoint:endPoint1], FailureHandling.STOP_ON_FAILURE)
println returnString1

//Retrieve first company details
def valueId = returnString1.items[0].id
def valueName = returnString1.items[0].name
println(‘…Id extracted is : ’ +valueId)
println(’…Name extracted is : ’ +valueName)
GlobalVariable.companyId = valueId
def CompanyId = valueId
def CompanyName = valueName

//Create a new GET object using builder and process the data file
def fileName = GlobalVariable.GetFile
for (def row=1; row<=findTestData(fileName).getRowNumbers(); row++) {

String columnD = findTestData(fileName).getValue('method', row)
**String columnE = findTestData(fileName).getValue('endPoint', row)**
String columnF = findTestData(fileName).getValue('body', row)
String columnG = findTestData(fileName).getValue('status', row)
String columnH = findTestData(fileName).getValue('id', row)
String columnI = findTestData(fileName).getValue('parameters', row)
String columnJ = findTestData(fileName).getValue('testing', row)

def method = columnD
**def endPoint = URLEncoder.encode(columnE, "UTF-8")**

//If I use def endPoint = “companies/”+CompanyId it works
def body = columnF
def status = columnG
def id = columnH
def parameters = columnI
def testing = columnJ

if (columnD.equals("GET")) {
if (columnJ.equals("test")) {

// if (columnE.equals(“companies/+${CompanyId}”)) {
if (columnH.equals(“yes”)) {

def returnString = WS.callTestCase(findTestCase(‘Test Cases/1_Reusable_UDF_Scripts/RequestBuilder’),
[method:method, body:body, status:status, endPoint:endPoint, fileName:fileName],
FailureHandling.STOP_ON_FAILURE)
println returnString


3. Script in called test case: (Test Cases/1_Reusable_UDF_Scripts/RequestBuilder)

def builder = new RestRequestObjectBuilder()
def requestObject = builder
.withRestRequestMethod(method)
.withRestUrl(“${GlobalVariable.RestURL}”+endPoint)
.withHttpHeaders([
new TestObjectProperty(“Content-Type”, ConditionType.EQUALS, “application/json”),
new TestObjectProperty(“Authorization”, ConditionType.EQUALS, GlobalVariable.token)
])
.withTextBodyContent(body)
.build()

//‘Send the request’
def response = WS.sendRequest(requestObject)

//Verify status code
WS.verifyResponseStatusCode(response, ((status) as int))

//Write response details to the Console
KeywordUtil.logInfo(response.responseBodyContent)

//Parse response
def jsonSlurper = new JsonSlurper()
def testString = jsonSlurper.parseText(response.getResponseBodyContent())
return testString
}
}
}
}

  1. HAR:

Hi @santie.laing,
Please use Groovy GStringTemplateEngine to evaluate your service endpoint properly.

1 Like

Let me add an example how to use Template Engines in Groovy.

Create a Test Case as follows:

/*
 * See Baeldung "Template Engines in Groovy" at 
 * https://www.baeldung.com/groovy-template-engines
 */

import groovy.text.SimpleTemplateEngine

def CompanyId = '12345678'
def smsTemplate = 'companies_${CompanyId0}'
def bindMap = ['CompanyId0': CompanyId]
def smsText = new SimpleTemplateEngine().createTemplate(new StringReader(smsTemplate)).make(bindMap)
println "template:${smsTemplate} with value [CompanyId0:${CompanyId}] -> text:${smsText}"

When ran it will emit the following message in console:

template:companies_${CompanyId0} with value [CompanyId0:12345678] -> text:companies_12345678

1 Like

@santie.laing

In future when you make a post with code fragments, please enclose the code with a pair of back ticks to form BLOCK QUOTE.

スクリーンショット 2020-11-30 11.00.05
This makes your code look like this

    your "code" 'here'

BLOCK QUOTE is useful in 2 points

  1. Indentations are preserved.
  2. Straight quotes are preserved; NOT turned into fancy quotes: " vs ", ' vs ’
1 Like

Thank you for the feedback. It helps me to learn more.
I applied the binding for the endpoint field only for now. Will do the rest later.
I still get the illegal character failure when I use the data file though. I must be missing something trivial.
The value returned in the console is correct.
println(template:$endPoint with value [CompanyId0:$CompanyId] → text:$smsText)
template:“companies/${CompanyId0}” with value [CompanyId0:1] → text:“companies/1”

Could you assist please? Thank you

Code:

import static com.kms.katalon.core.checkpoint.CheckpointFactory.findCheckpoint
import static com.kms.katalon.core.testcase.TestCaseFactory.findTestCase
import static com.kms.katalon.core.testdata.TestDataFactory.findTestData
import static com.kms.katalon.core.testobject.ObjectRepository.findTestObject
import static com.kms.katalon.core.testobject.ObjectRepository.findWindowsObject
import com.kms.katalon.core.checkpoint.Checkpoint as Checkpoint
import com.kms.katalon.core.cucumber.keyword.CucumberBuiltinKeywords as CucumberKW
import com.kms.katalon.core.mobile.keyword.MobileBuiltInKeywords as Mobile
import com.kms.katalon.core.model.FailureHandling as FailureHandling
import com.kms.katalon.core.testcase.TestCase as TestCase
import com.kms.katalon.core.testdata.TestData as TestData
import com.kms.katalon.core.testobject.TestObject as TestObject
import com.kms.katalon.core.webservice.keyword.WSBuiltInKeywords as WS
import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI
import com.kms.katalon.core.windows.keyword.WindowsBuiltinKeywords as Windows
import internal.GlobalVariable as GlobalVariable
import groovy.json.JsonSlurper as JsonSlurper
import com.kms.katalon.core.testobject.ConditionType as ConditionType
import com.kms.katalon.core.testobject.RestRequestObjectBuilder
import com.kms.katalon.core.testobject.TestObjectProperty as TestObjectProperty
import com.kms.katalon.core.webservice.keyword.WSBuiltInKeywords as WS
import com.kms.katalon.core.util.KeywordUtil as KeywordUtil
import groovy.xml.XmlUtil as XmlUtil
import com.kms.katalon.core.testobject.ConditionType as ConditionType
import com.kms.katalon.core.testobject.RestRequestObjectBuilder
import com.kms.katalon.core.testobject.TestObjectProperty as TestObjectProperty
import com.kms.katalon.core.webservice.keyword.WSBuiltInKeywords as WS
import java.lang.Integer as Integer
import groovy.text.SimpleTemplateEngine


//Get companies
def method1 = "GET"
def endPoint1 = "companies"
def body1 = ''
def status1 = 200
def returnString1 = WS.callTestCase(findTestCase('Test Cases/1_Reusable_UDF_Scripts/RequestBuilder'),
	 [body:body1, method:method1, status:status1, endPoint:endPoint1], FailureHandling.STOP_ON_FAILURE)
println returnString1

//Retrieve first company details
def valueId = returnString1.items[0].id
def valueName = returnString1.items[0].name
println('...Id extracted is : ' +valueId)
println('...Name extracted is : ' +valueName)
def CompanyId = valueId
def CompanyName = valueName

//Create a new GET object using builder and process the data file
def fileName = GlobalVariable.GetFile
for (def row=1; row<=findTestData(fileName).getRowNumbers(); row++) {
	
	String columnD = findTestData(fileName).getValue('method', row)
	String columnE = findTestData(fileName).getValue('endPoint', row)
	String columnF = findTestData(fileName).getValue('body', row)
	String columnG = findTestData(fileName).getValue('status', row)
	String columnH = findTestData(fileName).getValue('id', row)
	String columnI = findTestData(fileName).getValue('parameters', row)
	String columnJ = findTestData(fileName).getValue('testing', row)
	
	def method = columnD
	def endPoint = columnE
	def bindMap = ['CompanyId0': CompanyId]
//	def endPoint = "companies/${CompanyId0}"
	def body = columnF
	def status = columnG
	def id = columnH
	def parameters = columnI
	def testing = columnJ
	def smsText = new SimpleTemplateEngine().createTemplate(new StringReader(endPoint)).make(bindMap)
	println "template:${endPoint} with value [CompanyId0:${CompanyId}] -> text:${smsText}"
	
	if (columnD.equals("GET")) {
	if (columnJ.equals("test")) {
		
	def returnString = WS.callTestCase(findTestCase('Test Cases/1_Reusable_UDF_Scripts/RequestBuilder'),
	[method:method, body:body, status:status, endPoint:endPoint, CompanyId0:CompanyId],
	FailureHandling.STOP_ON_FAILURE)
	println returnString
 		}
	}
}

Failure:
2020-11-30 17:48:55.686 ERROR c.k.k.core.keyword.internal.KeywordMain - :x: Unable to send request (Root cause: java.lang.IllegalArgumentException: Illegal character in path at index 59: https://xxx.xxxxx.xxxxx.com/backend/api/v2018-07-17/“companies/${CompanyId0}”

Data
image
Value (copy and pasted from file
= '“companies/${CompanyId0}”

You got this error:

You have unnecessary quotation characters in the URL.

"companies/${CompanyId0}"

This is the problem. " is illegal character in a URL string. You can not have it in a URL string.

How you got the quotations? I guess you need to check the following line:

String columnE = findTestData(fileName).getValue('endPoint', row)
println('>>>' + columnE + '<<<')

Possiblly you will see

>>>"${CompanyId0}"<<<

This implies that findTestData(fileName).getValue('endPoint', row) returns a string encolosed with double quotations "${CompanyId0}".

How to remove the unnecessary " characters? There could be many ways. An example:

String columnE = findTestData(fileName).getValue('endPoint', row)
columnE = columnE.replace('"', '')

See https://docs.oracle.com/javase/7/docs/api/java/lang/String.html#replace(char,%20char)

1 Like

In the screenshot you posted, I found that the Excel cell value contains enclosing double quotation characters.
スクリーンショット 2020-12-01 8.20.14

Why you have "..." here? looks unnecessary stuff. It looks just confusing you.

Thank you @huynguyen and @kazurayam
I added the replacing of the double quotes and removed the double quotes from the file, but still got issues with the illegal character.
Then I replaced the following
def endPoint = columnE
def bindMap = [‘CompanyId0’: CompanyId]
def smsText = new SimpleTemplateEngine().createTemplate(new StringReader(endPoint)).make(bindMap)
println “template:${endPoint} with value [CompanyId0:${CompanyId}] → text:${smsText}”
with
def endPoint2 = columnE
def bindMap = [‘CompanyId0’: CompanyId]
def endPoint = new SimpleTemplateEngine().createTemplate(new StringReader(endPoint2)).make(bindMap)
println “template:${endPoint2} with value [CompanyId0:${CompanyId}] → text:${endPoint}”
and all the combined changes seemed to have solved the issue.
Thank you for your assistance. Much appreciated.

import static com.kms.katalon.core.checkpoint.CheckpointFactory.findCheckpoint
import static com.kms.katalon.core.testcase.TestCaseFactory.findTestCase
import static com.kms.katalon.core.testdata.TestDataFactory.findTestData
import static com.kms.katalon.core.testobject.ObjectRepository.findTestObject
import static com.kms.katalon.core.testobject.ObjectRepository.findWindowsObject
import com.kms.katalon.core.checkpoint.Checkpoint as Checkpoint
import com.kms.katalon.core.cucumber.keyword.CucumberBuiltinKeywords as CucumberKW
import com.kms.katalon.core.mobile.keyword.MobileBuiltInKeywords as Mobile
import com.kms.katalon.core.model.FailureHandling as FailureHandling
import com.kms.katalon.core.testcase.TestCase as TestCase
import com.kms.katalon.core.testdata.TestData as TestData
import com.kms.katalon.core.testobject.TestObject as TestObject
import com.kms.katalon.core.webservice.keyword.WSBuiltInKeywords as WS
import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI
import com.kms.katalon.core.windows.keyword.WindowsBuiltinKeywords as Windows
import internal.GlobalVariable as GlobalVariable
import groovy.json.JsonSlurper as JsonSlurper
import com.kms.katalon.core.testobject.ConditionType as ConditionType
import com.kms.katalon.core.testobject.RestRequestObjectBuilder
import com.kms.katalon.core.testobject.TestObjectProperty as TestObjectProperty
import com.kms.katalon.core.webservice.keyword.WSBuiltInKeywords as WS
import com.kms.katalon.core.util.KeywordUtil as KeywordUtil
import groovy.xml.XmlUtil as XmlUtil
import com.kms.katalon.core.testobject.ConditionType as ConditionType
import com.kms.katalon.core.testobject.RestRequestObjectBuilder
import com.kms.katalon.core.testobject.TestObjectProperty as TestObjectProperty
import com.kms.katalon.core.webservice.keyword.WSBuiltInKeywords as WS
import java.lang.Integer as Integer
import groovy.text.SimpleTemplateEngine


//Get companies
def method1 = "GET"
def endPoint1 = "companies"
def body1 = ''
def status1 = 200
def returnString1 = WS.callTestCase(findTestCase('Test Cases/1_Reusable_UDF_Scripts/RequestBuilder'),
	 [body:body1, method:method1, status:status1, endPoint:endPoint1], FailureHandling.STOP_ON_FAILURE)
println returnString1

//Retrieve first company details
def valueId = returnString1.items[0].id
def valueName = returnString1.items[0].name
println('...Id extracted is : ' +valueId)
println('...Name extracted is : ' +valueName)
def CompanyId = valueId
def CompanyName = valueName

def fileName = GlobalVariable.GetFile
for (def row=1; row<=findTestData(fileName).getRowNumbers(); row++) {
	
	String columnD = findTestData(fileName).getValue('method', row)
	String columnE = findTestData(fileName).getValue('endPoint', row)
	println('>>>' + columnE + '<<<')
	columnE = columnE.replace('"','')
	String columnF = findTestData(fileName).getValue('body', row)
	String columnG = findTestData(fileName).getValue('status', row)
	String columnH = findTestData(fileName).getValue('id', row)
	String columnI = findTestData(fileName).getValue('parameters', row)
	String columnJ = findTestData(fileName).getValue('testing', row)
	
	def method = columnD
	def endPoint2 = columnE
	def bindMap = ['CompanyId0': CompanyId]
	def endPoint = new SimpleTemplateEngine().createTemplate(new StringReader(endPoint2)).make(bindMap)
	println "template:${endPoint2} with value [CompanyId0:${CompanyId}] -> text:${endPoint}"
	def body = columnF
	def status = columnG
	def id = columnH
	def parameters = columnI
	def testing = columnJ
		
	if (columnD.equals("GET")) {
	if (columnJ.equals("test")) {
		
	def returnString = WS.callTestCase(findTestCase('Test Cases/1_Reusable_UDF_Scripts/RequestBuilder'),
	[method:method, body:body, status:status, endPoint:endPoint], FailureHandling.STOP_ON_FAILURE)
	println returnString
 		}
	}
}
	

Data in file: