Response body: JSON value is equal or greater than

The power of cooperation. :slight_smile: Nice!

1 Like

I played a bit in IDEA (i cannot make KatalonGUI to work properly in Linux … yet).
Maybe you can use this too:

import groovy.json.JsonSlurper

jsonString = """
{
  "total":6.98,
  "available":0.0,
  "pending":0.0
}
"""

//'Set variables'
//
//String jsonString = response.getResponseText()
//
JsonSlurper slurper = new JsonSlurper()
Map parsedJson = slurper.parseText(jsonString)

List keysToCheck = Arrays.asList('total', 'available', 'pending', 'paid')
keysToCheck.each { it ->
    valToCheck = (parsedJson.get(it) != null) ? parsedJson.get(it) : -1.0
    println("${it} : ${valToCheck}")
    //    CustomKeywords.'verify.VerifyValues.valueIsPresent'(valToCheck)
    //    CustomKeywords.'verify.VerifyValues.valueIsEqualOrGreaterThan'(valToCheck)
}

OUTPUT
total : 6.98
available : 0.0
pending : 0.0
paid : -1.0

For the complete Json the output is:

total : 6.98
available : 0.0
pending : 0.0
paid : 6.98
1 Like

More ‘groovyfication’ (We don’t need def or explicit types in a groovy script :wink: . Also, it is the default iterator so we can get rid of it)

slurper = new JsonSlurper()
parsedJson = slurper.parseText(jsonString)

keysToCheck = ['total', 'available', 'pending', 'paid']

keysToCheck.each {
    valToCheck = (parsedJson.get(it) != null) ? parsedJson.get(it) : -1.0
    println("${it} : ${valToCheck}")
    //    CustomKeywords.'verify.VerifyValues.valueIsPresent'(valToCheck)
    //    CustomKeywords.'verify.VerifyValues.valueIsEqualOrGreaterThan'(valToCheck)
}
1 Like

@Ibus,

I have this keyword:

class VerifyValues {
	@Keyword
	def valueIsPresentAndIsEqualOrGreaterThan(BigDecimal jsonValue) {
		if(jsonValue == -1.0) {
			KeywordUtil.markFailedAndStop("Key is not present")
		}
		BigDecimal cBase = 0.0
		if(jsonValue.compareTo(cBase) == -1) {
			KeywordUtil.markFailedAndStop("Key has incorrect value")
		}
	}
}

And this verification code:

JsonSlurper slurper = new JsonSlurper()
parsedJson = slurper.parseText(jsonString)

keysToCheck = ['total', 'available', 'pending', 'paid']

keysToCheck.each { it ->
    valToCheck = (parsedJson.get(it) != null) ? parsedJson.get(it) : -1.0
    println("${it} : ${valToCheck}")
	CustomKeywords.'verify.VerifyValues.valueIsPresentAndIsEqualOrGreaterThan'(valToCheck)
}

If my input is one of those variation all works fine:

String jsonString = '''{
	"total":6.98,
	"available":0.0,
	"pending":-20.0,
	"paid":6.98
}'''

or

String jsonString = '''{
	"total":6.98,
	"available":0.0,
	"pendingssssssssss":0.0,
	"paid":6.98
}'''

But if I use:

String jsonString = '''{
	"total":6.98,
	"available":0.0,
	"pending":-1.0,
	"paid":6.98
}'''

I get Key is not present because it matches -1.0 in valToCheck = (parsedJson.get(it) != null) ? parsedJson.get(it) : -1.0

Well I can use some other number and hope that there will never be matches.

And if I use:

String jsonString = '''{
	"total":6.98,
	"available":0.0,
	"pending":,
	"paid":6.98
}'''

I get:

Verification FAILED Reason: groovy.json.JsonException: Unable to determine the current character, it is not a string, number, array, or object

In this way parsedJson.get(it) != null is not processed correctly. I think I don’t understand something :slight_smile:

I was expecting this :smiley:

With case by case verification, more situations can occur (as you just find out, value missing, or value is a string etc etc)

Therefore … JSON schema validation to the rescue, otherwise you will have to treat all such edge cases and the code will start to be full of exceptions

If not in hurry, tomorrow I will be back at work and i will share again the code of the keyword I use (I am lazy today to re-write it)

the last error you get is due to a malformed Json.

I already also think so :slight_smile:

I’m not hurry, thanks for all you help!

@alexfeel
As a quick fix, for the -1.0 case, we can revert a bit the code.
e.g. just do:

valToCheck = parsedJson.get(it)

and put back in the keyword to check against null. using get() will prevent the code to crash at the variable initialization point

	def valueIsPresent(BigDecimal jsonValue) {
		if(jsonValue == null) {
			KeywordUtil.markFailedAndStop("Key is not present")
		}

Also, value validation should happen only if the key is present, so in the testcase you can do something like:

if (CustomKeywords.'verify.VerifyValues.valueIsPresent'(valToCheck)) {
    CustomKeywords.'verify.VerifyValues.valueIsEqualOrGreaterThan'(valToCheck)
}

For the other cases (string, malformed Json) i will give tomorrow more solutions

LE: the verification against null can happen also in the testcase, there is no need for a dedicated keyword

if (valToCheck != null) {
   CustomKeywords.'verify.VerifyValues.valueIsEqualOrGreaterThan'(valToCheck)
} else {
    //throw some error here
}
1 Like

To prevent the code to stop, try - catch may work also:

try {
    assert valToCheck != null
    CustomKeywords.'verify.VerifyValues.valueIsEqualOrGreaterThan'(valToCheck)
} catch (AssertionError e) {
	println "Something bad happened: " + e.getMessage()
}
1 Like

@Ibus,

I’ve tried that:

Keyword:

package verify

import org.everit.json.schema.Schema
import org.everit.json.schema.ValidationException
import org.everit.json.schema.loader.SchemaLoader
import org.json.JSONObject
import org.json.JSONTokener
import com.kms.katalon.core.annotation.Keyword
import com.kms.katalon.core.util.KeywordUtil

@Keyword
def verifyJsonSchema(String jsonString, String schemaString) {
	JSONObject rawSchema = new JSONObject(new JSONTokener(schemaString))
	Schema schema = SchemaLoader.load(rawSchema)
	try {
		schema.validate(new JSONObject(jsonString))
		KeywordUtil.markPassed("Valid schema")
	} catch (Exception e) {
		KeywordUtil.markFailed("Invalid schema")
	}
}

Verify:

import com.kms.katalon.core.testobject.RequestObject
import com.kms.katalon.core.testobject.ResponseObject
import com.kms.katalon.core.webservice.keyword.WSBuiltInKeywords as WS
import com.kms.katalon.core.webservice.verification.WSResponseManager

import internal.GlobalVariable as GlobalVariable

RequestObject request = WSResponseManager.getInstance().getCurrentRequest()

ResponseObject response = WSResponseManager.getInstance().getCurrentResponse()

'Verify response code'

WS.verifyResponseStatusCode(response, 200)

'Set variables'

'jsonString = response.getResponseText()'

jsonString = '''{
	"total":6.98,
	"available":0.0,
	"pending":0.0,
	"paid":6.98
}'''

schemaString = '''{
	"type" : "object",
	"required" : ["total", "available", "pending", "paid"],
	"properties: {
		"total" : {
			"type" : "number",
			"minimum" : 0.0
		},
		"available" : {
			"type" : "number",
			"minimum" : 0.0
		},
		"pending" : {
			"type" : "number",
			"minimum" : 0.0
		},
		"paid" : {
			"type" : "number",
			"minimum" : 0.0
		}
	}
}'''

CustomKeywords.'verify.Verify.verifyJsonSchema'(jsonString, schemaString)

But with no success:

I leave it till tomorrow :thinking:

@alexfeel tomorrow is another day :slight_smile:
we don’t have to fix every issue in just one day, even God needs 7

will take a deeper look tomorrow, now is movie time for me.

LE. however, you got an JSON exception … which means, as i like to talk with our development team … ‘the error is between chair and keyboard’. it can be fixed, no worries!

I’m totally new to that so I’m sure you’re right :slight_smile:

@alexfeel found the issue, here’s the right schema:

{
	"type": "object",
	"required": ["total", "available", "pending", "paid"],
	"properties": {
		"total": {
			"type": "number",
			"minimum": 0.0
		},
		"available": {
			"type": "number",
			"minimum": 0.0
		},
		"pending": {
			"type": "number",
			"minimum": 0.0
		},
		"paid": {
			"type": "number",
			"minimum": 0.0
		}
	}
}

It was missing some double-quotes at "properties :smiley:
Hint: use a json formatter when in doubt , e.g https://jsonlint.com/

LE: this is the proof I was right, the error was between (my) chair and keyboard

3 Likes

@Ibus, thank you very much!

So thanks to all and with help of this post from @Ibus, the final steps and code for validating JSON response including if keys’ values are equal or greater than 0 are:

At first let’s imagine you want to verify that response:

{
  "total":6.98,
  "available":0.0,
  "pending":0.0,
  "paid":6.98
}

You can use for that JSON Schema specification.

  1. Add to your project these jars:

or use @Ibus’s method from this post.

  1. Create custom keyword:
package verify

import org.everit.json.schema.Schema
import org.everit.json.schema.ValidationException
import org.everit.json.schema.loader.SchemaLoader
import org.json.JSONObject
import org.json.JSONTokener
import com.kms.katalon.core.annotation.Keyword
import com.kms.katalon.core.util.KeywordUtil

@Keyword
def verifyJsonSchema(String jsonString, String schemaString) {
	JSONObject rawSchema = new JSONObject(new JSONTokener(schemaString))
	Schema schema = SchemaLoader.load(rawSchema)
	try {
		schema.validate(new JSONObject(jsonString))
		KeywordUtil.markPassed("Valid schema")
	} catch (Exception e) {
		StringBuffer outmessage = new StringBuffer()
		outmessage << e.getMessage() << "\n"
		e.getAllMessages().each { msg -> outmessage << "$msg \n"}
		KeywordUtil.markFailed(outmessage as String)
	}
}
  1. Use this verification code:
import com.kms.katalon.core.testobject.RequestObject
import com.kms.katalon.core.testobject.ResponseObject
import com.kms.katalon.core.webservice.keyword.WSBuiltInKeywords as WS
import com.kms.katalon.core.webservice.verification.WSResponseManager

import internal.GlobalVariable as GlobalVariable

RequestObject request = WSResponseManager.getInstance().getCurrentRequest()

ResponseObject response = WSResponseManager.getInstance().getCurrentResponse()

jsonString = response.getResponseText()

schemaString = '''
{
	"type": "object",
	"required": ["total", "available", "pending", "paid"],
	"properties": {
		"total": {
			"type": "number",
			"minimum": 0.0
		},
		"available": {
			"type": "number",
			"minimum": 0.0
		},
		"pending": {
			"type": "number",
			"minimum": 0.0
		},
		"paid": {
			"type": "number",
			"minimum": 0.0
		}
	}
}
'''

CustomKeywords.'verify.Verify.verifyJsonSchema'(jsonString, schemaString)

Also you can use schema from the file system, check how in this post.

1 Like

Nice! @Ibus @alexfeel Would you guys willing to create a separate article with a quick guide how to verify JSON schemas in Katalon? What is required and how should it look like + few sample scenarios? I believe it would be really valuable for other QA engineers.

I think it’s not a big problem :slight_smile: But as I said before I’m totally new to this so without @Ibus’ help it may become not so good.

@Marek_Melocik will do (I have to update also the keywords code, since the one in this example is validating just JSON objects, if the response is an array slight modifications are needed)

@alexfeel to avoid headache, will be good if you add some additional verification’s before to actually check the schema, e.g:

WS.comment("Check response status")
WS.verifyResponseStatusCode(response, 200)

WS.comment("Check response contentType")
assert response.isJsonContentType()

WS.comment("Validate response schema")
jsonString= response.getResponseBodyContent()
CustomKeywords.'verify.Verify.verifyJsonSchema'(jsonString, schemaString)

There is no point to check the schema if the API call has failed or the response is not a valid Json.

2 Likes

It is in my final code, I just deleted it in the example for more simple representing.

Thanks!

@Ibus, can you tell me what schema is needed to validate this response (object where its only key has a value as array of objects):

{
  "rewards":[
    {
      "id":14325250,
      "timestamp":1544111851,
      "paccount":70128103,
      "taccount":1000022917,
      "tvolume":0.01,
      "amount":0.1610
    },
    {
      "id":14228666,
      "timestamp":1544099842,
      "paccount":70128103,
      "taccount":1000017612,
      "tvolume":0.01,
      "amount":0.5000
    }]
}

I use this schema:

{
	"type": "object",
	"required": ["rewards"],
	"properties": {
		"rewards": {
			"type": "array",
			"items": {
				"type": "object",
				"required": ["id", "timestamp", "paccount", "taccount", "tvolume", "amount"],
				"properties": {
					"id": {
						"type": "number",
						"minimum": 1
					},
					"timestamp": {
						"type": "number",
						"minimum": 1483228800
					},
					"paccount": {
						"type": "number",
						"minimum": 1
					},
					"taccount": {
						"type": "number",
						"minimum": 1
					},
					"tvolume": {
						"type": "number",
						"minimum": 0.01
					},
					"amount": {
						"type": "number",
						"minimum": 0.01
					}
				}
			}
		}
	}
}

But it doesn’t work as expected, for example, when I change the name of a key in the schema to something that is not in the response, the test still passes. It looks like "required": ["id", "timestamp", "paccount", "taccount", "tvolume", "amount"] doesn’t work.

It really looks as my swagger definition:

image

And have to work :thinking:

i will check a bit later, out for lunch now
however, required has a back effect. i a key which is not specified is present in response, the validation still pass, is just ignored. i have to look in the specs to see if there is any other feature we can use for such scenario

LE: at first look your schema is ok … will try on my pc to see what is goin on

Oh, I don’t need to verify if there are keys which is not specified in the schema, by changing key name in the schema I just want to model a case when there is no required key in response…

Omg :slight_smile: When I wrote it I understood that I need to change the key name in "required": ["id", "timestamp", "paccount", "taccount", "tvolume", "amount"] not in "properties" to model that. It’s my bad. So all is OK :blush:

P.S. JSON Schema Validation is really great thing!

nicely done, young padawan :slight_smile:

1 Like