Verify ComplexJsonResponses

I have developed a sample Test Case in Katalon Studio v8.6.8 which validates a JSON document responded from a REST API

using the networknk json validator.

The schema in this example shows the following points:

  1. express data type of each items : string, interger, array
  2. express which items are required (all the rest is optional)
  3. express minimum integer value
  4. enumerates acceptable string values
  5. express pattern of string value using regular expression

I think it is a tough task to write a usefull JSON schema. You need to learn a lot. You would spend a long time to write and debug the schema.


I found I have to write:

            "pattern": "[\\\\.0-9]+"

to match with a string value "163.131.26.7".

The four backslashes \\\\ looks puzzling and ugly, but is required. Why?

  • The Regular Expression requires \. to match a single dot . character; so we should write \.

  • The JSON syntax rule requires a single backslash character in a JSON string literal to be escaped by a prepended backslash; so we should write \\. in JSON

  • Groovy syntax rule requires a single backslash character in a Groovy string literal to be escaped by a prepended backslash; so we have to write \\\\.

That’s true. It has a learning curve and is not trivial.
Before to start using it, it is a good idea to take a look at the specifications, e.g for Draft 4:

(the user may want do read also the changes from version to version to figure it out what it is more appropiate. personally, I only used draft 4, was good enough for my needs)

May look a bit ‘criptic’ at the beginning, but with a bit of practice you can reach a level where you can write pretty complex schemas from scratch in ~ 10 mins.

Looks like you produce the schema by code. That is fine, but will add one more level of complexity.
The way I used to do it was to write the schemas as text files and simple read. Due to laziness, of-course, so I only had to focus on the draft specs.

hints on how to master json schemas:

  • start with a verry basic one, e
    g if the root is an object or array. validate the type. do not attempt to write the full schema from the begining
  • after every change you may like to run your test
  • if yout object contains other objects, add them as fields and check the type as before
  • further add the object fields. check the types
  • add more validators as you like e.g is a requires field or not, pattern etc
  • and so on…

and keep practicing

My recommendation to a API testers who want to validate a JSON response in Katalon Studio,

  1. You can start with the primitive “compare 2 JSONs” approach. You can create the expected JSON by any method; you can just hand-write it or make a request to the URL and store the response into a local file. You can make a request for the actual JSON and you want to compare them textually. By this exercise, you will get in touch with your JSON. You will understand it better. You would be able to implement this comparison processing fairly quickly as I explained above.

  2. Sooner or later, you will find out some critical points (problematic items in the JSON) which you want to pay more attentions to.

  3. Then you want to build the validation processing using JSON Schema. You would develop a JSON Schema which verifies the subject JSON with your focused interest. It may take long hours.

  4. eventually you would utilize both approaches

1 Like

@kazurayam Thanks for all the input you provided above .
I am using TextsDiffer.diffStrings() and it perfectly writes the differences to file (*.md)
If I know that a particular difference can be ignored? is there a way to exclude that in the result?

1 Like

No. My TextsDiffer is NOT capable to do it.

Alternatively you can pre-process the input JSON using some other JSON processing tool. For example, jq.

You can transform a JSON to produce another text, which contains portions you are interested in. In other words, you can exclude the portions you do not like to see. Once you have got a pair of pre-processed text files, you can compare them with the TextsDiffer.

1 Like

@kazurayam Thanks. I was able to filter it out using other logic and integrate it with TextsDiffer.

I wanted to check if there is a way to ignore the ones which are equal , I am trying to minimize the file with only changed, deleted and inserted values?

If I get that it would be a great help.

  • inserted rows: 14
  • deleted rows : 0
  • changed rows : 5
  • equal rows: : 146
1 Like

No, my TextsDiffer does not work as such. I do not like to change my source code to implement that idea.


But the source code is availabe at GitHub.

You can download it and modify as you like.

I have updated my project to support “Compact diff report”. See the following description: