Nested Maps: Find a value without key specification

[yakovlieva.olena]

Hello,

Could you please help me to understand how to check if JSON contains some value for any key
response:

[
  {
    "id":"76D12D26",
    "name":"FM-10",
    "machine_name":"sssb",
    "history":[
      {
        "track":{
          "id":"826B2557",
          "name":"8"          
        },
        "output":{
          "wear":6,
          "id":"5F4B9864",
          },
        "up_machine_name":"sssa"
      },
      {
        "track":{
          "id":"D5D38891",
          "name":"8",
          },
        "output":{
          "wear":1,
          "id":"66CB401A"
           },
        "up_machine_name":"sssb"
      }],
    "refurbishments":[
      {
        "id":"AE019767",
        "type":"Double"       
      }]
  },
  {
    "id":"BFB38570",
    "name":"FM-10.1",
    "history":[
      {
        "track":{
          "id":"1189D324",
          "name":"6"         
        },
        "output":{
          "wear":6,
          "id":"5F4B9864"
        },
        "up_machine_name":"sssb"
      },
      {
        "track":{
          "id":"623F14BA",
          "name":"6"
           },
        "output":{
          "wear":1,
          "id":"66CB401A"          
        },
        "extraordinary":true,
        "up_machine_name":"sssf"
      }],
    "refurbishments":[
      {
        "id":"6C28A13B",
        "refurbishment_event_id":"5809C9FE"        
      }]
  }]

I try, but I can’t understand what is wrong with type of data. I try nested closures. The keys are checked only on the upper level.

JsonSlurper slurper = new JsonSlurper()
def m = slurper.parseText(response)

m.eachWithIndex { entry, i -> 
	println i
	println "entry $entry"
	println entry.size()
	entrySize1 = entry.size()
	def result3 = entry.findAll{entry.value == "sssf" || entry.value == "sssa"  }
	println " result3 $result3"

	
	entry.eachWithIndex { entry_2 , i1 ->
		println " $i1 INSIDE JSON" 
		println " $entry_2"
		println entry_2.getClass()
		println entry_2.getMetaClass()
		def result4 = entry_2.findAll{entry_2.value == "sssf" || entry_2.value == "sssa"  }
		println " result4 $result4"
		
				
		entry_2.eachWithIndex { entry_3 , i3 ->
			println " $i3 INSIDE JSON  3"
			println " $entry_3"
			println entry_3.getClass()
			println entry_3.getMetaClass()
			def result5 = entry_3.findAll{entry_3.value == "sssf" || entry_3.value == "sssa"  }
			println " result5 $result5"
		}
	}
		
}

logs

2024-07-10 12:48:13.467 INFO  c.k.katalon.core.main.TestCaseExecutor   - --------------------
2024-07-10 12:48:13.477 INFO  c.k.katalon.core.main.TestCaseExecutor   - START Test Cases/Tools/Web plus API/in work/test - Copy - Copy
2024-07-10 12:48:14.960 DEBUG testcase.test - Copy - Copy              - 1: response = "[
  {
    "id":"76D12D26",
    "name":"FM-10",
	"machine_name":"sssb",
    "history":[
      {
        "track":{
          "id":"826B2557",
          "name":"8"          
        },
        "output":{
          "wear":6,
          "id":"5F4B9864",
          },
        "up_machine_name":"sssa"
      },
      {
        "track":{
          "id":"D5D38891",
          "name":"8",
          },
        "output":{
          "wear":1,
          "id":"66CB401A"
           },
        "up_machine_name":"sssb"
      }],
    "refurbishments":[
      {
        "id":"AE019767",
        "type":"Double"       
      }]
  },
  {
    "id":"BFB38570",
    "name":"FM-10.1",
	"history":[
      {
        "track":{
          "id":"1189D324",
          "name":"6"         
        },
        "output":{
          "wear":6,
          "id":"5F4B9864"
        },
        "up_machine_name":"sssb"
      },
      {
        "track":{
          "id":"623F14BA",
          "name":"6"
           },
        "output":{
          "wear":1,
          "id":"66CB401A"          
        },
        "extraordinary":true,
        "up_machine_name":"sssf"
      }],
    "refurbishments":[
      {
        "id":"6C28A13B",
        "refurbishment_event_id":"5809C9FE"        
      }]
  }]"
2024-07-10 12:48:14.968 DEBUG testcase.test - Copy - Copy              - 2: slurper = new groovy.json.JsonSlurper()
2024-07-10 12:48:15.026 DEBUG testcase.test - Copy - Copy              - 3: m = slurper.parseText(response)
2024-07-10 12:48:15.067 DEBUG testcase.test - Copy - Copy              - 4: map1 = "[{"disc_name":"FM-10.1"}]"
2024-07-10 12:48:15.086 DEBUG testcase.test - Copy - Copy              - 5: m.eachWithIndex({ java.lang.Object entry, java.lang.Object i -> ... })
0
entry [id:76D12D26, name:FM-10, machine_name:sssb, history:[[track:[id:826B2557, name:8], output:[wear:6, id:5F4B9864], up_machine_name:sssa], [track:[id:D5D38891, name:8], output:[wear:1, id:66CB401A], up_machine_name:sssb]], refurbishments:[[id:AE019767, type:Double]]]
5
 result3 [:]
 0 INSIDE JSON
 id=76D12D26
class java.util.LinkedHashMap$Entry
org.codehaus.groovy.runtime.HandleMetaClass@32091c14[groovy.lang.MetaClassImpl@32091c14[class java.util.LinkedHashMap$Entry]]
 result4 []
 0 INSIDE JSON  3
 id=76D12D26
class java.util.LinkedHashMap$Entry
org.codehaus.groovy.runtime.HandleMetaClass@32091c14[groovy.lang.MetaClassImpl@32091c14[class java.util.LinkedHashMap$Entry]]
 result5 []
 1 INSIDE JSON
 name=FM-10
class java.util.LinkedHashMap$Entry
org.codehaus.groovy.runtime.HandleMetaClass@32091c14[groovy.lang.MetaClassImpl@32091c14[class java.util.LinkedHashMap$Entry]]
 result4 []
 0 INSIDE JSON  3
 name=FM-10
class java.util.LinkedHashMap$Entry
org.codehaus.groovy.runtime.HandleMetaClass@32091c14[groovy.lang.MetaClassImpl@32091c14[class java.util.LinkedHashMap$Entry]]
 result5 []
 2 INSIDE JSON
 machine_name=sssb
class java.util.LinkedHashMap$Entry
org.codehaus.groovy.runtime.HandleMetaClass@32091c14[groovy.lang.MetaClassImpl@32091c14[class java.util.LinkedHashMap$Entry]]
 result4 []
 0 INSIDE JSON  3
 machine_name=sssb
class java.util.LinkedHashMap$Entry
org.codehaus.groovy.runtime.HandleMetaClass@32091c14[groovy.lang.MetaClassImpl@32091c14[class java.util.LinkedHashMap$Entry]]
 result5 []
 3 INSIDE JSON
 history=[{track={id=826B2557, name=8}, output={wear=6, id=5F4B9864}, up_machine_name=sssa}, {track={id=D5D38891, name=8}, output={wear=1, id=66CB401A}, up_machine_name=sssb}]
class java.util.LinkedHashMap$Entry
org.codehaus.groovy.runtime.HandleMetaClass@32091c14[groovy.lang.MetaClassImpl@32091c14[class java.util.LinkedHashMap$Entry]]
 result4 []
 0 INSIDE JSON  3
 history=[{track={id=826B2557, name=8}, output={wear=6, id=5F4B9864}, up_machine_name=sssa}, {track={id=D5D38891, name=8}, output={wear=1, id=66CB401A}, up_machine_name=sssb}]
class java.util.LinkedHashMap$Entry
org.codehaus.groovy.runtime.HandleMetaClass@32091c14[groovy.lang.MetaClassImpl@32091c14[class java.util.LinkedHashMap$Entry]]
 result5 []
 4 INSIDE JSON
 refurbishments=[{id=AE019767, type=Double}]
class java.util.LinkedHashMap$Entry
org.codehaus.groovy.runtime.HandleMetaClass@32091c14[groovy.lang.MetaClassImpl@32091c14[class java.util.LinkedHashMap$Entry]]
 result4 []
 0 INSIDE JSON  3
 refurbishments=[{id=AE019767, type=Double}]
class java.util.LinkedHashMap$Entry
org.codehaus.groovy.runtime.HandleMetaClass@32091c14[groovy.lang.MetaClassImpl@32091c14[class java.util.LinkedHashMap$Entry]]
 result5 []
1
entry [id:BFB38570, name:FM-10.1, history:[[track:[id:1189D324, name:6], output:[wear:6, id:5F4B9864], up_machine_name:sssb], [track:[id:623F14BA, name:6], output:[wear:1, id:66CB401A], extraordinary:true, up_machine_name:sssf]], refurbishments:[[id:6C28A13B, refurbishment_event_id:5809C9FE]]]
4
 result3 [:]
 0 INSIDE JSON
 id=BFB38570
class java.util.LinkedHashMap$Entry
org.codehaus.groovy.runtime.HandleMetaClass@32091c14[groovy.lang.MetaClassImpl@32091c14[class java.util.LinkedHashMap$Entry]]
 result4 []
 0 INSIDE JSON  3
 id=BFB38570
class java.util.LinkedHashMap$Entry
org.codehaus.groovy.runtime.HandleMetaClass@32091c14[groovy.lang.MetaClassImpl@32091c14[class java.util.LinkedHashMap$Entry]]
 result5 []
 1 INSIDE JSON
 name=FM-10.1
class java.util.LinkedHashMap$Entry
org.codehaus.groovy.runtime.HandleMetaClass@32091c14[groovy.lang.MetaClassImpl@32091c14[class java.util.LinkedHashMap$Entry]]
 result4 []
 0 INSIDE JSON  3
 name=FM-10.1
class java.util.LinkedHashMap$Entry
org.codehaus.groovy.runtime.HandleMetaClass@32091c14[groovy.lang.MetaClassImpl@32091c14[class java.util.LinkedHashMap$Entry]]
 result5 []
 2 INSIDE JSON
 history=[{track={id=1189D324, name=6}, output={wear=6, id=5F4B9864}, up_machine_name=sssb}, {track={id=623F14BA, name=6}, output={wear=1, id=66CB401A}, extraordinary=true, up_machine_name=sssf}]
class java.util.LinkedHashMap$Entry
org.codehaus.groovy.runtime.HandleMetaClass@32091c14[groovy.lang.MetaClassImpl@32091c14[class java.util.LinkedHashMap$Entry]]
 result4 []
 0 INSIDE JSON  3
 history=[{track={id=1189D324, name=6}, output={wear=6, id=5F4B9864}, up_machine_name=sssb}, {track={id=623F14BA, name=6}, output={wear=1, id=66CB401A}, extraordinary=true, up_machine_name=sssf}]
class java.util.LinkedHashMap$Entry
org.codehaus.groovy.runtime.HandleMetaClass@32091c14[groovy.lang.MetaClassImpl@32091c14[class java.util.LinkedHashMap$Entry]]
 result5 []
 3 INSIDE JSON
 refurbishments=[{id=6C28A13B, refurbishment_event_id=5809C9FE}]
class java.util.LinkedHashMap$Entry
org.codehaus.groovy.runtime.HandleMetaClass@32091c14[groovy.lang.MetaClassImpl@32091c14[class java.util.LinkedHashMap$Entry]]
 result4 []
 0 INSIDE JSON  3
 refurbishments=[{id=6C28A13B, refurbishment_event_id=5809C9FE}]
class java.util.LinkedHashMap$Entry
org.codehaus.groovy.runtime.HandleMetaClass@32091c14[groovy.lang.MetaClassImpl@32091c14[class java.util.LinkedHashMap$Entry]]
 result5 []
2024-07-10 12:48:15.444 INFO  c.k.katalon.core.main.TestCaseExecutor   - END Test Cases/Tools/Web plus API/in work/test - Copy - Copy

Thank you in advance for your answers.

[system]

Hi there,

Thank you very much for your topic. Please note that it may take a little while before a member of our community or from Katalon team responds to you.

Thanks!

[kazurayam]

@yakovlieva.olena

What is your question?
I don’t see what you want to ask to us.

[yakovlieva.olena]

@kazurayam thank you for your answer.
The question is:
how to check that following values are present (contains) among the all values of a json response

“sssb”
“sssa”
“sssf”

[kazurayam]

@yakovlieva.olena

The question is:how to check that following values are present (contains) among the all values of a json response

“sssb”

If your question is really as what you described, you do not need to process the response text as a JSON. You can process the response as a plain text without any grammar. The following Groovy code will do what you want.

String response = '''
  {
    "id":"76D12D26",
    "name":"FM-10",
	"machine_name":"sssb",
    "history":[
      {
        "track":{
          "id":"826B2557",
          "name":"8"          
        },
        "output":{
          "wear":6,
          "id":"5F4B9864",
          },
        "up_machine_name":"sssa"
      },
...
'''

boolean result = response.contains("sssb")
assert result == true

This approach is easy to implement. It is good enough to solve your problem.

[kazurayam]

If you want do something more sophisticated in a JSON-grammar-aware fashion, then you should learn a technical domain of “making query against JSON” or “validation against schema” .

See the following post for Query languages for JSON:

Sample usage of BrowserMob Proxy in Katalon Studio

How to post-process the HAR file as JSON using Jayway JsonPath Additionally, I wanted to automate verifying the content of HAR file. I expect that the sample.har file contains a record of a HTTP request to a url that ends with jquery.min.js. { "log": { ... "entries": [ ... { "request": { "method": "GET", "url": "https://cdnjs.cloudflare.com/ajax/libs/jquery/1.11.3/jquery.min.js", ... } } ... ] } } How can I …

See the following post for JSON Schema:

How to validate the JSON response with expected JSON schema

I have swagger JSON schema URL, when I get JSON response from the server, I need to validate with the JSON schema to match with the expected schema. Is there any sample code? Something similar to this https://docs.optimizory.com/display/vrest/Schema+Validation

As you see, these requires a seasoned skill of programming and your efforts.

It is up to you how much of sophistication you would put into your work.

[yakovlieva.olena]

@kazurayam thank you for your answer.
the JSON response that I need to check is about 60000 lines. So I can’t use String methods.
That is why I tried to apply some filtering on parsed JSON.

[kazurayam]

@yakovlieva.olena

the JSON response … is about 60000 lines. So I can’t use String methods.

Why not?

You wrote you just want to see if a string "sssb" is contained anywhere in the text. Then, I believe, the test string.contains("sssb") is the best answer.

If this is not what you want to achieve, then you should refine your problem analysis. You need to specify in more detail what the software should do. Unless the specification is clearly defined, the guys in this forum wouldn’t have any idea what to suggest.

---

Yes, the size of input JSON will matter. Therefore, I think, a simple string.contains("sssb") test is most appropriate.

Let me assume the average length of lines in the response JSON to be 50 characters, then 60000 lines will make 3 mega characters.

If you try to parse the text as a JSON and to make structured queries, it will be a pretty CPU-hungry and time-consuming processing.

However, if you process it as a single plain text, then the test string.contains("sssb") will finish fast enough.

[kazurayam]

If you desperately want to parse the 60000 lines of JSON to filter the portion you want, then you should NOT use JsonSlurper, which could be slow and memory-hungry.

Rather, you should use a “Streaming API” for JSON. For example,

Jackson Streaming API | Baeldung

This could be fast and lightweighted.

[kazurayam]

I wrote an example of using Jackson Streaming API to process a HAR file.

Sample usage of BrowserMob Proxy in Katalon Studio

Process HAR using Jackson Streaming API How large the sample.har is? $ wc sample.har 4152 10745 1017938 sample.har The sample.har files has 4152 lines, 10745 words, 1017938 characters. Well, it’s large. Now I want to pick up all URL strings contained in the sample.har file. The list of URLs should be unique, sorted alphabetically. How can I do it? Jackson Streaming API is an ideal solution for this problem. I made a Test Cases/process_har_with_streaming_api import java.nio.file.Path impor…

[kazurayam]

My code above took 25 seconds to process a JSON file of 4152 lines. Assuming the same speed, the code will take 360 seconds to process a file of 60000 lines.

… Too long. I can’t wait for 6 minutes.

If I were given with a JSON of 60000 lines, I would ask my boss if it is possible to reduce the size of JSON somehow.

[
  {
    "id":"76D12D26",
    "name":"FM-10",
    "machine_name":"sssb",
    "history":[
      {
        "track":{
          "id":"826B2557",
          "name":"8"          
        },
        "output":{
          "wear":6,
          "id":"5F4B9864",
          },
        "up_machine_name":"sssa"
      },
      {
        "track":{
          "id":"D5D38891",
          "name":"8",
          },
        "output":{
          "wear":1,
          "id":"66CB401A"
           },
        "up_machine_name":"sssb"
      }],
    "refurbishments":[
      {
        "id":"AE019767",
        "type":"Double"       
      }]
  },
   ...

This seems to be a result set of DB query. Then it should be possible to reduce the size by specifying some appropriate “WHERE” conditions. For example, @yakovlieva.olena should be able to write a DB query that selects the records which contain:

“sssb”
“sssa”
“sssf”

as the value somewhere.

The DB would work far faster than a JSON text processor.

[mohit.kumar]

boolean result = response.contains(“sssb”)
assert result == true

I agree with @kazurayam that using ‘contains’ is the best solution for this situation.

[wiky.hendra]

just an idea.
Rather than checking all of JSON text, is it better for taking (ws.getelementpropertyvalue) machine_name and up_machine_name value first, then check with contains ?
I have these problems but i solved it a long way.

[albert.vu]

Hi there @yakovlieva.olena,

Just checking in to see if you have been able to find a solution to your question or not based on the comments in this thread. Thanks

[yakovlieva.olena]

yes, for now the problem is solved similar as you described:

ResponseObject response = WSResponseManager.getInstance().getCurrentResponse()

JsonSlurper slurper = new JsonSlurper()
def m = slurper.parseText(response.getResponseBodyContent())

theNode1 = m.refurbishments
theNode2 = m.history.up_machine_name
theNode1_str =  theNode1.toString()
theNode2_str =  theNode2.toString()
refurbishments = (theNode1_str.contains('XXXX')) ?  KeywordUtil.markFailed ('discs call contains in refurbishments XXXX values') : WebUI.comment("discs call does NOT contains XXXX  value") 
history = (theNode2_str.contains('XXXX')) ? KeywordUtil.markFailed ('disc call contains in history XXXX values') : WebUI.comment("discs call does NOT contains XXXX  value")

For now I can’t invest more time to perform this check in more sophisticated way. @kazurayam thank you very much for your advices. I plan to rework my current temporary solution using one of your suggestions.

[kazurayam]

@yakovlieva.olena

The question is:
how to check that following values are present (contains) among the all values of a json response

“sssb”
“sssa”
“sssf”

I think that I have found out a way to implement the logic that @yakovlieva.olena desired.

I have revised my previous post

Sample usage of BrowserMob Proxy in Katalon Studio

I have published a GitHub project (updated at 24 July 2024) Background In the Katalon User Forum, there is a Frequently Asked Questions. I want to do a Web UI test using browsers, and at the same time I want to record the payload of HTTP Requests and Responses exchanged between a browser and servers. I want to save the information into a local file and later reuse it. How can I do it? Katalon Studio does not support recording the payload of HTTP messages exchanged between a browser and …

Now this post includes a description how to transform a large JSON into a smaller one while specifying which entries to select up and how to drop the rest off. You can specify the selection criteria by short declarative expressions, like this:

    // I am interested in a HTTP request of which URL contains a string ".jquery.min.js"
	Filter filter1 = Filter.filter(
		Criteria.where("request.url")
				.regex(Pattern.compile('.*jquery\\.min\\.js')));

	// I am also interested in  "fontawesome-webfont.woff2"
	Filter filter2 = Filter.filter(
		 Criteria.where("request.url")
		 		.regex(Pattern.compile('.*fontawesome\\-webfont\\.woff2.*')));

	Filter filter = filter1.or(filter2)

The filtering is implemented on top of the Jayway JsonPath library.

Of course, my project is not an immediate answer to the case of @yakovlieva.olena. However, I believe that @yakovlieva.olena can reuse my design to solve his/her issue.

[kazurayam]

@yakovlieva.olena

among the all values of a json response

If @yakovlieva.olena has multiple keys that could have value that match “sssb”, “sssa”, “sssf”, then he/she would want to create multiple instances of Filter class. Then he/she want to concatenate them into one using .or() method. For example, the following code shows the case to apply 2 Json Paths:

• $[?(.machine_name =~ /sss[abf]/)]
• $[?(.history[*].up_machine_name =~ /sss[abf]/)]

Closure cls = { Path inputJson ->
    Filter filter1 = Filter.filter(Criteria.where("machine_name").regex(Pattern.compile('sss[abf]')));
    Filter filter2 = Filter.filter(
        Criteria.where("['history'][*]['up_machine_name']").regex(Pattern.compile('sss[abf]')));

    Filter filter = filter1.or(filter2)

    List<Map<String, Object>> result =
		JsonPath.parse(Files.newInputStream(inputJson))
			.read("\$[?]", filter)

	return result
}

Do you have more keys to look at? OK. You can create as many com.jayway.jsonpath.Filer objects as you need, and concatenate them to form a single instance of Fiter using Filter.or(Filter) method.

See a sample project

[albert.vu]

Hi @yakovlieva.olena,

Could you help to check out Kazuyaram’s last two comments to see if they may be able to help you?

And, if yes, then don’t forget to mark helpful replies as a solution so that others who may be asking similar questions / running into similar problems can find the solution as well!