Runtime Engine not matching Studio test results

Product Forums Katalon Studio
,
1

I currently have a test suite where all tests pass successfully when running through Katalon Studio. When the Runtime Engine runs the suite, all the tests fail. Looking through the reports, it seems as if what is being returned from the API is different depending which project runs the tests.

For example, getting a state’s information in Katalon Studio would look like [state:AK, description:Alaska, country:USA], but in Runtime Engine, it’s [country:USA, description:Alaska, state:AK].
This is causing asserts to fail due to the data being reordered.

I don’t know if this is a bug with the Runtime Engine or something I can correct, but any guidance would be greatly appreciated. Thanks!

2

How do you check the result?
By comparing whith a string?
If you parse the json and assert by key:value the order doesnt matter.

3

This is how I’ve gotten around the issue for now, but the Runtime Engine should not be reordering responses.