Tests pass when run from command line, but fail when run in Visual Studio build

I have a test suite collection that runs in Chrome, Firefox, Edge, and IE on Windows 10. When I run the collection from the command line, they all pass. When I run it through the Visual Studio build on the same machine with the same user, Chrome and Firefox pass, but Edge fails to open each time with the below error.

I’ve made sure that the MicrosoftWebDriver version matches my Edge version (both 16299). And since it runs successfully from the command line, I feel like the issue must lie with the VS integration. I attached an image of the VS integration options. The full argument is below the error message.

Error:

2018-05-04 20:24:42 - [TEST_CASE][FAILED] - Test Cases/Create_delete_group_activity_conversation: Test Cases/Create_delete_group_activity_conversation FAILED because (of) Unable to open browser with url: '' (Root cause: org.openqa.selenium.WebDriverException: Unable to parse remote response: Unknown errorBuild info: version: '3.7.1', revision: '8a0099a', time: '2017-11-06T21:07:36.161Z'
System info: host: 'xx', ip: '192.168.0.xx', os.name: 'Windows 10', os.arch: 'amd64', os.version: '10.0', java.version: '1.8.0_102'
Driver info: driver.version: CEdgeDriver)VS Options:Full argument:-runMode=console -projectPath="$(Build.SourcesDirectory)\Katalon\Projects\MBEWeb_Tests\Simple Generic Test\Simple Generic Test.prj" -retry=0 -reportFolder="Reports/Chrome, Firefox, IE, Edge Test Suite" -testSuiteCollectionPath="Test Suites/Chrome, Firefox, IE, Edge Test Collection" -noSplash

VSIntegration.PNG

I’ve looked into this a bit more and found that it’s not necessarily related to Visual Studio, but more just due to automated execution.

I wrote a batch file that executes the tests from the command line. When I double click on the batch file, the tests execute perfectly.

I then set up a task in the Windows Task Scheduler that executes the same batch file. When it runs, Edge fails to open with the above error message. It also runs headless for some reason (no CMD window, no browser windows, etc.).

Do I need to set the test suite up differently to handle being run via automation vs. being run manually? Or is there maybe a preference in Edge that disallows automated execution without a user logged in at the console?

Here’s my batch file:

cd "c:\Katalon\Katalon_Studio_Windows_64-5.4.1"
katalon.exe -runMode=console -projectPath="C:\Users\{file path here}\Simple Generic Test.prj" -retry=0 -reportFolder="Reports/Chrome, Firefox, IE, Edge Test Suite" -testSuiteCollectionPath="Test Suites/Chrome, Firefox, IE, Edge Test Collection" -noSplash -consoleLog
1 Like

I fancy, but can’t prove, that this is going to be permissions related, maybe even Group Policy settings.

If you find a solution, please report back so the KB grows B)

Just an update on this:

The issue ended up being resolved by logging in an interactive user before running the tests. Still feels like more of a workaround than a solution, but it works for now.

2 Likes

Bingo.

And thanks for reporting back, Axel.

1 Like

Jenkins has a similar issue:

1 Like