I tried to reproduce your issue on my side usig the alert.html you shared. Yes, I could.
I executed thiis:
import org.openqa.selenium.By
import org.openqa.selenium.WebDriver
import org.openqa.selenium.WebElement
import com.kms.katalon.core.model.FailureHandling as FailureHandling
import com.kms.katalon.core.webui.driver.DriverFactory
import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI
WebUI.openBrowser("")
WebUI.navigateToUrl("http://localhost/alert.html") // I have a HTTP Server at http://localhost which makes the katalon project accessible as a web site. It does nothing special.
WebDriver wd = DriverFactory.getWebDriver();
WebElement elementOpen = wd.findElement(By.xpath("/html/body/button"));
elementOpen.click()
boolean present1 = WebUI.verifyAlertPresent(5, FailureHandling.STOP_ON_FAILURE)
// this line passed; it meas an Aleart is there.
println("present1 :" + present1)
WebUI.delay(3)
// we will check if the Aler is still there...
boolean present2 = WebUI.verifyAlertPresent(1, FailureHandling.CONTINUE_ON_FAILURE)
// Chrome+FF+Edge; this line failed; it means no Alert is there.
// Safari; thiis line passed; it means an Alert is thre. Safari seems to be a bit loose.
println("present2? :" + present2)
WebUI.closeBrowser()
on Chrome, Edge Chromium, FireFox, the 2nd WebUI.verifyAlertPresent() failed because the Alert has already disappeared automatically.
Interesting enough, on Safari, the 2nd WebUI.verifyAleartPresent() passed. I could see the Alert dialog stayed displayed on the screen.
So I guess that the newer version of security-conscious browsers (Chrome, Edge, FireFox) intentionally automatically closes Alert dialogs when a browser is controlled by software automation
like Selenium. I tried to search on Google for relevant articles on this issue, but I couldn’t find any.
I guess, Safari is outdated. Safari seems to be less security-conscious.
@Matthew_Loo
You may want to use Safari for this test. Safari seems to be your sole friend. 
But even Safari may change in future.