I'd like to know how to implement onauthreject rule when
authentication scheme is Windows Authentication.
In my environment, when user isn't authenticated by IIS, the browser receives a pop-up asking for credentials.
I have read the documentation about this configuration :
"Note: If a user authentication fails in NTLM authentication, the
authentication process continues until the browser stops it.
To resolve the issue, create the following redirect responses
that redirect the user to a custom page when the authentication fails:
Rule with onauthreject and onauthusernotfound
Response with Webagent-onreject-redirect"
I've tried to set it, but I still get the pop-up in the browser. Why ?
You still get a popup, because the use case differs than the one from documentation. Your use case is that the IIS cannot authenticate the User when you have configured Windows Authentication Scheme. But the one from documentation is related when user gets authenticated at the IIS level with Windows Autentication Scheme, but the Policy Server cannot authenticate it.
From the documentation, the configuration of onauthreject and onauthusernotfound
is related to the following use case :
"After successful authentication at the Windows level (the SPS library),
the Policy Server fails to find the user in the User Store, and as such,
the Web Agent will ask again and again the Windows credentials. The request
will go in loop. The browser page will be blank and no popup occurs. The
message "Page cannot be displayed" will be shown in the browser when you stop
manually this loop."
And it's to prevent that loop that the note has been added in the documentation.
KB : TEC1825795