Symantec Access Management

Expand all | Collapse all

503 Service unavailable while resetting password via APS

Anon Anon

Anon AnonAug 24, 2015 12:49 PM

  • 1.  503 Service unavailable while resetting password via APS

    Posted Aug 23, 2015 05:37 PM

    I am using Apache 2.4 in  web server after installation of  webagent R12.52 we are  facing 503 service unavailable message while submitting request for password change from browser but we are able to change the password using SmCPW command.

     

    Any idea ?



  • 2.  Re: 503 Service unavailable while resetting password via APS

    Posted Aug 23, 2015 06:32 PM

    Hi mikegray_2015,

     

    Please confirm if the error 503 is returned when request is redirected to the CPW_PATH defined in the aps.cfg file. Is the URL formed accordingly?

     

    Please share with us the trace messages associated to the request from webagent trace.

     

    Best regards,

    Kelly



  • 3.  Re: 503 Service unavailable while resetting password via APS

    Posted Aug 23, 2015 06:40 PM

    I couldnt find any file with aps.cfg could you please tel me the correct path



  • 4.  Re: 503 Service unavailable while resetting password via APS

    Posted Aug 23, 2015 06:48 PM

    Hi mikegray_2015,

     

    aps.cfg file resides under <Policy Server>\bin directory.

     

    Please confirm if smaps.dll is under the same directory.

     

    Best regards,

    Kelly



  • 5.  Re: 503 Service unavailable while resetting password via APS

    Posted Aug 23, 2015 06:55 PM

    [08/23/2015][12:20:28][31704][2606753536][CSmLowLevelAgent.cpp:1200][AuthenticateUser][0000000000000000000000000d0d02aa-7bd8-55da1cfc-9b5fe700-22a3412ceba6][*170.220.58.92][][app.login.test.agent][/CPW/SmCPW][testuser][User 'uid=testuser,ou=people,o=example,o=dcx.com' is authenticated by Policy Server.]

    [08/23/2015][12:20:28][31704][2606753536][CSmResponseManager.cpp:193][ProcessResponses][0000000000000000000000000d0d02aa-7bd8-55da1cfc-9b5fe700-22a3412ceba6][*170.220.58.92][][app.login.test.agent][/CPW/SmCPW][testuser][Calling SM_WAF_HTTP_PLUGIN->ProcessResponses.]

    [08/23/2015][12:20:28][31704][2606753536][CSmHttpPlugin.cpp:2966][CSmHttpPlugin::ProcessResponses][0000000000000000000000000d0d02aa-7bd8-55da1cfc-9b5fe700-22a3412ceba6][*170.220.58.92][][app.login.test.agent][/CPW/SmCPW][testuser][Processing Authentication responses.]

    [08/23/2015][12:20:28][31704][2606753536][CSmResponseManager.cpp:231][ProcessResponses][0000000000000000000000000d0d02aa-7bd8-55da1cfc-9b5fe700-22a3412ceba6][*170.220.58.92][][app.login.test.agent][/CPW/SmCPW][testuser][SM_WAF_HTTP_PLUGIN->ProcessResponses returned SmSuccess.]

    [08/23/2015][12:20:28][31704][2606753536][CSmSessionManager.cpp:209][CSmSessionManager::CreateSession][0000000000000000000000000d0d02aa-7bd8-55da1cfc-9b5fe700-22a3412ceba6][*170.220.58.92][][app.login.test.agent][/CPW/SmCPW][testuser][Calling SM_WAF_HTTP_PLUGIN->CreateSession.]

    [08/23/2015][12:20:28][31704][2606753536][CSmHttpPlugin.cpp:1582][CSmHttpPlugin::CreateSession][0000000000000000000000000d0d02aa-7bd8-55da1cfc-9b5fe700-22a3412ceba6][*170.220.58.92][][app.login.test.agent][/CPW/SmCPW][testuser][Generated SMSESSION cookie.]

    [08/23/2015][12:20:28][31704][2606753536][CSmSessionManager.cpp:247][CSmSessionManager::CreateSession][0000000000000000000000000d0d02aa-7bd8-55da1cfc-9b5fe700-22a3412ceba6][*170.220.58.92][][app.login.test.agent][/CPW/SmCPW][testuser][SM_WAF_HTTP_PLUGIN->CreateSession returned SmSuccess.]

    [08/23/2015][12:20:28][31704][2606753536][CSmLowLevelAgent.cpp:2768][AuthorizeUser][0000000000000000000000000d0d02aa-7bd8-55da1cfc-9b5fe700-22a3412ceba6][*170.220.58.92][][app.login.test.agent][/CPW/SmCPW][testuser][User 'uid=testuser,ou=people,o=example,o=dcx.com' is authorized by Policy Server.]

    [08/23/2015][12:20:28][31704][2606753536][CSmResponseManager.cpp:193][ProcessResponses][0000000000000000000000000d0d02aa-7bd8-55da1cfc-9b5fe700-22a3412ceba6][*170.220.58.92][][app.login.test.agent][/CPW/SmCPW][testuser][Calling SM_WAF_HTTP_PLUGIN->ProcessResponses.]

    [08/23/2015][12:20:28][31704][2606753536][CSmHttpPlugin.cpp:3244][CSmHttpPlugin::ProcessResponses][0000000000000000000000000d0d02aa-7bd8-55da1cfc-9b5fe700-22a3412ceba6][*170.220.58.92][][app.login.test.agent][/CPW/SmCPW][testuser][Processing Authorization responses.]

    [08/23/2015][12:20:28][31704][2606753536][CSmHttpPlugin.cpp:3251][CSmHttpPlugin::ProcessResponses][0000000000000000000000000d0d02aa-7bd8-55da1cfc-9b5fe700-22a3412ceba6][*170.220.58.92][][app.login.test.agent][/CPW/SmCPW][testuser][Removing HTTP cache request headers.]

    [08/23/2015][12:20:28][31704][2606753536][CSmHttpPlugin.cpp:3343][CSmHttpPlugin::ProcessResponses][0000000000000000000000000d0d02aa-7bd8-55da1cfc-9b5fe700-22a3412ceba6][*170.220.58.92][][app.login.test.agent][/CPW/SmCPW][testuser][Setting custom HTTP header variable: 'HTTP_CPW_USER=LDAP://ldapsrv.example.com:2636 ldapsrv2.example.com:2636,ldapsrv2.example.com:2636 ldapsrv.example.com:2636/uid=testuser,ou=people,o=example,o=dcx.com']

    [08/23/2015][12:20:28][31704][2606753536][CSmResponseManager.cpp:231][ProcessResponses][0000000000000000000000000d0d02aa-7bd8-55da1cfc-9b5fe700-22a3412ceba6][*170.220.58.92][][app.login.test.agent][/CPW/SmCPW][testuser][SM_WAF_HTTP_PLUGIN->ProcessResponses returned SmSuccess.]

    [08/23/2015][12:20:28][31704][2606753536][CSmHighLevelAgent.cpp:791][ProcessRequest][0000000000000000000000000d0d02aa-7bd8-55da1cfc-9b5fe700-22a3412ceba6][*170.220.58.92][][app.login.test.agent][/CPW/SmCPW][testuser][AuthorizationManager returned SmYes, end new request.]

    [08/23/2015][12:20:28][31704][2606753536][CSmHighLevelAgent.cpp:911][ProcessRequest][0000000000000000000000000d0d02aa-7bd8-55da1cfc-9b5fe700-22a3412ceba6][*170.220.58.92][][app.login.test.agent][/CPW/SmCPW][testuser][End new request.]

    [08/23/2015][12:20:28][31704][2606753536][CSmLowLevelAgent.cpp:3567][ReportHealthData][][][][][][][Accumulating HealthMonitorCtxt.]

    [08/23/2015][12:20:29][31619][2722072320][CSmHighLevelAgent.cpp:321][ProcessRequest][0000000000000000000000000d0d02aa-7b83-55da1cfd-a23f8700-45f7176aab22][][][][][][Sta



  • 6.  Re: 503 Service unavailable while resetting password via APS

    Posted Aug 23, 2015 06:56 PM

    Please see the above log you can see it forming the url header



  • 7.  Re: 503 Service unavailable while resetting password via APS

    Posted Aug 23, 2015 07:00 PM

    hi wonsa03

     

    Also please note that this is working fine in lot of apache 2.2 with out any issue.



  • 8.  Re: 503 Service unavailable while resetting password via APS

    Posted Aug 23, 2015 07:22 PM

    Hi mikegray_2015,

     

    Thanks for the log snippet.

     

    It doesn't seem like the error is from webagent. Please check the Apache error log and let us know the error message associated to the request. Is the Apache on Unix or Windows platfrom?

     

    Best regards,

    Kelly



  • 9.  Re: 503 Service unavailable while resetting password via APS

    Posted Aug 23, 2015 07:32 PM

    wonsa03

     

    I could see only the below error in error log, any other chance?

     

    [Sun Aug 23 16:28:10.339215 2015] [authz_core:debug] [pid 31626:tid 139803718809344] mod_authz_core.c(809): [client 192.2.13.2:39296] AH01626: authorization result of Require all granted: granted

    [Sun Aug 23 16:28:10.339319 2015] [authz_core:debug] [pid 31626:tid 139803718809344] mod_authz_core.c(809): [client 192.2.13.2:39296] AH01626: authorization result of <RequireAny>: granted

    [Sun Aug 23 16:28:10.888214 2015] [authz_core:debug] [pid 31620:tid 139803634890496] mod_authz_core.c(809): [client 192.2.13.2:59339] AH01626: authorization result of Require all granted: granted

    [Sun Aug 23 16:28:10.888288 2015] [authz_core:debug] [pid 31620:tid 139803634890496] mod_authz_core.c(809): [client 192.2.13.2:59339] AH01626: authorization result of <RequireAny>: granted



  • 10.  Re: 503 Service unavailable while resetting password via APS

    Posted Aug 23, 2015 08:14 PM

    Hi mikegray_2015,

     

    The error seems to relate to the Apache's authentication settings:

    Why isn't Apache Basic authentication working? - Server Fault

     

    Best regards,

    Kelly



  • 11.  Re: 503 Service unavailable while resetting password via APS

    Posted Aug 23, 2015 08:39 PM

    hi wonsa03,

     

    But this error not related to my actual 503 issue.



  • 12.  Re: 503 Service unavailable while resetting password via APS

    Posted Aug 23, 2015 08:40 PM

    Hi mikegray_2015,

     

    Could you please help clarify couple of questions ?

     

    • Are you using custom change password form or the OOTB change password form provided by SmCPW.exe ?
    • Are you able to access the SmCPW in browser , e.g http://server.domain/CPW/SmCPW.exe ?
    • Does the SmCPW directory have execute privilege ?
    • Could you provide snippet of POST to SmCPW.exe from the web agent trace logs ?
    • Also, the Policy server trace snippet that you provided just captures the IsAuthenticated & IsAuthorized calls for /CPW/SmCPW/ realm. It doesn't capture the actual APS password change processing...This could be because the request might not have reached any further
    • If you could provide http header trace (e.g Fiddler trace ) that would be very useful

     

    Based on the response to above question, we might be able to provide further instructions in troubleshooting this issue.

     

    Cheers,

    Ujwol Shrestha



  • 13.  Re: 503 Service unavailable while resetting password via APS

    Posted Aug 23, 2015 09:21 PM
    Are you using custom change password form or the OOTB change password form provided by SmCPW.exe ?
    yes we are using the same

     

      [me@linuxdev bin]$ cd web

    [me@linuxdev web]$ ls

    APSAdmin  CPW  FPS  PWC  resources

    [me@linuxdev web]$ cd CPW

    [me@linuxdev CPW]$ ls

    resources  SmCPW

    [me@linuxdev CPW]$ cd ..

    [me@linuxdev web]$ cd  PWC

    [me@linuxdev PWC]$ ls

    PasswordChange.html  SmCPW  test.html  Warning

    [me@linuxdev PWC]$

     

     

    Are you able to access the SmCPW in browser , e.g http://server.domain/CPW/SmCPW.exe ?

     

      yes i am able to and am able to see the form for password reset

    old password
    new password
    confrim password
       
    Does the SmCPW directory have execute privilege ?
    yes
    Could you provide snippet of POST to SmCPW.exe from the web agent trace logs ?

      [08/23/2015][12:21:44][31619][2732562176][SmPluginUtilities.cpp:481][HandleCredCollectorReturn][0000000000000000000000000d0d02aa-7b83-55da1d48-a2df9700-19a819b8d0a9][*170.220.58.92][][app.login.test.agent][/CPW/SmCPW][][POST preservation, handling return from credential collector.]

    [08/23/2015][12:21:44][31619][2732562176][SmPluginUtilities.cpp:618][HandleCredCollectorReturn][0000000000000000000000000d0d02aa-7b83-55da1d48-a2df9700-19a819b8d0a9][*170.220.58.92][][app.login.test.agent][/CPW/SmCPW][][http response https://test-dev.dev.example.com/CPW/SmCPW]

    [08/23/2015][12:21:44][31619][2732562176][CSmCredentialManager.cpp:260][CSmCredentialManager::GatherAdvancedAuthCredentials][0000000000000000000000000d0d02aa-7b83-55da1d48-a2df9700-19a819b8d0a9][*170.220.58.92][][app.login.test.agent][/CPW/SmCPW][][SM_WAF_HTTP_PLUGIN->ProcessAdvancedAuthCredentials returned SmSuccess.]

    [08/23/2015][12:21:44][31626][2417936128][CSmHighLevelAgent.cpp:321][ProcessRequest][0000000000000000000000000d0d02aa-7b8a-55da1d48-901ec700-47ea79d3e867][][][][][][Start new request.]

    [08/23/2015][12:21:44][31626][2417936128][CSmResourceManager.cpp:75][CSmResourceManager::ProcessResource][0000000000000000000000000d0d02aa-7b8a-55da1d48-901ec700-47ea79d3e867][][][][][][Calling SM_WAF_HTTP_PLUGIN->ProcessResource.]

    [08/23/2015][12:21:44][31626][2417936128][SmApache24WebFilterCtxt.cpp:1709][CSmApache24WebFilterCtxt::SetP3PCompactPolicy][][][][][][][sP3PCompactPolicy: '']

    [08/23/2015][12:21:44][31626][2417936128][CSmHttpPlugin.cpp:399][CSmHttpPlugin::ProcessResource][0000000000000000000000000d0d02aa-7b8a-55da1d48-901ec700-47ea79d3e867][][][][][][Resolved HTTP_HOST: '.example.com'.]

    [08/23/2015][12:21:44][31626][2417936128][CSmHttpPlugin.cpp:5223][Entered CSmHttpPlugin::ResolveFQServerName sHost: ][][][][][][][.example.com]

    [08/23/2015][12:21:44][31626][2417936128][CSmHttpPlugin.cpp:490][CSmHttpPlugin::ProcessResource][0000000000000000000000000d0d02aa-7b8a-55da1d48-901ec700-47ea79d3e867][][][][][][Resolved hostname: '.example.com'.]

    [08/23/2015][12:21:44][31626][2417936128][CSmHttpPlugin.cpp:9844][CSmHttpPlugin::DoDNSLookup ][][][][][][][Entered Function server: .example.com, port: :80]

    [08/23/2015][12:21:44][31626][2417936128][CSmHttpPlugin.cpp:9920][CSmHttpPlugin::DoDNSLookup ][][][][][][][addrinfo lookup failed Name or service not known]

    [08/23/2015][12:21:44][31626][2417936128][CSmHttpPlugin.cpp:9929][CSmHttpPlugin::DoDNSLookup ][][][][][][][Leaving Function]

    [08/23/2015][12:21:44][31626][2417936128][CSmHttpPlugin.cpp:509][CSmHttpPlugin::ProcessResource][0000000000000000000000000d0d02aa-7b8a-55da1d48-901ec700-47ea79d3e867][][][][][][Resolved agentname: 'app.login.test.agent'.]

    [08/23/2015][12:21:44][31626][2417936128][CSmHttpPlugin.cpp:5561][CSmHttpPlugin::ResolveClientIp][0000000000000000000000000d0d02aa-7b8a-55da1d48-901ec700-47ea79d3e867][][][app.login.test.agent][][][Failed To Resolve ClientIP from CustomHeader.]

    [08/23/2015][12:21:44][31626][2417936128][CSmHttpPlugin.cpp:657][CSmHttpPlugin::ProcessResource][0000000000000000000000000d0d02aa-7b8a-55da1d48-901ec700-47ea79d3e867][][][app.login.test.agent][][][Resolved URL: '/siteminderagent/forms/deny.fcc'.]

    [08/23/2015][12:21:44][31626][2417936128][CSmHttpPlugin.cpp:5655][CSmHttpPlugin::AutoAuthorizedUrl][][][][][][][Auto-authorizing resource, matches IgnoreExt filter.]

    [08/23/2015][12:21:44][31626][2417936128][CSmHttpPlugin.cpp:690][CSmHttpPlugin::ProcessResource][0000000000000000000000000d0d02aa-7b8a-55da1d48-901ec700-47ea79d3e867][][][app.login.test.agent][/siteminderagent/forms/deny.fcc][][Autoauthorizing URL : 'https://.example.com/siteminderagent/forms/deny.fcc' , Method: 'GET' ]

    [08/23/2015][12:21:44][31626][2417936128][CSmHttpPlugin.cpp:773][CSmHttpPlugin::ProcessResource][0000000000000000000000000d0d02aa-7b8a-55da1d48-901ec700-47ea79d3e867][][][app.login.test.agent][/siteminderagent/forms/deny.fcc][][Resolved METHOD: 'GET'.]

    [08/23/2015][12:21:44][31626][2417936128][CSmHttpPlugin.cpp:826][CSmHttpPlugin::ProcessResource][0000000000000000000000000d0d02aa-7b8a-55da1d48-901ec700-47ea79d3e867][][][app.login.test.agent][/siteminderagent/forms/deny.fcc][][Resolved cookie domain: '.dev.example.com'.]

    [08/23/2015][12:21:44][31626][2417936128][CSmResourceManager.cpp:112][CSmResourceManager::ProcessResource][0000000000000000000000000d0d02aa-7b8a-55da1d48-901ec700-47ea79d3e867][][][app.login.test.agent][/siteminderagent/forms/deny.fcc][][SM_WAF_HTTP_PLUGIN->ProcessResource returned SmSuccess.]

    [08/23/2015][12:21:44][31626][2417936128][CSmSessionManager.cpp:82][CSmSessionManager::EstablishSession][0000000000000000000000000d0d02aa-7b8a-55da1d48-901ec700-47ea79d3e867][][][app.login.test.agent][/siteminderagent/forms/deny.fcc][][Calling SM_WAF_HTTP_PLUGIN->EstablishSession.]

    [08/23/2015][12:21:44][31626][2417936128][CSmSessionManager.cpp:126][CSmSessionManager::EstablishSession][0000000000000000000000000d0d02aa-7b8a-55da1d48-901ec700-47ea79d3e867][][][app.login.test.agent][/siteminderagent/forms/deny.fcc][][SM_WAF_HTTP_PLUGIN->EstablishSession returned SmNoAction.]

    [08/23/2015][12:21:44][31626][2417936128][CSmHighLevelAgent.cpp:395][ProcessRequest][0000000000000000000000000d0d02aa-7b8a-55da1d48-901ec700-47ea79d3e867][][][app.login.test.agent][/siteminderagent/forms/deny.fcc][][ProtectionManager returned SmNo, end new request.]

    [08/23/2015][12:21:44][31626][2417936128][CSmLowLevelAgent.cpp:3567][ReportHealthData][][][][][][][Accumulating HealthMonitorCtxt.]

    [08/23/2015][12:21:44][31626][2417936128][CSmHighLevelAgent.cpp:960][ProcessAdvancedAuthentication][0000000000000000000000000d0d02aa-7b8a-55da1d48-901ec700-47ea79d3e867][][][][][][Start new request.]

    :

     

     

    If you could provide http header trace (e.g Fiddler trace ) that would be very useful

    provide me email address i will send you wireshark capture

     

     

    Based on the response to above question, we might be able to provide further instructions in troubleshooting this issue.



  • 14.  Re: 503 Service unavailable while resetting password via APS

    Posted Aug 24, 2015 01:31 AM

    Hi,

     

    The web agent trace capture was incomplete.

    The POST to SMCPW.exe was  [ProcessId][ThreadID]= [31619][2732562176]

    So we will need complete transacation for this thread.

     

    Other threads were for different request.

    Also troubleshooting it with Wireshark will be quite difficult as it not only capture http traffic but the whole TCP traffic.

     

    Fiddler trace would be ideal for this:

    Fiddler free web debugging proxy

     

    Cheers,

    Ujwol Shrestha



  • 15.  Re: 503 Service unavailable while resetting password via APS

    Posted Aug 24, 2015 02:10 AM
      |   view attached

    Ujwol

     

    please see that attached complete log for a new id

    Attachment(s)

    zip
    sitetest.zip   44 KB 1 version


  • 16.  Re: 503 Service unavailable while resetting password via APS

    Posted Aug 24, 2015 12:49 PM

    any idea?



  • 17.  Re: 503 Service unavailable while resetting password via APS

    Posted Aug 24, 2015 07:52 PM

    Hi Mike,

     

    Some configuration isn't right.

     

    From your log, the change password POST is happening to url  '/CPW/SmCPW' instead of Resolved URL: '/CPW/SmCPW.exe'

    We will need full set of following logs to troubleshoot this :

     

    1. Fiddler logs ( to understand the redirection and cookies set if any )

    2. Policy server trace logs ( to understand the policy server side of processing of the password change request)

    3. APS.cfg ( to understand your APS setup)

    4. Web agent logs and traces ( to understand the web agent side of processing of the password change request)

     

    All of the above logs needs to be time synchronized as we will need to correlate the transactions.

     

    If you are not comfortable sharing logs in the community, request you to create a new support case and let me know the case # and we can take it from there.

     

    Regards,

    Ujwol Shrestha



  • 18.  Re: 503 Service unavailable while resetting password via APS

    Posted Aug 25, 2015 01:02 AM

    Ujwol

     

    00182296: WebAgent issue With Apache 2.4  please see the case. Please note that the case is closed but the issue still persist



  • 19.  Re: 503 Service unavailable while resetting password via APS

    Posted Aug 25, 2015 05:12 PM

    Have created another ticket but poor support form CA



  • 20.  Re: 503 Service unavailable while resetting password via APS
    Best Answer

    Posted Aug 28, 2015 01:09 AM

    Hi Sanoj,

     

    Thank you for your patience in waiting for the solution from CA on this case.

    We have finally been able to figure out the root cause of this issue and also provide a solution.

     

    I am updating our findings in this thread as well for the benefit of our fellow community members :

     

    Root Cause :

    ===========

    The issue was because of new functionality in added Apache 2.4.x in couple of modules such as mod_cgi,mod_cgid etc , due to which the HTTP headers containing invalid characters (including underscores) are now silently dropped.

    This is going to be problem when we have ACO parameter LegacyVariables=YES (which is default) in which case all our default HTTP headers will have underscores e.g HTTP_SM_USER, HTTP_USER_DN etc

    Reference : http://httpd.apache.org/docs/trunk/new_features_2_4.html

     

     

    For this particular change password use case, the one that was a problem was HTTP_SM_USER header which is required for POST to Change Password CGI to work.

    As this HTTP_SM_USER was getting dropped by the mod_cgi/mod_cgid module in Apache 2.4.x, we were falling into this condition and hence the “503 Service Unavailable” error.

     

    Solution:

    =======

    The solution is to set LegacyVariables=No to work with Apache 2.4.x.

    With LegacyVariables set to No , all the default SiteMinder HTTP headers are set without underscore eg. SMUSER,SMUSERDN etc. and hence there will not be any interference with the new functionality in Apache 2.4.x

     

    Regards,

    Ujwol Shrestha



  • 21.  Re: 503 Service unavailable while resetting password via APS

    Posted Aug 28, 2015 01:49 AM

    Ujwol,

     

    Thanks for the solution and its fixed our issue with APS . Appreciate your dedication,support, technical knowledge and prompt response.