Symantec Access Management

  • 1.  smpolicysrv service constant at 162% server CPU

    Posted May 03, 2016 05:48 PM

    Hello,

     

    We are in the process of upgrading our SiteMinder r12.0 SP3 to r12.52 SP1.  We are doing this in a parallel mode so we first installed the r12.52 policy server and then export/import the policy data and the policy server keys.  Our new r12.52 server has been up and running for a couple months now in all of our apps in the DEV environment pointing to the new policy server while under going testing.  Recently we noticed that the smpolicysrv service is using about 160% of the server CPU (duel core VM), but there does not seem to be any performance impact from the policy server.

     

    We have an active support case with CA on this but we are not confident in getting a resolution from CA support on this.  As soon as I start up the policy server, the smpolicysrv process will immediately jump to +160% and will remain between 161 - 163% until regardless of the application traffic loads with the web agent making requests to the policy server.  Again, despite the high server CPU utilization, the performance of the policy server does not seem to be impacted.  Our web application authentication/authorization and federation services seems to be processing at normal performance level, but due to the fact that the server CPU is unreasonably high, we are not comfortable enough with this to move our upgrade effort onto the next environment until this high CPU utilization is resolved or explained.

     

    We even restored our policy server back to the default installation state with no policy data other then the default XPS import after the policy server installation to ruled out possible policy data/configuration causing this issue, but even with no policy data/configuration the smpolicysrv still chunk away at about 163% CPU utilization.  The hardware is a Linux VM duel core processor with plenty of processing power.  The only hardware spec difference between this new r12.52 Linux VM compared to the existing r12.0 Linux VM is that the r12.0 VM uses a NAS file system while the new r12.52 Linux VM uses local file system.  Below are some more specifics on our environment:

     

    Server/OS = RHEL 2.6.32-573.18.1.el6.x86_64

    Policy server = r12.52 SP1 CR04

    Policy Store = CA Directory Server r12.0 SP17

     

    Any input or suggestions for troubleshooting would be much greatly appreciated.

     

    Thank you!



  • 2.  Re: smpolicysrv service constant at 162% server CPU
    Best Answer

    Posted May 04, 2016 09:49 AM

    Hi,

     

    This behavior could be caused due to, by some reason, having the named pipes owned by a different user in /tmp folder than the user who started the Policy Server. So, for example, if the files are owned by root, and the Policy Server was started by smuser, the process is not able to write in the pipes due to a lack of access.

     

    You could verify this, and if this is the case, you could remove the pipes as follow:

    1. Stop the Policy Server.
    2. Remove the /tmp/GCL-SiteMinder* and /tmp/snrrpni* pipe files.
    3. Start the Policy Server.

    Then, the memory usage should be back to normal.

     

    Please, let me know if this solves the issue, and also please mark my response as Correct so it can help others.

     

    Thanks



  • 3.  Re: smpolicysrv service constant at 162% server CPU

    Posted May 05, 2016 12:10 PM

    Albert,

     

    You are absolutely correct.. thank YOU!

     

    The GCL-SiteMinder-A.pipe and GCL-SiteMinder-B.pipe were somehow owned by my own Linux server login account instead of the "smuser" account.  After stopping the policy server and deleting these files then starting back up, the files are then created again but is not owned by "smuser" and this issue with the smpolicysrv service using high server CPU is no longer there.

     

    Again, thank you!



  • 4.  Re: smpolicysrv service constant at 162% server CPU

    Posted May 05, 2016 04:44 PM

    /tmp. file ownership and entropy in /dev/random  appear to be the biggest external OS hits to performance to SSO solution.

     

     

    Edit:  2018/06/14   - 

     

    Example of dev-ops processes in startup for SSO to reset permissions under /tmp/GCL files.

     

     

    echo "### Update ownership of siteminder folders  ###"
    chown -R $USER:$GROUP  /tmp/GCL*  > /dev/null 2>&1