Symantec Access Management

I am using CA Directory server version:cadirr120 on my RedHat 6 linux box as a SESSION STORE for R1252Cr4 policy server. Occasionally, our requests are failing , upon checking the logs I see:

[11/07/2016][00:15:54.815][00:15:54][22228][3992075120][LdapStore.cpp:2077][SmAddPLDAP][][][][][][][][][][][][][81][Can't contact LDAP server][][][][][Handle='0xc19ef70', DN='cn=fGu9D5PoNcio\=,ou=12:ExpType,ou=session,o=compnay,c=US'][][Return from call ldap_add_s][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][][]

This is a known issues from SM with idle timeouts. Where and how do I modify idle timeout settings on Directory Server?

Appreciate any insights. Thanks in advance.

Have you tried this Anil ?

https://support.ca.com/cadocs/0/CA%20Directory%2012%200%20SP14-ENU/Bookshelf_Files/HTML/idocs/index.htm?toc.htm?administration.htm

set user-idle-time Command

The set user-idle-time command specifies the maximum time a user is idle before being disconnected. 

When a user is idle for too long, that user is disconnected. This reduces the number of users connected and lets new users connect to the DSA.

This command has the following format: 

set user-idle-time = time;

time

Specifies the maximum idle time in seconds.

Note that "set user-idle-time = 0;" will set this to unlimited. This command has synonyms of 'bind-idle-time' and also 'idle-time' so it's worth checking for these in your configuration files (especially the limits file source by config/server/{dsa name}.dxi) to ensure the command isn't duplicated.

To verify it is in fact the idle timer causing the disconnect, you can check the warn-log under $DXHOME/logs for the DSA in question for something like: "Idle association <n> ( -) timed out after <n> seconds".

Thank You, Justin, WIll try it tomorrow. You guys are amazing. Appreciate it.

Hi Justin, I did have few timeouts in my warn_log :

[0] 20161108.101524.733 WARN : Idle association 23 timed out after 4763 seconds

I have added to my limits file the statement you provided , and it looks like:

My .dxc file in limits folder looks now like:

# Computer Associates DXserver/config/limits
# $RCSfile: default.dxc,v $ $Revision: 4.11 $
# This is a read-only default configuration file. If you need to make changes,
# copy this file and reference the new file from servers/<dsa>.dxi
#

# size limits
set max-users = 255;
set max-op-size = 10000;
set multi-write-queue = 20000;

# time limits
set max-bind-time = none;
set bind-idle-time = 3600;
set max-op-time = 600;
set user-idle-time = 0;

So, Anil, I hope you are not getting those Error 81 any more for this instance in policy server log ?

Please correct me if am wrong, but I guess bind-idle-time and user-idle-time are serving the same purpose. So, their values should also be same in that case. But in the above configuration, both are having conflicting values. will it cause some issue?

A policy server is creating a bind connection with the directory for user authentication, then how will it consider it to be idle: is this connection user based(i.e, w.r.t one bind request created for one user) or server based(one bind request created for one directory server for all authentication requests going to that server).

The two commands are internally the same. The sequence above is setting it to 3600 then to 0 (unlimited) so there is no issue.

A connection is considered idle if directory has not received any requests for the configured time limit. SM creates a connection using an administrative account. If no users authenticate for an extended period, then this connection would be considered idle as no requests would be generated.

Thanks for quick response, as usual. I looked at that but was worried about changes it makes on the back end. I am looking for file level changes so I can revert quickly if needed. Justin has provided that and I will try it tomorrow.

Hello Ujwol,

I am new to CA Directory concepts, thus can you please help me in understanding if setting this user-idle-time = time at directory level will conflict with the idle session timeout settings done at siteminder level.

At siteminder level we define session timeout for a user session over a particular resource and at directory level we are defining the idle session timeout for a user binding with dsa.

Also, is this setting validated for user stores as well or only for session stores?

Thanks in advance!

Hi Aditi,

This setting will not impact idle/max session timeout on SiteMinder.

Ok, let's go back a little bit into the internals.

User Store

The Policy Server creates three connections to a given LDAP directory: Ping, User and Search. Each connection is independent of the others. 

• The 'ping' thread runs periodically, and issues a command via its LDAP directory connection to test for connectivity.
• The 'bind' (or 'user') connection performs the authentication requests.
• The 'select' (or 'search') connection performs LDAP searches and other long running commands.

When there is no activity on either user or search connections, the connections may timeout. Such
timeouts, also called idle timeouts, may happen for several reasons such as:

- The LDAP server has the idle timeout option configured. In this case the server will close all
connections that haven’t been active for the duration of the timeout. ( this is is the issue we are trying to address here by setting the idle time out setting on CA Directory side)
- There is a state-full firewall between the policy server and the LDAP server.
Due to periodic activity on a ping connection, a ping connection will not timeout (assuming that the
idle timeout is less than 30 seconds). The next request will go against the user and search connections
that have been timed-out. The request will return with a network error causing re-initialization of the
connections

Session Store

In case of session store, a connection pool is maintained, When ever needed, worker thread will grab a connection from this pool and make necessary change in the session store.

For the same reason as user store, these connection could also idle out and the same idle time out configuration on the LDAP side will help avoiding these condition as well.

Please let me know if you have any further questions.

Cheers,

Ujwol

Can you please help with one more query, there are 2 parameters in dsa configuration, max-users and the user-idle-time. In our environment, we have an estimation of approx 10,000 concurrent user logins at a given point of time. As per CA documentation, max-users parameter set the max number of users which can bind to a user store concurrently and its max value can be set to 4096. Can you help in understanding how these two values can impact each other?

Yes directory has a hard limit of 4096 active connections. However, as touched on above the SM application only maintains 3 connections to CA Directory. That is, SM can support 10K active users, but these users are authenticated by the SM application using a single directory connection.

If there are a high number of user connecting directly to directory then the user-idle-time is useful in cleaning up stale connections.

Slight correction on the SM Side. 

SM goes by the LDAP banks.

So if you have user directory A with only one LDAP bank as below:

ldapserver:port

Then, yes it creates 3 connections. However, the number of LDAP connections can be increased by creating additional LDAP banks to the same server.

e.g.

ldapserver:port

ldapserver:port

ldapserver:port

So in the above case SM creates 3*3 = 9 connections to the same backend LDAP server.

Even better approach is to have it with different host names (internally still pointing to same IP using host file, there is a added advantage with this approach which I will explain may be next time  )

e.g.

Host1:port

Host2:port

Host3:port

Cheers,

Ujwol

Hello Ujwol,

Sorry to ask another basic query, but where to configure these additional LDAP banks?

Is it to be done while creating a user directory in Admin UI?

If yes, what is a better approach: failover or load-balancing for multiple connections to the same LDAP server?

Thanks in advance!

Hi Aditi,

Can we work on that in a new thread please as we have been deviating from the original discussion.

Regards,

Ujwol

Sure, we can start a new thread for the same.

HI Ujwol, Sorry to ask this basic question. How do I run this idle-timeout command? Is there seperate CLI for Directory server? I cant seem to run this from DXHome/dxserver/bin.

For nowm, I modified limits/sessionstore.dxc to add 

set user-idle-time = 0;


restarted my servers. Waiting for it to happen again. Even before, it would only happen very very rarely.

You have set this correctly

set bind-idle-time = 3600;
set max-op-time = 600;
set user-idle-time = 0;

Alternatively you could have just done the following as bind-idle-time and user-idle-time are synonyms for the same command

set bind-idle-time = 0;
set max-op-time = 600;

Just to double check you have updated the correct file. Can you double check the source line in $DXHOME/config/servers/{dsa name}.dxi.

For example,

# operational settings
source "../settings/default.dxc";

# service limits
source "../limits/default.dxc";

# access controls
clear access;
source "../access/default.dxc";

The source command for the server is the limits file you DSA is using.

Note: We don't recommend updating default.dxc directly as it is replaced on upgrade. To work around this, copy default.dxc to a different name, update the .dxi file to source the new name instead of default.dxc. Run dxsyntax to check for syntax errors and re-init the DSA.

DXserver does contain a CLI. In the knowledge file there is an item called console-port. If you telnet localhost {console-port} you can run any DXserver command. These will only be temporary as any configuration changes made aren't stored on disk.

Thank You , Justin. I did confirm that I have modified correct files.

However, I still see issues in my affwebserv.log while I cant find related traces in Policy server mapping to LDAP error 81.

Restarted my Policy servers for a clean connection with Directory Server. Will keep this thread informed.

Thanks everyone.