Hello friends, Getting staright to the point.
Policy Server: R1251FullVersion=12.51.1.972 (Cant be upgraded to 12.52 because of internal reasons)
ASA: PRODUCT_VERSION=12.0 , PRODUCT_NAME=eTrust SiteMinder Agent for IBM WebSphere, PRODUCT_UPDATE=0200 , PRODUCT_LABEL=210.
Both of them are running on Linux. Have been running fine for few years, until this happened today.
Policy Store and Key store are running in different AD LDS Stores.
While restarting application server(ASA) our Policy Server crashed, came back up by itself. ASA failed to start properly as it could not contact Policy Server. We restarted ASA again which crashed the policy server. When PS crashes and comes back up automatically, it comes back cleanly. There are no errors conencting to Policy store. It is able to communicate with other agents absolutely fine, however, during ASA start PS fails with 'Cant Connect to LDAP' error. It happened twice.
Finally as a last resort and without any good reason I tried restarting policy server and then restart ASA again, fortunately everything came up normally. However it leaves me worried very much. I wonder if there is a difference between policy server crash and policy server restart using stop-all/start-all
Could anyone suggest what could be happening here? did my PS try to open a new connection to the policy store and failed to create a new connection? is there something wrong with this particular thread? This very same thread is used for other transactions from webagent(not TAI) on the same box few seconds ago. Even if it, why would it fail twice?
Appreciate any insight. I dont want to create a ticket as it would lead me to nowhere.
smexec.log:
Thu Sep 1 10:10:16 2016: Enter: HandleSignals(BLOCK_ALL)
Thu Sep 1 10:10:16 2016: Enter: HandleSignals(UNBLOCK_ALL)
Thu Sep 1 10:10:16 2016: Service smpolicysrv died - restart #0, pid: 11287
Thu Sep 1 10:10:16 2016: Enter: HandleSignals(UNBLOCK_ALL)
Thu Sep 1 11:25:53 2016: Enter: HandleSignals(BLOCK_ALL)
Thu Sep 1 11:25:53 2016: Enter: HandleSignals(UNBLOCK_ALL)
Thu Sep 1 11:25:53 2016: Service smpolicysrv died - restart #0, pid: 11913
SMTRACE:
[09/01/2016][10:10:12.187][10:10:12][15310][4002368368][CServer.cpp:3152][CAgentAcceptHandler::HandleInput][][][][][][][][][][][][][][][][][][][][][Received connection request]
[09/01/2016][10:10:12.200][10:10:12][15310][4002368368][CServer.cpp:1801][CAgentMessageHandler::HandleInput][][][][][][][][][][][][][][][::ffff:IP_ADDRESS_OF_ASA(TAI)][19813][][][][][Enqueuing a High Priority Message, from IP ::ffff: IP_ADDRESS_OF_ASA(TAI) with Port No 19813. Current count is 1]
[09/01/2016][10:10:12.200][10:10:12][15310][4138736496][CServer.cpp:1382][ThreadPool::Run][][][][][][][][][][][][][][][::ffff:IP_ADDRESS_OF_ASA(TAI)][19813][][][][][Dequeuing a High Priority message, from IP ::ffff:IP_ADDRESS_OF_ASA(TAI) with Port No 19813. Current count is 0]
[09/01/2016][10:10:12.200][10:10:12][15310][4138736496][CServer.cpp:2058][CAgentMessageHandler::DoWork][][][][][][][][][][][][][][][::ffff:IP_ADDRESS_OF_ASA(TAI)][19813][][][][][New connection attempt from client host]
[09/01/2016][10:10:12.200][10:10:12][15310][4138736496][CServer.cpp:1842][GetSecretFunc][][][][][][][][][][][][][][][][][][][][][Getting current secret for the Agent portaltai8.5]
[09/01/2016][10:10:12.200][10:10:12][15310][4138736496][CServer.cpp:1917][GetSecretFunc][][][][][][][][][][][][][][][][][][][][][Marking the shared secret as used for the Agent portaltai8.5]
[09/01/2016][10:10:12.203][10:10:12][15310][4138736496][CServer.cpp:2073][CAgentMessageHandler::DoWork][][][][][][][][][][][][][][][::ffff:IP_ADDRESS_OF_ASA(TAI)][19813][][][][][Starting new client session #659]
[09/01/2016][10:10:12.206][10:10:12][15310][4002368368][CServer.cpp:1801][CAgentMessageHandler::HandleInput][][][][][][][][][][][][][][][::ffff:IP_ADDRESS_OF_ASA(TAI)][19813][][][][][Enqueuing a Normal Priority Message, from IP ::ffff:IP_ADDRESS_OF_ASA(TAI) with Port No 19813. Current count is 1]
[09/01/2016][10:10:12.206][10:10:12][15310][4012858224][CServer.cpp:1382][ThreadPool::Run][][][][][][][][][][][][][][][::ffff:IP_ADDRESS_OF_ASA(TAI)][19813][][][][][Dequeuing a Normal Priority message, from IP ::ffff:IP_ADDRESS_OF_ASA(TAI) with Port No 19813. Current count is 0]
[09/01/2016][10:10:12.206][10:10:12][15310][4012858224][CServer.cpp:5662][CServer::ProcessRequest][][][][][][][][][][][][][][][][][][][][][Enter function CServer::ProcessRequest]
[09/01/2016][10:10:12.206][10:10:12][15310][4012858224][CServer.cpp:5838][CServer::ProcessRequest][][][][][][][][][][][][][104][][][][][][][][Leave function CServer::ProcessRequest]
[09/01/2016][10:10:12.209][10:10:12][15310][4107266928][CServer.cpp:1382][ThreadPool::Run][][][][][][][][][][][][][][][::ffff:IP_ADDRESS_OF_ASA(TAI)][19815][][][][][Dequeuing a High Priority message, from IP ::ffff:IP_ADDRESS_OF_ASA(TAI) with Port No 19815. Current count is 0]
[09/01/2016][10:10:12.209][10:10:12][15310][4107266928][CServer.cpp:2058][CAgentMessageHandler::DoWork][][][][][][][][][][][][][][][::ffff:IP_ADDRESS_OF_ASA(TAI)][19815][][][][][New connection attempt from client host]
[09/01/2016][10:10:12.209][10:10:12][15310][4107266928][CServer.cpp:1842][GetSecretFunc][][][][][][][][][][][][][][][][][][][][][Getting current secret for the Agent portaltai8.5]
[09/01/2016][10:10:12.209][10:10:12][15310][4107266928][CServer.cpp:1917][GetSecretFunc][][][][][][][][][][][][][][][][][][][][][Marking the shared secret as used for the Agent portaltai8.5]
[09/01/2016][10:10:12.210][10:10:12][15310][4107266928][CServer.cpp:2073][CAgentMessageHandler::DoWork][][][][][][][][][][][][][][][::ffff:IP_ADDRESS_OF_ASA(TAI)][19815][][][][][Starting new client session #660]
[09/01/2016][10:10:12.210][10:10:12][15310][4002368368][CServer.cpp:1801][CAgentMessageHandler::HandleInput][][][][][][][][][][][][][][][::ffff:IP_ADDRESS_OF_ASA(TAI)][19815][][][][][Enqueuing a Normal Priority Message, from IP ::ffff:IP_ADDRESS_OF_ASA(TAI) with Port No 19815. Current count is 1]
[09/01/2016][10:10:12.210][10:10:12][15310][4012858224][CServer.cpp:1382][ThreadPool::Run][][][][][][][][][][][][][][][::ffff:IP_ADDRESS_OF_ASA(TAI)][19815][][][][][Dequeuing a Normal Priority message, from IP ::ffff:IP_ADDRESS_OF_ASA(TAI) with Port No 19815. Current count is 0]
[09/01/2016][10:10:12.210][10:10:12][15310][4012858224][CServer.cpp:5662][CServer::ProcessRequest][][][][][][][][][][][][][][][][][][][][][Enter function CServer::ProcessRequest]
[09/01/2016][10:10:12.210][10:10:12][15310][4012858224][CServer.cpp:5838][CServer::ProcessRequest][][][][][][][][][][][][][104][][][][][][][][Leave function CServer::ProcessRequest]
[09/01/2016][10:10:12.216][10:10:12][15310][
[CServer.cpp:1801][CAgentMessageHandler::HandleInput][][][][][][][][][][][][][][][::ffff:IP_ADDRESS_OF_ASA(TAI)][19813][][][][][Enqueuing a Normal Priority Message, from IP ::ffff:IP_ADDRESS_OF_ASA(TAI) with Port No 19813. Current count is 1]
[09/01/2016][10:10:12.216][10:10:12][15310][4012858224][CServer.cpp:1382][ThreadPool::Run][][][][][][][][][][][][][][][::ffff:IP_ADDRESS_OF_ASA(TAI)][19813][][][][][Dequeuing a Normal Priority message, from IP ::ffff:IP_ADDRESS_OF_ASA(TAI) with Port No 19813. Current count is 0]
[09/01/2016][10:10:12.216][10:10:12][15310][4012858224][CServer.cpp:5662][CServer::ProcessRequest][][][][][][][][][][][][][][][][][][][][][Enter function CServer::ProcessRequest]
[09/01/2016][10:10:12.216][10:10:12][15310][4012858224][SmAuthUser.cpp:1372][CSmAuthUser::CSmAuthUser][][][][][][][][][][][][][][][][][][][][][Enter function CSmAuthUser::CSmAuthUser]
[09/01/2016][10:10:12.216][10:10:12][15310][4012858224][SmAuthUser.cpp:1417][CSmAuthUser::CSmAuthUser][][][][][][][][][][][][][][][][][][][][][Leave function CSmAuthUser::CSmAuthUser]
[09/01/2016][10:10:12.216][10:10:12][15310][4012858224][Sm_Az_Message.cpp:154][CSm_Az_Message::ProcessMessage][][][][][][][][][][][][][][][][][][][][][Enter function CSm_Az_Message::ProcessMessage]
[09/01/2016][10:10:12.216][10:10:12][15310][4012858224][SmMessage.cpp:513][CSmMessage::ParseAgentMessage][s659/r2][][][][][][][][][][][][][][][][][][][host][Receive request attribute 143, data size is 4]
[09/01/2016][10:10:12.216][10:10:12][15310][4012858224][SmMessage.cpp:513][CSmMessage::ParseAgentMessage][s659/r2][][][][][][][][][][][][][][][][][][][pld1][Receive request attribute 144, data size is 4]
[09/01/2016][10:10:12.216][10:10:12][15310][4012858224][Sm_Az_Message.cpp:205][CSm_Az_Message::ProcessMessage][s659/r2][][][][][][][][][][][][][][][][][][][portaltai8.5][** Received agent request.]
[09/01/2016][10:10:12.216][10:10:12][15310][4012858224][Sm_Az_Message.cpp:387][CSm_Az_Message::AnalyzeAzMessage][][][][][][][][][][][][][][][][][][][][][Enter function CSm_Az_Message::AnalyzeAzMessage]
[09/01/2016][10:10:12.216][10:10:12][15310][4012858224][Sm_Az_Message.cpp:395][CSm_Az_Message::AnalyzeAzMessage][][][][][][][][][][][][][true][][][][][][][][Leave function CSm_Az_Message::AnalyzeAzMessage]
[09/01/2016][10:10:12.216][10:10:12][15310][4012858224][GetConfig.cpp:81][CSm_Az_Message::GetConfig][][][][][][][][][][][][][][][][][][][][][Enter function CSm_Az_Message::GetConfig]
[09/01/2016][10:10:12.217][10:10:12][15310][4012858224][GetConfig.cpp:218][CSm_Az_Message::GetConfig][][][][][][][][][][][][][][][][][][][][][Request Timeout Value found in Host config object.]
[09/01/2016][10:10:12.217][10:10:12][15310][4012858224][GetConfig.cpp:225][CSm_Az_Message::GetConfig][][portaltai8.5][][][][][][][][][][][][][::ffff:IP_ADDRESS_OF_ASA(TAI)][][][][][][Overwriting Request timeout for Agent (portaltai8.5_::ffff:IP_ADDRESS_OF_ASA(TAI)) to 60]
[09/01/2016][10:10:12.217][10:10:12][15310][4012858224][Sm_Az_Message.cpp:406][CSm_Az_Message::SendReply][][][][][][][][][][][][][][][][][][][][][Enter function CSm_Az_Message::SendReply]
[09/01/2016][10:10:12.217][10:10:12][15310][4012858224][Sm_Az_Message.cpp:825][CSm_Az_Message::FormatAttribute][s659/r2][][][][][][][][][][][][][][][][][][][MaxSocketsPerPort=600][Send response attribute 144, data size is 21]
[09/01/2016][10:10:12.217][10:10:12][15310][4012858224][Sm_Az_Message.cpp:825][CSm_Az_Message::FormatAttribute][s659/r2][][][][][][][][][][][][][][][][][][][PolicyServer=HOSTNAME_OF_THE_POLICYSERVER,44441,44442,44443][Send response attribute 144, data size is 59]
[09/01/2016][10:10:12.217][10:10:12][15310][4012858224][Sm_Az_Message.cpp:825][CSm_Az_Message::FormatAttribute][s659/r2][][][][][][][][][][][][][][][][][][][MinSocketsPerPort=2][Send response attribute 144, data size is 19]
[09/01/2016][10:10:12.217][10:10:12][15310][4012858224][Sm_Az_Message.cpp:825][CSm_Az_Message::FormatAttribute][s659/r2][][][][][][][][][][][][][][][][][][][RequestTimeout=60][Send response attribute 144, data size is 17]
[09/01/2016][10:10:12.217][10:10:12][15310][4012858224][Sm_Az_Message.cpp:825][CSm_Az_Message::FormatAttribute][s659/r2][][][][][][][][][][][][][][][][][][][NewSocketStep=2][Send response attribute 144, data size is 15]
[09/01/2016][10:10:12.217][10:10:12][15310][4012858224][Sm_Az_Message.cpp:825][CSm_Az_Message::FormatAttribute][s659/r2][][][][][][][][][][][][][][][][][][][RequestTimeout=60][Send response attribute 144, data size is 17]
[09/01/2016][10:10:12.217][10:10:12][15310][4012858224][Sm_Az_Message.cpp:825][CSm_Az_Message::FormatAttribute][s659/r2][][][][][][][][][][][][][][][][][][][NewSocketStep=2][Send response attribute 144, data size is 15]
[09/01/2016][10:10:12.217][10:10:12][15310][4012858224][Sm_Az_Message.cpp:825][CSm_Az_Message::FormatAttribute][s659/r2][][][][][][][][][][][][][][][][][][][EnableFailOver=YES][Send response attribute 144, data size is 18]
[09/01/2016][10:10:12.217][10:10:12][15310][4012858224][Sm_Az_Message.cpp:825][CSm_Az_Message::FormatAttribute][s659/r2][][][][][][][][][][][][][][][][][][][][Send response attribute 146, data size is 0]
[09/01/2016][10:10:12.217][10:10:12][15310][4012858224][Sm_Az_Message.cpp:825][CSm_Az_Message::FormatAttribute][s659/r2][][][][][][][][][][][][][][][][][][][][Send response attribute 147, data size is 0]
[09/01/2016][10:10:12.217][10:10:12][15310][4012858224][Sm_Az_Message.cpp:595][CSm_Az_Message::ProcessMessage][s659/r2][][][][][][][][][][][][][][][][][][][][** Status: GetConfig. ]
[09/01/2016][10:10:12.217][10:10:12][15310][4012858224][Sm_Az_Message.cpp:599][CSm_Az_Message::SendReply][][][][][][][][][][][][][][][][][][][][][Leave function CSm_Az_Message::SendReply]
[09/01/2016][10:10:12.217][10:10:12][15310][4012858224][GetConfig.cpp:273][CSm_Az_Message::GetConfig][][][][][][][][][][][][][ok][][][][][][][][Leave function CSm_Az_Message::GetConfig]
[09/01/2016][10:10:12.217][10:10:12][15310][4012858224][Sm_Az_Message.cpp:377][CSm_Az_Message::ProcessMessage][][][][][][][][][][][][][809][][][][][][][][Leave function CSm_Az_Message::ProcessMessage]
[09/01/2016][10:10:12.217][10:10:12][15310][4012858224][SmAuthUser.cpp:1422][CSmAuthUser::~CSmAuthUser][][][][][][][][][][][][][][][][][][][][][Enter function CSmAuthUser::~CSmAuthUser]
[09/01/2016][10:10:12.217][10:10:12][15310][4012858224][SmAuthUser.cpp:1474][CSmAuthUser::~CSmAuthUser][][][][][][][][][][][][][][][][][][][][][Leave function CSmAuthUser::~CSmAuthUser]
[09/01/2016][10:10:12.217][10:10:12][15310][4012858224][CServer.cpp:5838][CServer::ProcessRequest][][][][][][][][][][][][][809][][][][][][][][Leave function CServer::ProcessRequest]
[09/01/2016][10:10:12.219][10:10:12][15310][4002368368][CServer.cpp:3152][CAgentAcceptHandler::HandleInput][][][][][][][][][][][][][][][][][][][][][Received connection request]
[09/01/2016][10:10:12.220][10:10:12][15310][4002368368][CServer.cpp:1801][CAgentMessageHandler::HandleInput][][][][][][][][][][][][][][][::ffff:IP_ADDRESS_OF_ASA(TAI)][19817][][][][][Enqueuing a High Priority Message, from IP ::ffff:IP_ADDRESS_OF_ASA(TAI) with Port No 19817. Current count is 1]
[09/01/2016][10:10:12.220][10:10:12][15310][4128246640][CServer.cpp:1382][ThreadPool::Run][][][][][][][][][][][][][][][::ffff:IP_ADDRESS_OF_ASA(TAI)][19817][][][][][Dequeuing a High Priority message, from IP ::ffff:IP_ADDRESS_OF_ASA(TAI) with Port No 19817. Current count is 0]
[09/01/2016][10:10:12.220][10:10:12][15310][4128246640][CServer.cpp:2058][CAgentMessageHandler::DoWork][][][][][][][][][][][][][][][::ffff:IP_ADDRESS_OF_ASA(TAI)][19817][][][][][New connection attempt from client host]
[09/01/2016][10:10:12.220][10:10:12][15310][4128246640][CServer.cpp:1842][GetSecretFunc][][][][][][][][][][][][][][][][][][][][][Getting current secret for the Agent portaltai8.5]
[09/01/2016][10:10:12.220][10:10:12][15310][4128246640][CServer.cpp:1917][GetSecretFunc][][][][][][][][][][][][][][][][][][][][][Marking the shared secret as used for the Agent portaltai8.5]
[09/01/2016][10:10:12.221][10:10:12][15310][4128246640][CServer.cpp:2073][CAgentMessageHandler::DoWork][][][][][][][][][][][][][][][::ffff:IP_ADDRESS_OF_ASA(TAI)][19817][][][][][Starting new client session #661]
[09/01/2016][10:10:12.222][10:10:12][15310][4002368368][CServer.cpp:1801][CAgentMessageHandler::HandleInput][][][][][][][][][][][][][][][::ffff:IP_ADDRESS_OF_ASA(TAI)][19817][][][][][Enqueuing a Normal Priority Message, from IP ::ffff:IP_ADDRESS_OF_ASA(TAI) with Port No 19817. Current count is 1]
[09/01/2016][10:10:12.222][10:10:12][15310][4012858224][CServer.cpp:1382][ThreadPool::Run][][][][][][][][][][][][][][][::ffff:IP_ADDRESS_OF_ASA(TAI)][19817][][][][][Dequeuing a Normal Priority message, from IP ::ffff:IP_ADDRESS_OF_ASA(TAI) with Port No 19817. Current count is 0]
[09/01/2016][10:10:12.222][10:10:12][15310][4012858224][CServer.cpp:5662][CServer::ProcessRequest][][][][][][][][][][][][][][][][][][][][][Enter function CServer::ProcessRequest]
[09/01/2016][10:10:12.222][10:10:12][15310][4012858224][CServer.cpp:5838][CServer::ProcessRequest][][][][][][][][][][][][][104][][][][][][][][Leave function CServer::ProcessRequest]
[09/01/2016][10:10:12.223][10:10:12][15310][4002368368][CServer.cpp:3152][CAgentAcceptHandler::HandleInput][][][][][][][][][][][][][][][][][][][][][Received connection request]
[09/01/2016][10:10:12.224][10:10:12][15310][4002368368][CServer.cpp:1801][CAgentMessageHandler::HandleInput][][][][][][][][][][][][][][][::ffff:IP_ADDRESS_OF_ASA(TAI)][19819][][][][][Enqueuing a High Priority Message, from IP ::ffff:IP_ADDRESS_OF_ASA(TAI) with Port No 19819. Current count is 1]
[09/01/2016][10:10:12.224][10:10:12][15310][4096777072][CServer.cpp:1382][ThreadPool::Run][][][][][][][][][][][][][][][::ffff:IP_ADDRESS_OF_ASA(TAI)][19819][][][][][Dequeuing a High Priority message, from IP ::ffff:IP_ADDRESS_OF_ASA(TAI) with Port No 19819. Current count is 0]
[09/01/2016][10:10:12.224][10:10:12][15310][4096777072][CServer.cpp:2058][CAgentMessageHandler::DoWork][][][][][][][][][][][][][][][::ffff:IP_ADDRESS_OF_ASA(TAI)][19819][][][][][New connection attempt from client host]
[09/01/2016][10:10:12.224][10:10:12][15310][4096777072][CServer.cpp:1382][ThreadPool::Run][][][][][][][][][][][][][][][::ffff:IP_ADDRESS_OF_ASA(TAI)][19819][][][][][Dequeuing a High Priority message, from IP ::ffff:IP_ADDRESS_OF_ASA(TAI) with Port No 19819. Current count is 0]
[09/01/2016][10:10:12.224][10:10:12][15310][4096777072][CServer.cpp:2058][CAgentMessageHandler::DoWork][][][][][][][][][][][][][][][::ffff:IP_ADDRESS_OF_ASA(TAI)][19819][][][][][New connection attempt from client host]
[09/01/2016][10:10:12.224][10:10:12][15310][4096777072][CServer.cpp:1842][GetSecretFunc][][][][][][][][][][][][][][][][][][][][][Getting current secret for the Agent portaltai8.5]
[09/01/2016][10:10:12.224][10:10:12][15310][4096777072][CServer.cpp:1917][GetSecretFunc][][][][][][][][][][][][][][][][][][][][][Marking the shared secret as used for the Agent portaltai8.5]
[09/01/2016][10:10:12.225][10:10:12][15310][4096777072][CServer.cpp:2073][CAgentMessageHandler::DoWork][][][][][][][][][][][][][][][::ffff:IP_ADDRESS_OF_ASA(TAI)][19819][][][][][Starting new client session #662]
[09/01/2016][10:10:12.225][10:10:12][15310][4002368368][CServer.cpp:1801][CAgentMessageHandler::HandleInput][][][][][][][][][][][][][][][::ffff:IP_ADDRESS_OF_ASA(TAI)][19819][][][][][Enqueuing a Normal Priority Message, from IP ::ffff:IP_ADDRESS_OF_ASA(TAI) with Port No 19819. Current count is 1]
[09/01/2016][10:10:12.225][10:10:12][15310][4012858224][CServer.cpp:1382][ThreadPool::Run][][][][][][][][][][][][][][][::ffff:IP_ADDRESS_OF_ASA(TAI)][19819][][][][][Dequeuing a Normal Priority message, from IP ::ffff:IP_ADDRESS_OF_ASA(TAI) with Port No 19819. Current count is 0]
[09/01/2016][10:10:12.226][10:10:12][15310][4012858224][CServer.cpp:5662][CServer::ProcessRequest][][][][][][][][][][][][][][][][][][][][][Enter function CServer::ProcessRequest]
[09/01/2016][10:10:12.226][10:10:12][15310][4012858224][CServer.cpp:5838][CServer::ProcessRequest][][][][][][][][][][][][][104][][][][][][][][Leave function CServer::ProcessRequest]
[09/01/2016][10:10:12.228][10:10:12][15310][4002368368][CServer.cpp:1801][CAgentMessageHandler::HandleInput][][][][][][][][][][][][][][][::ffff:IP_ADDRESS_OF_ASA(TAI)][19817][][][][][Enqueuing a Normal Priority Message, from IP ::ffff:IP_ADDRESS_OF_ASA(TAI) with Port No 19817. Current count is 1]
[09/01/2016][10:10:12.228][10:10:12][15310][4012858224][CServer.cpp:1382][ThreadPool::Run][][][][][][][][][][][][][][][::ffff:IP_ADDRESS_OF_ASA(TAI)][19817][][][][][Dequeuing a Normal Priority message, from IP ::ffff:IP_ADDRESS_OF_ASA(TAI) with Port No 19817. Current count is 0]
[09/01/2016][10:10:12.228][10:10:12][15310][4012858224][CServer.cpp:5662][CServer::ProcessRequest][][][][][][][][][][][][][][][][][][][][][Enter function CServer::ProcessRequest]
[09/01/2016][10:10:12.228][10:10:12][15310][4012858224][SmAuthUser.cpp:1372][CSmAuthUser::CSmAuthUser][][][][][][][][][][][][][][][][][][][][][Enter function CSmAuthUser::CSmAuthUser]
[09/01/2016][10:10:12.228][10:10:12][15310][4012858224][SmAuthUser.cpp:1417][CSmAuthUser::CSmAuthUser][][][][][][][][][][][][][][][][][][][][][Leave function CSmAuthUser::CSmAuthUser]
[09/01/2016][10:10:12.228][10:10:12][15310][4012858224][Sm_Auth_Message.cpp:151][CSm_Auth_Message::ProcessMessage][][][][][][][][][][][][][][][][][][][][][Enter function CSm_Auth_Message::ProcessMessage]
[09/01/2016][10:10:12.228][10:10:12][15310][4012858224][AgentAuth.cpp:36][CSm_Auth_Message::ProcessAgentMessage][][][][][][][][][][][][][][][][][][][][][Enter function CSm_Auth_Message::ProcessAgentMessage]
[09/01/2016][10:10:12.228][10:10:12][15310][4012858224][SmMessage.cpp:513][CSmMessage::ParseAgentMessage][s661/r2][portaltai8.5][][][][][][][][][][][][][][][][][][0][Receive request attribute 199, data size is 1]
[09/01/2016][10:10:12.228][10:10:12][15310][4012858224][SmMessage.cpp:513][CSmMessage::ParseAgentMessage][s661/r2][portaltai8.5][][][][][][][][][][][][][][][][][][1472742612][Receive request attribute 159, data size is 10]
[09/01/2016][10:10:12.228][10:10:12][15310][4012858224][SmMessage.cpp:513][CSmMessage::ParseAgentMessage][s661/r2][portaltai8.5][][][][][][][][][][][][][][][][][][0][Receive request attribute 165, data size is 1]
[09/01/2016][10:10:12.228][10:10:12][15310][4012858224][SmMessage.cpp:513][CSmMessage::ParseAgentMessage][s661/r2][portaltai8.5][][][][][][][][][][][][][][][][][][128][Receive request attribute 166, data size is 3]
[09/01/2016][10:10:12.228][10:10:12][15310][4012858224][SmMessage.cpp:513][CSmMessage::ParseAgentMessage][s661/r2][portaltai8.5][][][][][][][][][][][][][][][][][][3][Receive request attribute 148, data size is 1]
[09/01/2016][10:10:12.229][10:10:12][15310][4012858224][SmMessage.cpp:513][CSmMessage::ParseAgentMessage][s661/r2][portaltai8.5][][][][][][][][][][][][][][][][][][PRODUCT_VERSION=12.0 , PRODUCT_NAME=eTrust SiteMinder Agent for IBM WebSphere, PRODUCT_UPDATE=0200 , PRODUCT_LABEL=210.][Receive request attribute 149, data size is 119]
[09/01/2016][10:10:12.229][10:10:12][15310][4012858224][AgentAuth.cpp:312][CSm_Auth_Message::AnalyzeAgentAuthMessage][][][][][][][][][][][][][][][][][][][][][Enter function CSm_Auth_Message::AnalyzeAgentAuthMessage]
[09/01/2016][10:10:12.229][10:10:12][15310][4012858224][AgentAuth.cpp:357][CSm_Auth_Message::AnalyzeAgentAuthMessage][][][][][][][][][][][][][true][][][][][][][][Leave function CSm_Auth_Message::AnalyzeAgentAuthMessage]
[09/01/2016][10:10:12.229][10:10:12][15310][4012858224][DoManagement.cpp:462][CSm_Auth_Message::DoManagement][][][][][][][][][][][][][][][][][][][][PRODUCT_VERSION=12.0 , PRODUCT_NAME=eTrust SiteMinder Agent for IBM WebSphere, PRODUCT_UPDATE=0200 , PRODUCT_LABEL=210.,UTC=1472742612,TZ=0,Crypto=128][AgentTli SetAgentInfo command.]
[09/01/2016][10:10:12.229][10:10:12][15310][4012858224][Sm_Auth_Message.cpp:3409][CSm_Auth_Message::SendReply][][][][][][][][][][][][][][][][][][][][][Enter function CSm_Auth_Message::SendReply]
[09/01/2016][10:10:12.229][10:10:12][15310][4012858224][DoManagement.cpp:462][CSm_Auth_Message::DoManagement][][][][][][][][][][][][][][][][][][][][PRODUCT_VERSION=12.0 , PRODUCT_NAME=eTrust SiteMinder Agent for IBM WebSphere, PRODUCT_UPDATE=0200 , PRODUCT_LABEL=210.,UTC=1472742612,TZ=0,Crypto=128][AgentTli SetAgentInfo command.]
[09/01/2016][10:10:12.229][10:10:12][15310][4012858224][Sm_Auth_Message.cpp:3409][CSm_Auth_Message::SendReply][][][][][][][][][][][][][][][][][][][][][Enter function CSm_Auth_Message::SendReply]
[09/01/2016][10:10:12.229][10:10:12][15310][4012858224][Sm_Auth_Message.cpp:3734][CSm_Auth_Message::SendReply][][][][][][][][][][][][][][][][][][][][][Leave function CSm_Auth_Message::SendReply]
[09/01/2016][10:10:12.229][10:10:12][15310][4012858224][AgentAuth.cpp:303][CSm_Auth_Message::ProcessAgentMessage][][][][][][][][][][][][][20][][][][][][][][Leave function CSm_Auth_Message::ProcessAgentMessage]
[09/01/2016][10:10:12.229][10:10:12][15310][4012858224][Sm_Auth_Message.cpp:164][CSm_Auth_Message::ProcessMessage][][][][][][][][][][][][][20][][][][][][][][Leave function CSm_Auth_Message::ProcessMessage]
[09/01/2016][10:10:12.229][10:10:12][15310][4012858224][SmAuthUser.cpp:1422][CSmAuthUser::~CSmAuthUser][][][][][][][][][][][][][][][][][][][][][Enter function CSmAuthUser::~CSmAuthUser]
[09/01/2016][10:10:12.229][10:10:12][15310][4012858224][SmAuthUser.cpp:1474][CSmAuthUser::~CSmAuthUser][][][][][][][][][][][][][][][][][][][][][Leave function CSmAuthUser::~CSmAuthUser]
[09/01/2016][10:10:12.229][10:10:12][15310][4012858224][CServer.cpp:5838][CServer::ProcessRequest][][][][][][][][][][][][][20][][][][][][][][Leave function CServer::ProcessRequest]
[09/01/2016][10:10:12.230][10:10:12][15310][4002368368][CServer.cpp:1801][CAgentMessageHandler::HandleInput][][][][][][][][][][][][][][][::ffff:IP_ADDRESS_OF_ASA(TAI)][19817][][][][][Enqueuing a Normal Priority Message, from IP ::ffff:IP_ADDRESS_OF_ASA(TAI) with Port No 19817. Current count is 1]
[09/01/2016][10:10:12.230][10:10:12][15310][4075797360][CServer.cpp:1382][ThreadPool::Run][][][][][][][][][][][][][][][::ffff:IP_ADDRESS_OF_ASA(TAI)][19817][][][][][Dequeuing a Normal Priority message, from IP ::ffff:IP_ADDRESS_OF_ASA(TAI) with Port No 19817. Current count is 0]
[09/01/2016][10:10:12.230][10:10:12][15310][4075797360][CServer.cpp:5662][CServer::ProcessRequest][][][][][][][][][][][][][][][][][][][][][Enter function CServer::ProcessRequest]
[09/01/2016][10:10:12.230][10:10:12][15310][4075797360][SmAuthUser.cpp:1372][CSmAuthUser::CSmAuthUser][][][][][][][][][][][][][][][][][][][][][Enter function CSmAuthUser::CSmAuthUser]
[09/01/2016][10:10:12.230][10:10:12][15310][4075797360][SmAuthUser.cpp:1417][CSmAuthUser::CSmAuthUser][][][][][][][][][][][][][][][][][][][][][Leave function CSmAuthUser::CSmAuthUser]
[09/01/2016][10:10:12.230][10:10:12][15310][4075797360][Sm_Auth_Message.cpp:151][CSm_Auth_Message::ProcessMessage][][][][][][][][][][][][][][][][][][][][][Enter function CSm_Auth_Message::ProcessMessage]
[09/01/2016][10:10:12.230][10:10:12][15310][4075797360][AgentAuth.cpp:36][CSm_Auth_Message::ProcessAgentMessage][][][][][][][][][][][][][][][][][][][][][Enter function CSm_Auth_Message::ProcessAgentMessage]
[09/01/2016][10:10:12.230][10:10:12][15310][4075797360][SmMessage.cpp:513][CSmMessage::ParseAgentMessage][s661/r3][portaltai8.5][][][][][][][][][][][][][][][][][][0][Receive request attribute 199, data size is 1]
[09/01/2016][10:10:12.230][10:10:12][15310][4075797360][SmMessage.cpp:513][CSmMessage::ParseAgentMessage][s661/r3][portaltai8.5][][][][][][][][][][][][][][][][][][1472742612][Receive request attribute 159, data size is 10]
[09/01/2016][10:10:12.230][10:10:12][15310][4 075797360][SmMessage.cpp:513][CSmMessage::ParseAgentMessage][s661/r3][portaltai8.5][][][][][][][][][][][][][][][][][][0][Receive request attribute 165, data size is 1]
[09/01/2016][10:10:12.230][10:10:12][15310][4075797360][SmMessage.cpp:513][CSmMessage::ParseAgentMessage][s661/r3][portaltai8.5][][][][][][][][][][][][][][][][][][128][Receive request attribute 166, data size is 3]
[09/01/2016][10:10:12.230][10:10:12][15310][4075797360][SmMessage.cpp:513][CSmMessage::ParseAgentMessage][s661/r3][portaltai8.5][][][][][][][][][][][][][][][][][][1][Receive request attribute 148, data size is 1]
[09/01/2016][10:10:12.230][10:10:12][15310][4075797360][SmMessage.cpp:513][CSmMessage::ParseAgentMessage][s661/r3][portaltai8.5][][][][][][][][][][][][][][][][][][][Receive request attribute 149, data size is 0]
[09/01/2016][10:10:12.230][10:10:12][15310][4075797360][AgentAuth.cpp:312][CSm_Auth_Message::AnalyzeAgentAuthMessage][][][][][][][][][][][][][][][][][][][][][Enter function CSm_Auth_Message::AnalyzeAgentAuthMessage]
[09/01/2016][10:10:12.230][10:10:12][15310][4075797360][AgentAuth.cpp:357][CSm_Auth_Message::AnalyzeAgentAuthMessage][][][][][][][][][][][][][true][][][][][][][][Leave function CSm_Auth_Message::AnalyzeAgentAuthMessage]
[09/01/2016][10:10:12.230][10:10:12][15310][4075797360][smldaputils.cpp:1209][SmSearchLDAP][][][][][][][][][][][][][][][][][][][Handle='0x9b1a640', Root='ou=PolicySvr4,ou=SiteMinder,ou=Netegrity,o=Company.com', Scope=1, Filter='(&(smAgentKeyOID4=1b-000bd9c6-be45-17bc-be3c-80740a98a05e)(objectcategory=smAgentKey4))', attrsonly=0][][Start of call ldap_search_st:Search LDAP.]
[09/01/2016][10:10:12.230][10:10:12][15310][4075797360][smldaputils.cpp:1215][SmSearchLDAP][][][][][][][][][][][][][81][Can't contact LDAP server][][][][][Handle='0x9b1a640', Root='ou=PolicySvr4,ou=SiteMinder,ou=Netegrity,o=Company.com', Scope=1, Filter='(&(smAgentKeyOID4=1b-000bd9c6-be45-17bc-be3c-80740a98a05e)(objectcategory=smAgentKey4))', attrsonly=0][][Return from call ldap_search_st]