We are experiencing issues with our CA Identity Suite vApp cluster in our Acceptance environment. All components appear to be working as expected with the exception of the Web listener (httpd) on the vApp cluster.
We have deployed 3 IAM environments:
- Development
- Test
- Acceptance
In the Development and Test we have the following
1x CA SSO Access Gateway (12.8) Linux
1x CA Identity Suite vApp (14.2) [Demo / Sandbox]
1x CA SSO/CA Advanced Auth Server. (12.8/8.x) Windows
1x MS SQL Server
We have no issues in Development or Test
In Acceptance we have:
2x CA SSO Access Gateway (12.8) on Linux
2x CA Identity Suite vApp (14.2) in cluster. [Production]
2x CA SSO/CA Advanced Auth Server. (12.8/8.x) on Windows Server
2x MS SQL Server
In acceptance we are using Load Balancers as follows:
Web VIP: LB for CA SSO Access GW (SSL 443)
IAM VIP: LB for CA vAPP (SSL 443)
AA VIP: LB for Advanced Auth Tomcat (SSL 8443)
SM VIP: LB for CA SSO auth/az (TCP 4444x)
The Load Balancers are:
configured to use stickiness based on source IP
health-check with http/https requests to the target servers
Problem:
We test with only one human tester (1 web client)
The problem we have is that the connections between the CA Access Gateways and the CA IAM vApp(s) grow until one of the vApps becomes unresponsive on port 443. If we pause and let the connections idle out and close then we can continue testing again until we soon hit the same issue. We recreate this within a few minutes. At this time the number of connections is around 300 (even though we have just one person testing). The access gateways are running the default connection pool settings.
We have tested the Acceptance environment without the load balancers (i.e. the Access GW proxy direct to the vApp 443) and we get the same problem, after a short time.
We have also tested by shutting down one of each server, and also with/without using the load balancers, this also has the same problem.
We have also tried to proxy direct to the Identity Portal port (8081) on the vApp and avoid the http listener. But this does not seem to work.
Questions
- Has anyone else seen this problem?
- What is the expected behaviour in this scenario?
- Are we correct in load-balancing the vApps http listener (443), or should we be bypassing this and proxying to the individual components, if so how?