Are you truly ready to roll out the Layer 7 Gateway into Production? Many customers feel that they have taken all the right steps to test and validate the configuration through the various environments but find once in production that some keys questions may not have been asked. These questions may range from do I have enough resources allocated to the Gateway, , do I understand the capacity expected of the Gateway and the systems being integrated with, how am I monitoring the health of the Gateway, to how do I get notified of key events on the Gateway such as expiring certificates, replication sync failure, or audit table filling up. This document and supporting documents are designed to help make sure that you ask all the right questions and get all the right answers prior to certifying that the Gateway is production ready.
All the right questions to ask
____________________________________________________________________________________________________________________________________________________________________________
Question 1: Do I have enough resources allocated to the Gateway?
We field this questions quite often as to what should the levels be set to for CPU, RAM, and Hard drive space on the gateway and typically the answer back is it depends. Whenever this question is asked we point to what our current hardware appliance capabilities are and work back from there. In the current released appliance when this document was being written is equipped with dual CPU 12 core with 16 GB of RAM and 250 GB hard drive.Each of these resources will affect different parts of the overall consumption and processing of requests so our recommendation will attempt to address the 85-95 percentile of customer's use cases.
Environment CPU/Cores RAM Hard Disk (Min)
Development 1/2 2-4 GB 50 GB
UAT 2/4-6 4-8 GB 50 GB
Production 2/4-6 8-16 GB 50 GB
Question 2: Do I understand the capacity expected of the Gateway and the systems being integrated with?
During the initial purchase and implementation of the product, the amount of traffic in regards to concurrency, size, and quantity level are normally not know and only after the project moves along to Production or is in Production that the volume is truly understood. Configuration changes can be made to the Gateway to accommodate certain scenarios and again we have provided configuration changes to are general best practices.
Concurrency: The Gateway has 2 configuration settings that manage inbound HTTP requests for both the base number of connections/threads available at all times which is defaulted to 185 and the max number of connections/threads that the Gateway will allow which is default to 215. Once the max connection level is reached connections will queue up until they can be processed or time out. Slow latency of the backend along with a large number of client connections will require a larger number of available connections.
All our load testing done by our QA department uses the default values outlined above but our lab environment does not always reflect the nature of how individual customer environments react so we have provided initial settings which can be increased as required.
These settings are configurable through the Policy Manager -> Tasks -> Manage Cluster Wide Properties
io.httpCoreConcurrency - 500
io.httpMaxConcurrency - 750
As the concurrency values are increase we also need to take into consideration the number of concurrent databases connections for auditing and other related tasks. The default value is 260 connections so increasing value should be httpCore + 100 so for 500 we need to set the value to 600. Further to this depending on the number of nodes in the cluster you will need to look at whether the max pool size X Number of nodes will be over 2625 and if so use the procedure below to control the value in the my.cnf file.
Modify /opt/SecureSpan/Gateway/node/default/etc/conf/node.properties
add the line
c3p0DataSource.maxPoolSize=600
Modify the my.cnf file (/etc/my.cnf) on both the primary and secondary database nodes
by updating the following line from max_connections=2625 to max_connections=10000
Size:
2) Modify /opt/SecureSpan/Gateway/node/default/etc/conf/system.properties
add the lines
com.l7tech.message.httpParamsMaxFormPost=2147483647
com.l7tech.http.maxParameterLength=1000000
org.apache.tomcat.util.http.ServerCookie.ALLOW_EQUALS_IN_VALUE=true
sun.net.maxDatagramSockets=1000
Additional Cluster Wide Properties to be considered if large Directory groups are used
principalSessionCache.maxPrincipalGroups to 1000
principalSessionCache.cache to 1000
ldap.group.searchMaxResults to 1000
ldap.searchMaxResults to 1000
Sincerely,
Stephen Hughes
Director, CA Support