Layer7 API Management

We are running performance tests with approximately 450,000 requests spread evenly over an hour.  We have one appliance that uses the MySQL database.  We monitor with App Dynamics.  Requests and responses are very small.  The request is just a URL with no body; the response is less than 1K.  In our test, we see many slow transactions - about 4,000 have response times > 1500ms.  The back end has an average response time of < 10ms.  App Dynamics reports that the gateway is having difficulty getting a pooled connection from a datasource (SSG-MySQL DB-LOCALHOST-5.5.48).

Are we attempting an unreasonable load for a single server?  Is there a way to tune to improve DB connectivity.  I'm not sure if these are the appropriate cluster properties, but jdbcConnection.pooling.maxPoolSize.defaultValue = 50 and jdbcConnection.pooling.minPoolSize.defaultValue = 30. 

Thanks.

Good afternoon,

Below are some settings that we have found to be beneficial (Note: You will need to restart the Gateway once the .properties and .cnf file changes have been made). Depending on the version you are running some may already be in place in the default code. Lastly ensure that you have at the very least 4 GB of RAM assigned to each of the nodes but for higher performance 16 GB is recommended.

1. node.properties
/opt/SecureSpan/Gateway/node/default/etc/conf/node.properties
c3p0DataSource.maxPoolSize=1150

2. system.properties
/opt/SecureSpan/Gateway/node/default/etc/conf/system.properties
com.l7tech.message.httpParamsMaxFormPost=5242880
com.l7tech.http.maxParameterLength=1000000
org.apache.tomcat.util.http.ServerCookie.ALLOW_EQUALS_IN_VALUE=true
com.l7tech.common.http.prov.apache.CommonsHttpClient.maxConnectionsPerHost=1500
com.l7tech.common.http.prov.apache.CommonsHttpClient.maxTotalConnections=3000
com.l7tech.server.policy.assertion.ServerHttpRoutingAssertion.statePool.enable=true

3. my.cnf
/etc/my.cnf

by changing this line:

max_connections=2625

to this:

max_connections=10000

4. ClusterWide Properties
io.httpConnectionIdleTimeout = 5s
io.httpCoreConcurrency = 750
io.httpMaxConcurrency = 1000

5. JDBC connection settings (if used)
Set the min and max pool size both to be a value of 150 on the jdbc connection settings.

We added the following properties to the jdbc connection
maxIdleTime 0 //c3p0 setting
maxConnectionAge 0 //c3p0 setting
idleConnectionTestPeriod 600 //c3p0 setting
EnableCancelTimeout true

Update Important Note: If you disable timeouts and connection max ages, ensure you have 8MB of memory per pooled connection to account for cached query data on top of the amount required for "normal" traffic processing.

Sincerely,

Stephen Hughes

Director, CA Support

Hi Stephen,

awesome article! thank you!

One thing though... I could not find the io.httpConnectionIdealTimeout variable in the list of CWPs, is this a 'special' one?

there are no references in the documentation or in the list of CWP in any of the gateway versions,

thanks

Maurizio

Maurizio,

I've corrected my original post as that CWP was typed wrong should be "io.httpConnectionIdleTimeout".

Sincerely,

Stephen Hughes

Director, CA Support