AnsweredAssumed Answered

New KB: Not seeing all APM CE transactions due to late SSL packets

Question asked by Hallett_German Employee on Nov 20, 2014
Latest reply on Nov 22, 2014 by Hallett_German

This is a new KB about a situation that some APM CE users may be having.



Not seeing all transactions due to late SSL packets

Document ID:  TEC1895461
Last Modified Date:  11/19/2014
ShowHide Technical Document Details  
  • Products  
    • CA Application Performance Management
  • Releases  
    • CA Application Performance Management:Release:9.5
  • Components  
In 21961067-1, customer is not seeing all of the transactions on their 9.5.3 MTP TIM. They are monitoring HTTP responses for Websphere Portal transactions.  TIM is reporting the malformed SSL record errors without actually having any loss of packets (i.e. recovered via re-transmissions).
The TIM logs contain the following:
Thu Nov  6 12:03:22 2014  5591   Trace: w7: ssl_analyze: unknown SSL content type 75 on []:60632->[]:443, conn 208703, packet 5783174
Thu Nov  6 12:03:22 2014  5591 ! Warning: w7: sslinterface: network_process_packet: error 3 (unspecified internal error), conn 208703, packet 5783174, []:60632->[]:443; ignoring further data
The current TIM design has the capability to re-assemble the packets when they arrive in out-of-order using standard seq/ack mechanism.
In this case,  the problem is the re-assembled TCP packets do not carry the full SSL/HTTPS record or carry multiple SSL/HTTPS records. As a result, TIM feeds the malformed SSL record to TIM's SSL dissector (i.e. ssldump) and eventually decryption fails. Thereafter, this leads to TIM ignoring the further data or no data on that connection. In other words, TIM does not know how to re-assemble packets not carrying the entire SSL/HTTPS record using seq/ack mechanism.
There are some possible solutions:
1) A Hotfix is available as needed that allows TIM to process/interpret/read the multiple or broken SSL records (that are re-assembled from TCP stream/segments) at the SSL layer/stack.
2) Consider using jumbo frames to avoid TCP segments getting out-of-order at the TCP layer. (See below links for reference.)
3) Use the following options to terminate/decrypt the SSL/HTTPS traffic and pass the HTTP traffic to TIM- monitored interface. So that re-assembling the TCP segments doesn't cause any issue:
·        Netronome SSL Inspector (Transparent SSL Proxy Appliance).
·        Reverse proxy setup (one connection for HTTPS and another for HTTP).