Hi All,
I have a service virtualization with a SOA service response very big.When I run the service in SoaUI, It shows "java.net.SocketException: Connection reset". The original service returns 14517625 response bytes.
How can I update the maximum response bytes in CA Lisa?.
Regards,
Hi Rosa, I have built virtual services that return response payloads that are several MB and have not seen this problem on the DevTest server (virtual service) side. I am not implying that there is no issue -- just that I have not seen one.
Are you thinking that DevTest is closing the HTTP connection before your service sends the entire response payload to the client? HTTP servers are designed to reset sockets for performance and concurrency reasons. So, it is possible that the server is experiencing an HTTP connection timeout and resetting the connection. But, if the Think Time on your 14+MB response is reasonably low, I am not sure that server would be resetting the socket. It is also possible that network latency is causing an issue as the client and server will chat with each other to complete the transmission of the request and response.
I would consider the following to try and narrow down the problem:
1) Take the network out of the equation. Run your service in ITR mode on your local workstation. Send a request from SoapUI on your localhost to your service on localhost. See if SoapUI throws the Socket exception when you run this pattern. If the response is successful, you may have narrowed the issue down to the network since everything is running locally.
2) Create a Webservice XML test step in DevTest on your workstation. Send your request from DevTest to the Virtual Service on the server and see if you get the entire 14+MB response payload. If you get the full response, you at least know that DevTest is sending the full response payload. So you may have identified something in SoapUI as an issue. If the Test Case fails, see if DevTest also throws the Socket Exception or if a different exception occurs. This may give a better view of what is happening.
3) Turn on a packet sniffing tool (such as Wireshark on Windows OS) and send your request. I would sniff on the DevTest Server side so you can see what is happening at the Server rather than the client. You should see the HTTP request packet arrive and the service try to send its response packets. Reviewing the packets on the server side may provide insight as to what / when / where the failure occurs.
Hi Joel,
I created a SOA raw request test case into of workstation, for the two first request in the same endpoint are working because the response is approximately 2500 bytes but the third service is not working the test case didn't return the response. Besides, the third service into the workstation didn't show an error message. I think that the server is not the issue.
Please, let me know if you have any idea.
Regards,
Hi Rosa, to be sure I understand.
- You created a RAW Soap request step that you submit from Workstation to your virtual service endpoint running on the VSE server.
- You send two transactions to the endpoint and they both return the responses. The responses are small (2,500+/- bytes).
- You send a request to the same service endpoint for the large response, but you do not see the response in Workstation. Nor, do you see an error in Workstation.
Questions:
1) Do you know if the logging.properties for your VSE is set to 'info'? If so, you may see useful output in the log files.
2) Do you have access to the VSE log files (vse.log, vse_matches.log, and the log for your specific service) on the server?
Debugging approach: Review the DevTest Log Files
Out of the box, the log files are usually placed in a directory inside LISA_HOME\lisatmp_*** where '***' is your version of DevTest. Sometimes, admins will move these logs outside of LISA_HOME so the location is dependent on how your installation is configured. For Workstation installations, on Windows, the log files are placed in c:\Users\<yourUID>\lisatmp_***. Note: Services running in VSE will not place output on your Workstation.
Log files to research:
vse.log = contains information about the VSE and heartbeats on services running on the VSE
VS_<yourServiceName>.log = a log file specific to your service that displays incoming request information and VSI matching information
vse_matches.log = a log file that often displays the 'response on the wire' sent by the service to the client
I would start with the vse_matches.log. Tail the end of the file right after sending your Raw Soap request for the large response, and see if you can see the "Response on the Wire" for your large payload. Because of size of output, it may take the logger several seconds to write this into the log. The output will contain something that follows a pattern similar to the following:
2016-04-04 15:03:29,058Z (10:03)[<yourServiceName> [VS_<yourServiceName>_Run]/1] INFO - Transaction Navigator Respond from: { ... some info about the VSI response... }
2016-04-04 15:03:29,075Z (10:03)[<yourServiceName> [VS_<yourServiceName>_Run]/1] INFO - Response on the Wire HTTP/1.1 <yourHTTPResponseCode> <yourHTTPResponseText>
Server: LISA/Virtual-Environment-Server
Date: Mon, 04 Apr 2016 10:03:29 CDT
X-Powered-By: LISA/9.1.0 (9.1.0.399)
Content-Type: text
Content-Length: 3717
<other meta data if your service has any>
< Followed by the response that was put on the wire. <-- this is the actual response your VSI matched on so it should be 14+ MB in size >
Also, look in these log files for Exceptions at the time you send your request. Something may be throwing an exception that is not passed back to your client. The date/time stamp in the logs will help you find this.
Potential Log File Problem: Because your third response is so large (14+MB), the logger may roll one or more of the above log files. In the DevTest logging.properties file on the server, the logs roll by default when they reach approximately 10MB in size. Your admin may or may not have changed this setting. If a high number of requests and/or large response payloads are being sent, your log files may be rolling such that you may have to look at multiple files such as vse.log, vse.log.1, vse.log.2 and so on. Similarly, your VS_<yourServiceName>.log may be rolling over as well. If the Admins have set logging from 'INFO' to 'WARN' or 'ERROR', you need to ask if they can switch to 'INFO' while you run a test and look at the output. You can apply changes to logging.properties while the VSE is running. VSE and services DO NOT need to be stopped / started. Note also that your vse_matches.log could have grown to a size that a Windows-based editor like Notepad cannot open it.
If your issue cannot be resolved by researching the logs, contact one of your Virtual Services developers for assistance. I would still suggest running the service locally in ITR mode so you can send your third Soap request and verify everything is good. If the service developer cannot help you find the issue, you may need to open a support ticket with CA for assistance.
Hi Joel,
I looked for into the logs files on server and I found "java.lang.OutOfMemory Error: Java heap space", the response file has 20MB.
2016-04-05 19:16:00,486Z (19:16) [SI-Read-Thread] ERROR System.err - Exception in thread "SI-Read-Thread" java.lang.OutOfMemoryError: Java heap space 2016-04-05 19:16:00,488Z (19:16) [Event Sink Thread Pool Thread 4] ERROR com.itko.lisa.simulator.SimulatorImpl - Got exception in pulse(), Coordinator died? java.rmi.RemoteException: Attempting to call method on local object, but object not found in colocated objects - it must have been deregistered: tcp://pasondemand44.economicalinsurance.com:2013/6DDED446FB6011E586FE02423CBD4B1B at com.itko.lisa.net.ClientBase.invokeMethod(ClientBase.java:269) at com.itko.lisa.net.ClientBase.invokeMethod(ClientBase.java:244) at com.itko.lisa.net.CoordinatorClient.getStatus(CoordinatorClient.java:203) at com.itko.lisa.simulator.SimulatorImpl.pulse(SimulatorImpl.java:639) at com.itko.util.Pulse.execPulse(Pulse.java:521) at com.itko.util.Pulse.eventReceipt(Pulse.java:491) at com.itko.util.EventThread.execEvents(ThreadedEventSink.java:79) at com.itko.util.EventThread.run(ThreadedEventSink.java:59)
What do you recommend for handling big responses?
Regards,
Hum, that's a new one. I have seen messages in the log such as "Whoa, this is a big response" or something to that affect. I have not seen an exception as a result though.
I would suggest using a VMOPTIONS file on your VSE and increase the memory. NOTE: You should review the settings below and ensure that your equipment has enough RAM to support any configuration change you make. An admin can help you determine the amount of available memory on the machine and advise on the proper settings.
In LISA_HOME/bin directory
1) Edit the file named VirtualServiceEnvironment.vmoptions
2) On a line by itself, add a directive
-Xmx2048m where -Xmx adjusts the maximum memory and 2048m = 2 GB memory. You can also use 1024m for 1GB, 3076m for 3 GB and so on
3) Save the file
4) Edit the file named VirtualServiceEnvironmentService.vmoptions
- Apply the same settings so both stay in sync. Also, I don't know if you are running VSE as a service or not.
5) Stop and Start your VSE service
6) Try your test again and verify logs.
If an -Xmx vmoptions memory setting has been applied (say 2GB or more), then I believe you have a different issue that needs to be diagnosed. Your large response is simply 'pushing it over the edge'.
Hi Joel,
We updated the VirtualServiceEnvironment.vmoptions to -Xmx2048m but it is not working.
It shows again "java.lang.OutOfMemory Error: Java heap space stream" issue.
I am using DevTest solutions 8.5.0.289.
What do you recommend for this case?.
Regards,
I would suggest that you open a support ticket with CA Support. Someone from CA can attend an online web sharing session where you can demo the error.
It is also a good idea to go ahead and zip the log file and upload it with the ticket so the support team can review the error.
I am aware of one client that had 20-100mb response file sizes so I know it is possible.
Hi Joel,
I have resolved the issue, it is a LISA issue when the XML file is big.
For this solution, I have updated the image and I have changed the XML document to JSON in the body and I have added the content type to "text/xml".
Regards,
Rosa Hormazabal
Hi Joel,
I have resolved the issue, it is a LISA issue when the XML file is big.
For this solution, I have updated the image and I have changed the XML document to JSON in the body and I have added the content type to "text/xml".
Regards,
Rosa Hormazabal