Service Virtualization

Hi Team,

Sometimes, few of the deployed services are disappearing from the portal. When I checked for the errors in logs, In VSE logs, I found that  " Failed to stage MAR java.io.FileNotFoundException: Unable to load file" error and the reference lads file is missing. So, currently I'm redeploying the service to bring back to the portal. Please find the below error and help me to resolve the issue. 

2019-04-15 09:02:31,663Z (11:02) [main] ERROR com.itko.lisa.coordinator.CoordinatorImpl - Failed to stage MAR
java.io.FileNotFoundException: Unable to load file: name=D:\DevTest\lisatmp_10.3.0\lads\33BC82425F5D11E9A719005056897A0F\<ServiceName>\VirtualServices\<ServiceName>.vsm path:[D:\DevTest\lisatmp_10.3.0\lads\33BC82425F5D11E9A719005056897A0F\<ServiceName>\VirtualServices\<ServiceName>.vsm] absolutePath:[D:\DevTest\lisatmp_10.3.0\lads\33BC82425F5D11E9A719005056897A0F\<ServiceName>\VirtualServices\<ServiceName>.vsm] canonicalPath:[D:\DevTest\lisatmp_10.3.0\lads\33BC82425F5D11E9A719005056897A0F\<ServiceName>\VirtualServices\<ServiceName>.vsm]
at com.itko.util.StreamHelp.fileOpenerWithInfo(StreamHelp.java:775)
at com.itko.util.StreamHelp.fileOpener(StreamHelp.java:670)
at com.itko.util.XMLUtils.openXMLDoc(XMLUtils.java:591)
at com.itko.lisa.LisaDocument.<init>(LisaDocument.java:56)
at com.itko.lisa.test.TestCaseDocument.<init>(TestCaseDocument.java:37)
at com.itko.lisa.test.TestCase.openTestCaseDoc(TestCase.java:1918)
at com.itko.lisa.coordinator.CoordinatorImpl.stageModelArchive(CoordinatorImpl.java:1210)
at com.itko.lisa.coordinator.VirtualServiceCoordinator.<init>(VirtualServiceCoordinator.java:152)
at com.itko.lisa.coordinator.VirtualServiceImpl.createVSCoordinator(VirtualServiceImpl.java:261)
at com.itko.lisa.coordinator.VirtualServiceImpl.<init>(VirtualServiceImpl.java:162)
at com.itko.lisa.coordinator.VirtualServiceEnvironmentImpl.loadInitialServices(VirtualServiceEnvironmentImpl.java:461)
at com.itko.lisa.coordinator.VirtualServiceEnvironmentImpl.main(VirtualServiceEnvironmentImpl.java:246)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.exe4j.runtime.LauncherEngine.launch(Unknown Source)
at com.exe4j.runtime.WinLauncher$2.run(Unknown Source)
Caused by: java.io.FileNotFoundException: D:\DevTest\lisatmp_10.3.0\lads\33BC82425F5D11E9A719005056897A0F\<ServiceName>\VirtualServices\<ServiceName>.vsm (The system cannot find the path specified)
at java.io.FileInputStream.open0(Native Method)
at java.io.FileInputStream.open(FileInputStream.java:195)
at java.io.FileInputStream.<init>(FileInputStream.java:138)
at java.io.FileInputStream.<init>(FileInputStream.java:93)
at com.itko.util.StreamHelp.fileOpenerWithInfo(StreamHelp.java:761)
... 17 more
2019-04-15 09:02:31,669Z (11:02) [main] WARN com.itko.lisa.coordinator.CoordinatorImpl - Test set to end but we have no load pattern (yet?).
2019-04-15 09:02:31,675Z (11:02) [main] WARN com.itko.lisa.test.LisaException - LisaException detail Could not stage: Unable to load file: name=D:\DevTest\lisatmp_10.3.0\lads\33BC82425F5D11E9A719005056897A0F\<ServiceName>\VirtualServices\<ServiceName>.vsm path:[D:\DevTest\lisatmp_10.3.0\lads\33BC82425F5D11E9A719005056897A0F\<ServiceName>\VirtualServices\<ServiceName>.vsm] absolutePath:[D:\DevTest\lisatmp_10.3.0\lads\33BC82425F5D11E9A719005056897A0F\<ServiceName>\VirtualServices\<ServiceName>.vsm] canonicalPath:[D:\DevTest\lisatmp_10.3.0\lads\33BC82425F5D11E9A719005056897A0F\<ServiceName>\VirtualServices\<ServiceName>.vsm] exception is java.io.FileNotFoundException: Unable to load file: name=D:\DevTest\lisatmp_10.3.0\lads\33BC82425F5D11E9A719005056897A0F\<ServiceName>\VirtualServices\<ServiceName>.vsm path:[D:\DevTest\lisatmp_10.3.0\lads\33BC82425F5D11E9A719005056897A0F\<ServiceName>\VirtualServices\<ServiceName>.vsm] absolutePath:[D:\DevTest\lisatmp_10.3.0\lads\33BC82425F5D11E9A719005056897A0F\<ServiceName>\VirtualServices\<ServiceName>.vsm] canonicalPath:[D:\DevTest\lisatmp_10.3.0\lads\33BC82425F5D11E9A719005056897A0F\<ServiceName>\VirtualServices\<ServiceName>.vsm]
2019-04-15 09:02:31,678Z (11:02) [main] ERROR com.itko.lisa.coordinator.VirtualServiceEnvironmentImpl - Could not load virtual service 'ZSMART_Volume_<ServiceName> 2'.

============================================================================
| Exception:
============================================================================
| Message: Could not stage: Unable to load file: name=D:\DevTest\lisatmp_10.3.0\lads\33BC82425F5D11E9A719005056897A0F\<ServiceName>\VirtualServices\<ServiceName>.vsm path:[D:\DevTest\lisatmp_10.3.0\lads\33BC82425F5D11E9A719005056897A0F\<ServiceName>\VirtualServices\<ServiceName>.vsm] absolutePath:[D:\DevTest\lisatmp_10.3.0\lads\33BC82425F5D11E9A719005056897A0F\<ServiceName>\VirtualServices\<ServiceName>.vsm] canonicalPath:[D:\DevTest\lisatmp_10.3.0\lads\33BC82425F5D11E9A719005056897A0F\<ServiceName>\VirtualServices\<ServiceName>.vsm]
----------------------------------------------------------------------------
| Trapped Exception: Unable to load file: name=D:\DevTest\lisatmp_10.3.0\lads\33BC82425F5D11E9A719005056897A0F\<ServiceName>\VirtualServices\<ServiceName>.vsm path:[D:\DevTest\lisatmp_10.3.0\lads\33BC82425F5D11E9A719005056897A0F\<ServiceName>\VirtualServices\<ServiceName>.vsm] absolutePath:[D:\DevTest\lisatmp_10.3.0\lads\33BC82425F5D11E9A719005056897A0F\<ServiceName>\VirtualServices\<ServiceName>.vsm] canonicalPath:[D:\DevTest\lisatmp_10.3.0\lads\33BC82425F5D11E9A719005056897A0F\<ServiceName>\VirtualServices\<ServiceName>.vsm]
| Trapped Message: java.io.FileNotFoundException: Unable to load file: name=D:\DevTest\lisatmp_10.3.0\lads\33BC82425F5D11E9A719005056897A0F\<ServiceName>\VirtualServices\<ServiceName>.vsm path:[D:\DevTest\lisatmp_10.3.0\lads\33BC82425F5D11E9A719005056897A0F\<ServiceName>\VirtualServices\<ServiceName>.vsm] absolutePath:[D:\DevTest\lisatmp_10.3.0\lads\33BC82425F5D11E9A719005056897A0F\<ServiceName>\VirtualServices\<ServiceName>.vsm] canonicalPath:[D:\DevTest\lisatmp_10.3.0\lads\33BC82425F5D11E9A719005056897A0F\<ServiceName>\VirtualServices\<ServiceName>.vsm]
----------------------------------------------------------------------------
STACK TRACE
java.io.FileNotFoundException: Unable to load file: name=D:\DevTest\lisatmp_10.3.0\lads\33BC82425F5D11E9A719005056897A0F\<ServiceName>\VirtualServices\<ServiceName>.vsm path:[D:\DevTest\lisatmp_10.3.0\lads\33BC82425F5D11E9A719005056897A0F\<ServiceName>\VirtualServices\<ServiceName>.vsm] absolutePath:[D:\DevTest\lisatmp_10.3.0\lads\33BC82425F5D11E9A719005056897A0F\<ServiceName>\VirtualServices\<ServiceName>.vsm] canonicalPath:[D:\DevTest\lisatmp_10.3.0\lads\33BC82425F5D11E9A719005056897A0F\<ServiceName>\VirtualServices\<ServiceName>.vsm]
at com.itko.util.StreamHelp.fileOpenerWithInfo(StreamHelp.java:775)
at com.itko.util.StreamHelp.fileOpener(StreamHelp.java:670)
at com.itko.util.XMLUtils.openXMLDoc(XMLUtils.java:591)
at com.itko.lisa.LisaDocument.<init>(LisaDocument.java:56)
at com.itko.lisa.test.TestCaseDocument.<init>(TestCaseDocument.java:37)
at com.itko.lisa.test.TestCase.openTestCaseDoc(TestCase.java:1918)
at com.itko.lisa.coordinator.CoordinatorImpl.stageModelArchive(CoordinatorImpl.java:1210)
at com.itko.lisa.coordinator.VirtualServiceCoordinator.<init>(VirtualServiceCoordinator.java:152)
at com.itko.lisa.coordinator.VirtualServiceImpl.createVSCoordinator(VirtualServiceImpl.java:261)
at com.itko.lisa.coordinator.VirtualServiceImpl.<init>(VirtualServiceImpl.java:162)
at com.itko.lisa.coordinator.VirtualServiceEnvironmentImpl.loadInitialServices(VirtualServiceEnvironmentImpl.java:461)
at com.itko.lisa.coordinator.VirtualServiceEnvironmentImpl.main(VirtualServiceEnvironmentImpl.java:246)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.exe4j.runtime.LauncherEngine.launch(Unknown Source)
at com.exe4j.runtime.WinLauncher$2.run(Unknown Source)
Caused by: java.io.FileNotFoundException: D:\DevTest\lisatmp_10.3.0\lads\33BC82425F5D11E9A719005056897A0F\<ServiceName>\VirtualServices\<ServiceName>.vsm (The system cannot find the path specified)
at java.io.FileInputStream.open0(Native Method)
at java.io.FileInputStream.open(FileInputStream.java:195)
at java.io.FileInputStream.<init>(FileInputStream.java:138)
at java.io.FileInputStream.<init>(FileInputStream.java:93)
at com.itko.util.StreamHelp.fileOpenerWithInfo(StreamHelp.java:761)
... 17 more
============================================================================

at com.itko.lisa.coordinator.CoordinatorImpl.stageModelArchive(CoordinatorImpl.java:1255)
at com.itko.lisa.coordinator.VirtualServiceCoordinator.<init>(VirtualServiceCoordinator.java:152)
at com.itko.lisa.coordinator.VirtualServiceImpl.createVSCoordinator(VirtualServiceImpl.java:261)
at com.itko.lisa.coordinator.VirtualServiceImpl.<init>(VirtualServiceImpl.java:162)
at com.itko.lisa.coordinator.VirtualServiceEnvironmentImpl.loadInitialServices(VirtualServiceEnvironmentImpl.java:461)
at com.itko.lisa.coordinator.VirtualServiceEnvironmentImpl.main(VirtualServiceEnvironmentImpl.java:246)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.exe4j.runtime.LauncherEngine.launch(Unknown Source)
at com.exe4j.runtime.WinLauncher$2.run(Unknown Source)

Hi Surya

           As you may be already aware, the files in lads folder are temporary and these are created when the VSE service starts. These files are cleaned up when VSE service shuts down. These files in lads folder are necessary for the Virtual Service to function properly.

Please could you check if the files in lads folders are deleted unintentionally ?

regards

Sankar

Hi Sankar,

I checked with the team and confirmed that, we've not deleted the files, but still facing the issue.

This issue is also happening when we restart the server. Please guide me on, Is there any other way to find the RC to this issue ?

Hi Surya

           Please kindly follow the steps below:

1. Stop VSE Service

2. Check the contents of lads folder

3. Ideally, this folder should be empty when the VSE Service is shut

4. If the folder still has some files, move all those remaining files to a different location so as to make sure the lads folder is clean

5. Start VSE Service

6. Observe the environment for re-occurance of the original problem

regards

Sankar

If Sankar's suggestion doesn't solve the issue then I would have a look if the .mari file inside your .mar file is "functionally ok.

I  say this because I'm puzzled why the <ServiceName> is not instantiated with an actual service name in the absolute path within the java error description:

java.io.FileNotFoundException: Unable to load file: name=D:\DevTest\lisatmp_10.3.0\lads\33BC82425F5D11E9A719005056897A0F\<ServiceName>\VirtualServices\<ServiceName>.vsm path:[D:\DevTest\lisatmp_10.3.0\lads\33BC82425F5D11E9A719005056897A0F\<ServiceName>\VirtualServices\<ServiceName>.vsm] absolutePath:

Rename your .mar file to a .zip file, open the zip file, and extract the .mari file, have a look inside at the xml and the data (or post it here), does everything look valid?

Cheers,

Danny

Good catch Danny !

Adding to the superb diagnosis by Danny, please check the .marinfo file and look at the tag PrimaryAsset. An example is shown below. You need to do this for ZSMART_Volume_<ServiceName> 2.mar.

Btw, is "ZSMART_Volume_<ServiceName> 2" the correct name of your virtual service ? 

<?xml version="1.0" ?>

<MarInfo>
<name>BL1</name>
<type>VIRTUAL_SERVICE</type>
<projectRoot>SE_Issues</projectRoot>
<optimized>true</optimized>
<deployInfo>
<PrimaryAsset>BL1.vsm</PrimaryAsset>
<Configuration>Configs/project.config</Configuration>
<ConcurrentCapacity>1</ConcurrentCapacity>
<ThinkTimePercent>100</ThinkTimePercent>
<AutoRestart>true</AutoRestart>
<StartOnDeploy>true</StartOnDeploy>
</deployInfo>
</MarInfo>

Hi Danny, 

In the log files, service name logged correctly. I replaced the actual service name with "<ServiceName>" due to the data privacy.