AnsweredAssumed Answered

JDEdwards RA random failures

Question asked by admin on Feb 26, 2018
Latest reply on Oct 17, 2018 by Roney.Uba

Like • Show 0 Likes0
Comment • 3

Good Afternoon,
I currently have a ticket open already for this, but if anyone else has had this issue, please let me know.
We just updated our Production environment from 11.2.1 to 11.2.6HF1 and also our Agents/ServiceManagers and RAs (including Informatica and JDEdwards).
We are now having random JDE job failures with this error:

+++++++++++++++++++++++++++++++++++++ Start of Runtime Exception ++++++++++++++++++++++++++++++++++++

org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 52; XML document structures must start and end within the same entity.

at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(Unknown Source)

at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Source)

at groovy.util.XmlParser.parse(XmlParser.java:175)

at groovy.util.XmlParser.parseText(XmlParser.java:208)

at groovy.util.XmlParser$parseText.call(Unknown Source)

at JDEdwardsAgentImpl.checkResultXML(JDEdwardsAgent.groovy:393)

at JDEdwardsAgentImpl$checkResultXML.callCurrent(Unknown Source)

at JDEdwardsAgentImpl.generateRequestXML(JDEdwardsAgent.groovy:740)

at JDEdwardsAgentImpl$generateRequestXML.call(Unknown Source)

at JDEJob.runJob(JDEdwardsAgent.groovy:3420)

at com.uc4.extensibility.types.CITAgent.dispatchJob(CITAgent.java:258)

at com.uc4.ex.cit.CITJob.execute(CITJob.java:196)

at com.uc4.ex.Job.run(Job.java:469)

at com.uc4.ex.ThreadPool$RunnableDecorator.run(ThreadPool.java:53)

at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)

at java.util.concurrent.FutureTask.run(Unknown Source)

at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

at java.lang.Thread.run(Unknown Source)

++++++++++++++++++++++++++++++++++++++ End of Runtime Exception ++++++++++++++++++++++++++++++++++++++

Though I'm waiting for support, if someone has come across this before, please let me know what you have found to work. I will also post what support finds out for us to see if it helps anyone in the future.
Thanks!!

No one else has this question

Outcomes

Carsten_Schmitz Feb 26, 2018 8:46 AM

Hey,

We don't have JDEdwards, but these xml documents that the parser complains about, it appears they are generated as part of the job. They should be somewhere on the disk, probably on the same server the agent is on, and are probably named base.xml, request.xml and (if it even gets this far) result.xml.

Have you had a look for (and at) those files? Looking at them e.g. with xmllint, or by visual inspection might hold a clue to what's wrong.

Hth,
Carsten

Actions

admin Feb 26, 2018 3:13 PM

Hello,

So, as usual, fantastic support experience on our end!

Even though this was the RA, the xml's that helped find the issue were actually in the base agents temp folder and not the RA temp folder. We were able to get even more details with support when we ran a trace on the RA that had the few failing jobs.

There is a new feature in the JDE RA that appears to be encrypting the XML file and creating 2 files. In a few of the job cases, the XML files were incomplete and causing a job failure. These jobs were also not executing in JDE as a result. Working with support, they were able to provide us with an amendment to the ini file for the RA. He had us add encrypt=false at the bottom of the [RA] section, bounce the RA Agents in question, and our jobs then ran successfully.

We are going to keep working with support to try reproducing this in our DEV environment so that they can find a possible root cause. But changing that setting in our environment had no adverse effects.

I hope this can help someone in the future!

Actions

Roney.Uba Oct 17, 2018 11:57 AM

Hey, @Antoine_Sauteron was helping me troubleshooting problems with the JDE Jobs and we did some findings.

- With the Agent trace set to ra=9 , the job report itself contains relevant debug messages that can help.

- I've also set the encrypt=false and this prevented one of the errors to happen:

2018-10-17 20:53:22 pool-1-thread-2: JDEdwardsAgentImpl: Run Encrypt Command? true
2018-10-17 20:53:22 pool-1-thread-2: JDEdwardsAgentImpl: Execute Encrypt:E:\JDEdwards\E900\DDP\system\bin32\runubexml E ENCRYPT_V1 E:\Automic\Agents\RA.JD.Edwards\work\UC4266405675587824437.tmp
2018-10-17 20:53:22 pool-1-thread-2: JDEdwardsAgentImpl: Runubexml Encryption Failed:null Exception: java.io.IOException: Cannot run program "E:\JDEdwards\E900\DDP\system\bin32\runubexml" (in directory "E:\JDEdwards\E900\DDP\system\bin32"): CreateProcess error=193, %1 is not a valid Win32 application

2018-10-17 21:41:47 pool-1-thread-4: JDEdwardsAgentImpl: Run Encrypt Command? false

- The command to submit the jobs was executed, but there was some problem with the user/password/environment combination. I've tested to run the command directly from the cmd, and the same error happens. I've also tested with another user/password/environment, running directly from the command line, and the job was successfully submitted, so I now that the runubexml is working.

2018-10-17 21:41:47 pool-1-thread-4: JDEdwardsAgentImpl: Return code:1
2018-10-17 21:41:47 pool-1-thread-4: JDEdwardsAgentImpl: runubexml: Could not initialize an ENV handle for User in Environment

- Even with the return message from the debug report, the Agent log contains a java error that is misleading:

It is weird because this is caused by the generateRequestXML, but the Xml request was already generated and submitted and the runubexml returned the return code 1.

Maybe it's because the response xml file is empty and there is no xml structure to be processed, but if this is the case, the Agent needs to be fixed.

groovy.lang.MissingPropertyException: No such property: t for class: JDEdwardsAgentImpl
Possible solutions: type
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.unwrap(ScriptBytecodeAdapter.java:53)
at org.codehaus.groovy.runtime.callsite.GetEffectivePogoPropertySite.getProperty(GetEffectivePogoPropertySite.java:87)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callGroovyObjectGetProperty(AbstractCallSite.java:307)
at JDEdwardsAgentImpl.generateRequestXML(JDEdwardsAgent.groovy:713)

I hope this helps on further troubleshooting.

Cheers,

Roney Uba

PS:

20181017/213756.703 - U02000000 Started program Agent 'JDE', version '12.2.0+build.970', changelist '1528885996'.
20181017/213756.718 - U02000232 Build Date: '2018-06-13', '13:12:07'
20181017/213756.718 - U02000090 Java Runtime Environment version: '1.8.0_181'
20181017/213756.718 - U02000091 Java Runtime Environment vendor: 'Oracle Corporation'
20181017/213756.718 - U02000192 Operating system: 'Windows Server 2012 R2', version: '6.3'.
20181017/213756.718 - U02000193 JVM Architecture: 'amd64'
20181017/213756.718 - U02000110 Maximum Heap Memory: '455' MB

20181017/213759.812 - U02013344 uc4-ra.jar has RA Framework version '7.1.4+build.1'
20181017/213759.812 - U02013345 'JDEdwardsAgent' requires RA Framework '6.4'
20181017/213759.812 - JDEdwardsAgent 2.1.1+build.802
20181017/213759.812 - Build timestamp 2018-08-09 23:16:14.727

1 person found this helpful

Actions

3 Replies

Carsten_Schmitz Feb 26, 2018 8:46 AM
Mark Correct
Correct Answer
Hey,

We don't have JDEdwards, but these xml documents that the parser complains about, it appears they are generated as part of the job. They should be somewhere on the disk, probably on the same server the agent is on, and are probably named base.xml, request.xml and (if it even gets this far) result.xml.

Have you had a look for (and at) those files? Looking at them e.g. with xmllint, or by visual inspection might hold a clue to what's wrong.

Hth,
Carsten
Like • Show 0 Likes0
Actions
admin Feb 26, 2018 3:13 PM
Mark Correct
Correct Answer
Hello,

So, as usual, fantastic support experience on our end!

Even though this was the RA, the xml's that helped find the issue were actually in the base agents temp folder and not the RA temp folder. We were able to get even more details with support when we ran a trace on the RA that had the few failing jobs.

There is a new feature in the JDE RA that appears to be encrypting the XML file and creating 2 files. In a few of the job cases, the XML files were incomplete and causing a job failure. These jobs were also not executing in JDE as a result. Working with support, they were able to provide us with an amendment to the ini file for the RA. He had us add encrypt=false at the bottom of the [RA] section, bounce the RA Agents in question, and our jobs then ran successfully.

We are going to keep working with support to try reproducing this in our DEV environment so that they can find a possible root cause. But changing that setting in our environment had no adverse effects.

I hope this can help someone in the future!
Like • Show 0 Likes0
Actions
Roney.Uba Oct 17, 2018 11:57 AM
Mark Correct
Correct Answer
Hey, @Antoine_Sauteron was helping me troubleshooting problems with the JDE Jobs and we did some findings.

- With the Agent trace set to ra=9 , the job report itself contains relevant debug messages that can help.

- I've also set the encrypt=false and this prevented one of the errors to happen:
2018-10-17 20:53:22 pool-1-thread-2: JDEdwardsAgentImpl: Run Encrypt Command? true
2018-10-17 20:53:22 pool-1-thread-2: JDEdwardsAgentImpl: Execute Encrypt:E:\JDEdwards\E900\DDP\system\bin32\runubexml E ENCRYPT_V1 E:\Automic\Agents\RA.JD.Edwards\work\UC4266405675587824437.tmp
2018-10-17 20:53:22 pool-1-thread-2: JDEdwardsAgentImpl: Runubexml Encryption Failed:null Exception: java.io.IOException: Cannot run program "E:\JDEdwards\E900\DDP\system\bin32\runubexml" (in directory "E:\JDEdwards\E900\DDP\system\bin32"): CreateProcess error=193, %1 is not a valid Win32 application
2018-10-17 21:41:47 pool-1-thread-4: JDEdwardsAgentImpl: Run Encrypt Command? false

- The command to submit the jobs was executed, but there was some problem with the user/password/environment combination. I've tested to run the command directly from the cmd, and the same error happens. I've also tested with another user/password/environment, running directly from the command line, and the job was successfully submitted, so I now that the runubexml is working.
2018-10-17 21:41:47 pool-1-thread-4: JDEdwardsAgentImpl: Return code:1
2018-10-17 21:41:47 pool-1-thread-4: JDEdwardsAgentImpl: runubexml: Could not initialize an ENV handle for User in Environment

- Even with the return message from the debug report, the Agent log contains a java error that is misleading:
It is weird because this is caused by the generateRequestXML, but the Xml request was already generated and submitted and the runubexml returned the return code 1.
Maybe it's because the response xml file is empty and there is no xml structure to be processed, but if this is the case, the Agent needs to be fixed.
groovy.lang.MissingPropertyException: No such property: t for class: JDEdwardsAgentImpl
Possible solutions: type
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.unwrap(ScriptBytecodeAdapter.java:53)
at org.codehaus.groovy.runtime.callsite.GetEffectivePogoPropertySite.getProperty(GetEffectivePogoPropertySite.java:87)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callGroovyObjectGetProperty(AbstractCallSite.java:307)
at JDEdwardsAgentImpl.generateRequestXML(JDEdwardsAgent.groovy:713)

I hope this helps on further troubleshooting.
Cheers,
Roney Uba

PS:
20181017/213756.703 - U02000000 Started program Agent 'JDE', version '12.2.0+build.970', changelist '1528885996'.
20181017/213756.718 - U02000232 Build Date: '2018-06-13', '13:12:07'
20181017/213756.718 - U02000090 Java Runtime Environment version: '1.8.0_181'
20181017/213756.718 - U02000091 Java Runtime Environment vendor: 'Oracle Corporation'
20181017/213756.718 - U02000192 Operating system: 'Windows Server 2012 R2', version: '6.3'.
20181017/213756.718 - U02000193 JVM Architecture: 'amd64'
20181017/213756.718 - U02000110 Maximum Heap Memory: '455' MB

20181017/213759.812 - U02013344 uc4-ra.jar has RA Framework version '7.1.4+build.1'
20181017/213759.812 - U02013345 'JDEdwardsAgent' requires RA Framework '6.4'
20181017/213759.812 - JDEdwardsAgent 2.1.1+build.802
20181017/213759.812 - Build timestamp 2018-08-09 23:16:14.727
1 person found this helpful
Like • Show 0 Likes0
Actions

JDEdwards RA random failures

Outcomes

Related Content

Recommended Content