Automic Workload Automation

Expand all | Collapse all

Java Communications Process (JCP)

  • 1.  Java Communications Process (JCP)

    Posted Oct 02, 2017 12:18 PM
    I noticed this warning in the AE v12.1 documentation page about the new Java Communications Process:
    When using multiple JCPs it is imperative that each JCP uses its own Automation Engine configuration file (ucsrv.ini).
    This is a bit unusual, but there’s a simple reason for this restriction. As the documentation for the AE server config file ucsrv.ini makes clear, the REST section of the INI file has just a single port option. In other words, there is no way to specify multiple port numbers for JCPs as one typically does with WPs and CPs.

    Using a single canonical port number makes sense though, because this port number is the one that external applications will use when accessing the Automation Engine via the new REST API.

    This raises another question though: how does the JCP communicate with the Automation Engine database? Does the JCP connect directly to the DB using the JDBC driver?


  • 2.  Java Communications Process (JCP)

    Posted Oct 03, 2017 09:44 AM
    The documentation pages for the Service Manager SMD and SMC files currently lack examples for the JCP. However, the page Installing the JCP includes an example:
    java -Xmx512M -jar ucsrvjp.jar -Iucsrv.ini -svc%port% -rest
    I adapted this for our needs:
    VAR SRV_STARTPATH;/opt/uc4/server
    VAR SRV_INIFILE;ucsrv_EXP2-A.ini
    VAR SRV_JAVAPATH;/opt/uc4/java/jre1.8.0_131
    VAR CP_STARTCMD;*SRV_STARTPATH/ucsrvcp *SRV_STARTPATH/*SRV_INIFILE -svc%port%
    VAR WP_STARTCMD;*SRV_STARTPATH/ucsrvwp *SRV_STARTPATH/*SRV_INIFILE -svc%port%
    VAR JWP_STARTCMD;*SRV_JAVAPATH/bin/java -Xmx512M -Dsun.security.krb5.debug=true -jar *SRV_STARTPATH/ucsrvjp.jar -svc%port% -I*SRV_STARTPATH/*SRV_INIFILE
    VAR JCP_STARTCMD;*SRV_JAVAPATH/bin/java -Xmx512M -jar *SRV_STARTPATH/ucsrvjp.jar -svc%port% -I*SRV_STARTPATH/*SRV_INIFILE -rest
    VAR WP_STARTCMD_COLD;*WP_STARTCMD -parm"StartMode=Cold"
    VAR WP_STARTCMD_STOP;*WP_STARTCMD -parm"SystemStop=Yes"
    VAR WP_STARTCMD_COLDSTOP;*WP_STARTCMD -parm"StartMode=Cold;SystemStop=Yes"
    !
    DEFINE UC4 WP1;*WP_STARTCMD;*SRV_STARTPATH;START1=(Coldstart,*WP_STARTCMD_COLD);START2=(Systemstop,*WP_STARTCMD_STOP);START3=(Coldstart with Systemstop,*WP_STARTCMD_COLDSTOP)
    ! ... other commands to start WPs
    DEFINE UC4 JWP57;*JWP_STARTCMD;*SRV_STARTPATH

    ! ... other commands to start JWPs
    DEFINE UC4 CP1;*CP_STARTCMD;*SRV_STARTPATH
    DEFINE UC4 CP3;*CP_STARTCMD;*SRV_STARTPATH
    DEFINE UC4 CP5;*CP_STARTCMD;*SRV_STARTPATH

    DEFINE UC4 JCP7;*JCP_STARTCMD;*SRV_STARTPATH
    It’s also necessary to update the ucsrv.ini file to include the port number dedicated to the JCP:
    cp7=8088
    Correction:
    The port number for the JCP listed under the [PORTS] section of the INI file should be different from the one listed under the [REST] section.
    [PORTS]
    ...
    cp7=22126
    ...
    [REST]
    port=8088


  • 3.  Java Communications Process (JCP)

    Posted Oct 03, 2017 09:49 AM
    I quickly ran into another problem. When the JCP starts up, it dies immediately. The log shows the following error:
    20171003/152952.355 - U00003590 UCUDB - DB error: '22019', 'ORA-00911: invalid character
    ', '911', 'java.sql.SQLSyntaxErrorException'
    20171003/152952.355 -           SQL Statement which caused this DB error:
    20171003/152952.355 -           create table MQ1CP007 as select * from MQ1CP001 where 0=1;ALTER TABLE MQ1CP007 ADD CONSTRAINT PK_MQ1CP007 PRIMARY KEY (MQCP_PK)
    ;CREATE INDEX NK_MQ1CP007_BAcv ON MQ1CP007(MQCP_BAcv) TABLESPACE UC4_INDEX;CREATE INDEX NK_MQ1CP007_Prio ON MQ1CP007(MQCP_Priority, MQCP_SchedTime, MQCP_PK, MQCP_Status) TABLESPACE UC4_INDEX;CREATE INDEX NK_MQ1CP007_SchTime ON MQ1CP007(MQCP_SchedTime) TABLESPACE UC4_INDEX;
    20171003/152952.356 - U00045142 Cannot create table 'MQ1CP007'.
    20171003/152952.469 - U00003432 Termination of Server 'UC4_EXP2#CP007' initiated.
    This error suggests to me that the new JCP assumes that the Automation Engine database uses the default tablespace names (UC4_DATA and UC4_INDEX). We use a single customized tablespace name that is unique to each AE system. I have reported the problem to Automic Support.



  • 4.  Java Communications Process (JCP)

    Posted Oct 03, 2017 09:55 AM
    I also noticed another minor bug. If the AE server config file ucsrv.ini does not have a port defined for the JCP, the following error will appear in the JCP log:
    U00045058 Failed to get a free port for this JWP. Check the [PORTS] section of the configuration file.
    The error message should probably be updated to reflect that the same program is used for both JWP & JCP. E.g.,
    U00045058 Failed to get a free port for this JWP/JCP. Check the [PORTS] section of the configuration file.


  • 5.  Java Communications Process (JCP)

    Posted Oct 03, 2017 04:32 PM
    Hi Michael,

    I am a long-time troll of the Atomic Community but rarely post or reply.  I was searching for benefits provided by the JWP for v11/12 and stumbled on your post.  At Clemson University, we have a DEMO v12, 2 Non-Production v11 and 2 Production v11 systems currently.  All run on Oracle Enterprise Linux 6 (app servers are UEK 6) and have Oracle 12 backends.

    Below is a few screen shots of how I set up our environment.  I know the JWP is functioning because I am using it to do Secure (TLS) LDAP authentication of users.

    I have one JWP on each of the v11 systems and 5 on the Demo system (per the documentation for mid-sized AE systems).  All my JWP (v11/12) use the JDBC method using the JDBC tags in the ~/AutomationEngine/bin/ucsrv.ini....

    e3qd8z2n6mjf.pnghttps://us.v-cdn.net/5019921/uploads/editor/m4/e3qd8z2n6mjf.png" width="975">
    My JDBC section...

    i96ovfkcsdlo.pnghttps://us.v-cdn.net/5019921/uploads/editor/5t/i96ovfkcsdlo.png" width="975">

    Ports on v12...

    rjbdwcngpa0l.pnghttps://us.v-cdn.net/5019921/uploads/editor/ky/rjbdwcngpa0l.png" width="96">

    Service Manager Dialogue of v12 AE processes...

    ypxd64gtxd3a.pnghttps://us.v-cdn.net/5019921/uploads/editor/fq/ypxd64gtxd3a.png" width="475">

    JWP configuration in the uc4.smc/d...

    7nyaoonplqlq.pnghttps://us.v-cdn.net/5019921/uploads/editor/90/7nyaoonplqlq.png" width="492">

    and finally, a log file of the from JWP 002 (PID 15814 from the SMD screen shot)...

    o13xy4ak42xh.pnghttps://us.v-cdn.net/5019921/uploads/editor/qo/o13xy4ak42xh.png" width="777">6wp4w9c091oi.pnghttps://us.v-cdn.net/5019921/uploads/editor/rq/6wp4w9c091oi.png" width="777">
    Finally, the sorted list of UC4 ports that are actively listening...

    fsigatrcesbm.pnghttps://us.v-cdn.net/5019921/uploads/editor/rv/fsigatrcesbm.png" width="700">

    If you are not in a comatose state or bored out of your mind, then I hope my setup can help you with troubleshooting.  I have no idea why I have so many application processes running compared to typical V11 setups but I tried to follow the setup instructions verbatim.  Up to this point, I have simply configured the v12 application but have not put it through its paces.  My v11 engines handle kernel/OS patching for PeopleSoft and Banner but run about 1/3 of the total processes.  We have around 400-500 agents running on v11 with about 25% of them on the production instance. 




  • 6.  Java Communications Process (JCP)

    Posted Oct 03, 2017 05:16 PM

    Terry McClintock wrote:

    I am a long-time troll of the Atomic Community but rarely post or reply.  I was searching for benefits provided by the JWP for v11/12 and stumbled on your post. 

    If you are not in a comatose state or bored out of your mind, then I hope my setup can help you with troubleshooting.

    The Java Communications Process (JCP) is different from the Java Work Process (JWP). The JCP was introduced in Automation Engine v12.1, released last week. This thread is about the JCP.

    I’m quite familiar with the JWP. Here are couple of my discussion related to the JWP:



  • 7.  Java Communications Process (JCP)

    Posted Oct 04, 2017 04:34 AM
    Michael Lowry wrote:
    ...
    This raises another question though: how does the JCP communicate with the Automation Engine database? Does the JCP connect directly to the DB using the JDBC driver?
    I think I have an answer to this question:
    ...
    The log shows the following error:
    20171003/152952.355 - U00003590 UCUDB - DB error: '22019', 'ORA-00911: invalid character
    ', '911', 'java.sql.SQLSyntaxErrorException'
    20171003/152952.355 -           SQL Statement which caused this DB error:
    20171003/152952.355 -           create table MQ1CP007 as select * from MQ1CP001 where 0=1;ALTER TABLE MQ1CP007 ADD CONSTRAINT PK_MQ1CP007 PRIMARY KEY (MQCP_PK)
    ;CREATE INDEX NK_MQ1CP007_BAcv ON MQ1CP007(MQCP_BAcv) TABLESPACE UC4_INDEX;CREATE INDEX NK_MQ1CP007_Prio ON MQ1CP007(MQCP_Priority, MQCP_SchedTime, MQCP_PK, MQCP_Status) TABLESPACE UC4_INDEX;CREATE INDEX NK_MQ1CP007_SchTime ON MQ1CP007(MQCP_SchedTime) TABLESPACE UC4_INDEX;
    20171003/152952.356 - U00045142 Cannot create table 'MQ1CP007'.
    20171003/152952.469 - U00003432 Termination of Server 'UC4_EXP2#CP007' initiated
    ...
    From this it looks like the JCP does indeed use JDBC to communicate directly with the AE database, using the details specified in the JDBC section of the ucsrv.ini file.

    Incidentally, this SQL statement is trying to create a new table in the database. Usually, only the AE DB Load program ucybdbld does this sort of thing.

    Update:
    It turns out that the JCP opens two listening sockets: one for communication with other AE processes, and a second for receiving REST requests. These two port numbers must be different.


  • 8.  Java Communications Process (JCP)

    Posted Oct 04, 2017 08:30 AM
    The problem with the creation of MQ tables for CPs is documented in PRB00138707. There is a work-around. Contact Automic Support if you are experiencing this problem.


  • 9.  Java Communications Process (JCP)

    Posted Oct 04, 2017 11:46 AM
    The port number for the JCP listed under the [PORTS] section of the INI file must be different from the one listed under the [REST] section. E.g.:
    [PORTS]
    ...
    cp7=22126
    ...
    [REST]
    port=8088
    It works like this:
    1. When the JCP starts, it first binds to the port listed in the [PORTS] section of the ucsrv.ini file, and begins to establish connections to the non-JWP work processes.
    2. The JCP then starts the REST server, which binds to the port listed in the [REST] section of the ucsrv.ini file.
    If these two ports are not different, then the JCP will abort shortly after starting, with the following error message:
    20171004/165031.340 -           Jetty: FAILED SocketConnector@0.0.0.0:8088: java.net.BindException: Address already in use (Bind failed)
    20171004/165031.341 - U00045014 Exception 'java.net.BindException: "Address already in use (Bind failed)"' at 'java.net.PlainSocketImpl.socketBind()'.
    20171004/165031.341 -           Jetty: FAILED org.eclipse.jetty.server.Server@ceede0b: java.net.BindException: Address already in use (Bind failed)
    20171004/165031.341 - U00045014 Exception 'java.net.BindException: "Address already in use (Bind failed)"' at 'java.net.PlainSocketImpl.socketBind()'.
    20171004/165031.342 - U00045014 Exception 'java.net.BindException: "Address already in use (Bind failed)"' at 'java.net.PlainSocketImpl.socketBind()'.
    20171004/165031.342 - U00003620 Routine 'com.automic.rest.RestComponent' forces trace because of error.



  • 10.  Re: Java Communications Process (JCP)

    Posted Feb 04, 2019 04:18 AM

    In October 2017, I reported that JCP v12.1 was able to create its own DB objects, but that the JCP assumes default table space names (UC4_DATA and UC4_INDEX). This means that if you assign a JCP a number higher than 5 and use non-default table space names, the JCP will fail to create the required DB objects.

     

    Today I saw the problem also in v12.2.1.

    20190204/093425.995 - 26     U00003486 ListenSocket with port number '0.0.0.0:22129' successfully created.
    20190204/093425.995 - 26     U00003486 ListenSocket with port number '0.0.0.0:22129' successfully created.
    20190204/093426.000 - 26               JWP/JCP registered as AE_EXP2 ae_exp2.mycompany.com 192.168.34.56
    20190204/093426.008 - 26     U00003488 Server 'JCP ae_exp2.mycompany.com 192.168.34.56 ' was started as instance '10'.
    20190204/093426.019 - 26     U00003590 UCUDB - DB error: '42000', 'ORA-00959: tablespace 'UC4_DATA' does not exist
    20190204/093426.019 - 26     ', '959', 'java.sql.SQLSyntaxErrorException'
    20190204/093426.019 - 26               SQL Statement which caused this DB error:
    20190204/093426.021 - 26               CREATE TABLE MQ1CP010( MQCP_PK NUMBER(38,0) NOT NULL, MQCP_System VARCHAR2 (8 CHAR) NULL, MQCP_CAddr VARCHAR2 (32 CHAR) NULL, MQCP_CSRName VARCHAR2 (8 CHAR) NULL, MQCP_CAcv NUMBER(38,0) NULL, MQCP_BAddr VARCHAR2 (32 CHAR) NULL, MQCP_BSRName VARCHAR2 (8 CHAR) NULL, MQCP_BAcv NUMBER(38,0) NULL, MQCP_FAddr VARCHAR2 (32 CHAR) NULL, MQCP_LogAddr VARCHAR2 (32 CHAR) NULL, MQCP_PhysAddr VARCHAR2 (32 CHAR) NULL, MQCP_BTable VARCHAR2 (8 CHAR) NULL, MQCP_SchedTime DATE NULL, MQCP_Status NUMBER(38,0) NULL, MQCP_Priority NUMBER(38,0) NULL, MQCP_DRole VARCHAR2 (32 CHAR) NULL, MQCP_LAddr VARCHAR2 (32 CHAR) NULL, MQCP_Len NUMBER(38,0) NULL, MQCP_Msg BLOB NULL, CONSTRAINT PK_MQ1CP010 PRIMARY KEY ( MQCP_PK ) using index TABLESPACE UC4_INDEX) LOB(MQCP_Msg) STORE AS(ENABLE STORAGE IN ROW CACHE) TABLESPACE UC4_DATA ENABLE ROW MOVEMENT NOCOMPRESS
    20190204/093426.022 - 26     U00045142 Cannot create table 'MQ1CP010'.
    Logfile changed to /var/uc4/server/CPsrv_EXP2_log_010_00.txt
    20190204/093426.577 - 31     U00003432 Termination of Server 'AE_EXP2#CP010' initiated.
    20190204/093426.611 - 31     U00045000 Database connection with session ID '3877' was closed.
    20190204/093426.613 - 31     U00003410 Server 'UC4_EXP2#CP010' version '12.2.1+build.1539954746581' ended abnormally.

     

    I have reported this problem to CA Support.



  • 11.  Re: Java Communications Process (JCP)

    Posted Feb 04, 2019 04:22 AM

    While investigating the aforementioned JCP problem related to tablespace names, I discovered something new:

    UC4 AE_EXP2#CP-Server Version 12.2.1+build.1539954746581 (PID=29532)
    Further console output supressed. run with -DlogToConsole to also log to console
    Logfile changed to /var/uc4/server/CPsrv_EXP2_log_010_00.txt

     

    The logToConsole Java system property can be used to instruct the JCP to write logging information to standard output. I tested it, and it works. E.g.,

    /usr/bin/java -DlogToConsole -Xmx512M -jar /opt/uc4/ucsrvjp.jar -svc%port% -I/opt/uc4/ucsrv.ini -rest

    With this system property enabled, the JCP will write logging information to both the log file and standard output.



  • 12.  Re: Java Communications Process (JCP)

    Posted Feb 04, 2019 12:13 PM

    Knowledge base article KB000112281 – SQL statements to create additional CP has instructions for creating the missing DB objects when creating CPs with numbers greater than 5. I tried this with my v12.2.1 system, and it appears to have worked fine.



  • 13.  Re: Java Communications Process (JCP)

    Posted Feb 11, 2019 05:45 AM

    I received confirmation from CA Support that KB000112281 is also valid for v12.2. Presumably it is valid for 12.1 as well.



  • 14.  Re: Java Communications Process (JCP)

    Posted Feb 11, 2019 05:47 AM

    This morning I create the following JCP-related enhancement request:

    AWI: display informative error message if no JCP is running 



  • 15.  Re: Java Communications Process (JCP)

    Posted Feb 12, 2019 08:09 AM

    If you use non-standard table space names in your Oracle AE DB, CA has put together instructions for how to modify the SQL the AE uses to create new DB objects for new CPs.

    KB000126471 – JCP/CP table creation assumes default tablespace names

     

    The SQL statement mentioned in that KB article is stored in the initial data file utilities/db/general/12.2/uc_ini.txt:

    3594 F001CCREATE_MQCP
    3595 F002CORACLE
    3596 F003CCREATE TABLE &NEW_MQ#( MQCP_PK NUMBER(38,0) NOT NULL, MQCP_System VARCHAR2 (8 CHAR) NULL, MQCP_CAddr VARCHAR2 (32 CHAR) NULL, MQCP_CSRName VARCHAR2 (8 CHAR) NULL, MQCP_CAcv NUMBER(38,0) NULL, MQCP_BAddr VARCHAR2 (32 CHAR) NULL, MQCP_BSRName VARCHAR2 (8 CHAR) NULL, MQCP_BAcv NUMBER (38,0) NULL, MQCP_FAddr VARCHAR2 (32 CHAR) NULL, MQCP_LogAddr VARCHAR2 (32 CHAR) NULL, MQCP_PhysAddr VARCHAR2 (32 CHAR) NULL, MQCP_BTable VARCHAR2 (8 CHAR) NULL, MQCP_SchedTime DATE NULL, MQCP_Status NUMBER(38,0) NULL, MQCP_Priority NUMBER(38,0) NULL, MQCP_DRole VARCHAR2 (32 CHAR) NULL , MQCP_LAddr VARCHAR2 (32 CHAR) NULL, MQCP_Len NUMBER(38,0) NULL, MQCP_Msg BLOB NULL, CONSTRAINT PK_&NEW_MQ# PRIMARY KEY ( MQCP_PK ) using index TABLESPACE UC4_INDEX) LOB(MQCP_Msg) STORE AS(ENABLE STORAGE IN ROW CACHE) TABLESPACE UC4_DATA ENABLE ROW MOVEMENT NOCOMPRESS ; CREATE SEQUENCE SQ_&NEW_MQ# INCREMENT BY 1 START WITH 1 MAXVALUE 2147483647MINVALUE 1 CYCLE CACHE 1000 NOORDER ;CREATE INDEX NK_&NEW_MQ#_BAcv ON &NEW_MQ#(MQCP_BAcv) TABLESPACE UC4_INDEX ;CREATE INDEX NK_&NEW_MQ#_Prio ON &NEW_MQ#(MQCP_Priority, MQCP_SchedTime, MQCP_PK, MQCP_Status) TABLESPACE UC4_INDEX ;CREATE INDEX NK_&NEW_MQ#_SchTime ON &NEW_MQ#(MQCP_SchedTime) TABLESPACE UC4_INDEX ;
    3597 R
    This is the only place in the uc_ini.txt file where the table space names are mentioned. We’re already updating table space names in all initial data .sql files. If we also change the table space names in the uc_ini.txt file prior to the DB load, then we won’t need to change the value later in the DB. We’ll consider this approach because it will make upgrades easier.