Automic Workload Automation

Hi.

Can anyone, from Automic or otherwise, share any insights into the following:

What is the role of DWP specificially with Version 12 of AE, and what does it mean for sizing requirements?

With versions 11.2 and below, the DWP were said to be responsible for handling mostly communication with the "Fat Client", dealing with heavy queries etc.

With version 12, it's my understanding that AWI replaces the "Fat Client", and that AWI internally uses the REST API, which itself is provided by the JCP. What does that mean for the number of DWP, does it stay unchanged from older releases regardless?

There was a similar discussion here, but that one generated no answers.

Thanks.

Hi Carsten,

the AWI only uses the REST API partly, as not all functionality is available via the REST API yet. The communication between AWI and the backend is still mainly via the Java API. The Java API uses the same technology to communicate with the backend as the "Fat Client" did. Therefore it is still handled by the DWP. 

Regarding sizing, it of course always depends on what you are doing. If you do a lot of Search operations it will be done via the REST API and does not require DWPs anymore. However, if you do a lot of object editing operations, you will still need the DWPs as this is done via the java API. 

I hope this helps. 

Best regards,

Robert

Hi Robert,

good information, many thanks!

Best regards,

Carsten

Robert Thullner wrote:

the AWI only uses the REST API partly, as not all functionality is available via the REST API yet. The communication between AWI and the backend is still mainly via the Java API. The Java API uses the same technology to communicate with the backend as the "Fat Client" did. Therefore it is still handled by the DWP. 

 

Regarding sizing, it of course always depends on what you are doing. If you do a lot of Search operations it will be done via the REST API and does not require DWPs anymore. However, if you do a lot of object editing operations, you will still need the DWPs as this is done via the java API.

Hi Robert:

Could you provide a bit more information on how the AWI-AE communication works? Specifically, I'd like a breakdown of which requests are sent to the CP and which to the JCP. I’m interested in using this information to update my AWA v12.2 inter-process communication diagram.   If you have a few moments, I’d also like more information about the caches that the AWI maintains and uses to respond to search requests. I know the AWI keeps a cache of the AE folder tree and AE objects. Is this cache stored locally on the AWI server node? Does the AWI also cache AE task executions? How does the AWI build these caches? (Using what APIs)? How often? Is this configurable? Do these all work the same way, behind the scenes? SearchIndex Java API class  search REST API resource Search box in AWI Advanced search screen in AWI

Also, currently there is no way to run more than one JCP per AE server node, because there is only one port number (the port option under the [REST] section of the uvsrv.ini file). Does this not create a potential bottleneck? Are there plans to add the capability to run more than one JCP per AE server node?

Thanks!

Hi Michael, 

at the moment there are not many requests which use REST. In AWI 12.2 Search, all Version Control features and the new comboboxes use REST. All other requests are still on the java API. There will be more request based on REST in the upcoming AWI 12.3

Regarding caches - as far as I know, the caches are kept on the server where the JWP runs. Task executions are not cached. A cache is invalidated and rebuilt as soon as new information is received from the backend (e.g. a new folder was created). A cache is done only for the user session, not longer. 

I would like to ask, why it is important for you to have more information on our caches? Are you experiencing some inconsistencies or other problems?

SearchIndex Java API class, search REST API resource, Searchbox in AWI and advanced search all work the same way in the background. 

Currently it is not planned to run more than one JCP per AE server node. For high availability, you should run a seconde AE server and have a second JCP there. In this setup, it is important to have a load balancer which will route the requests to the correct JCP. 

Please let me know, if this information is sufficient for you or if you need further infos. 

Best regards,

Robert

Hi Robert,

Are you experiencing some inconsistencies or other problems?

FWIW, I recently noticed that when I edit e.g. an objects title, and then go to a box where I can mouse over the object and get the title displayed as a tool tip, the second-oldest edit is shown as the tool tip, at least for some time. Albeit with a version of AWI that is a few months old. We've dediced it's not something we'd actively pursue with high priority at this time though.

Best regards,

Carsten

Robert:

Thanks for the information. To answer your question, I’m interested in the underlying architecture because this knowledge helps with:

1. Troubleshooting problems
2. Designing, developing, and operating applications that interact with the Automation Engine
3. Capacity planning

I just got our v12.2.1 system fully operational again after a brief JCP outage. (The problem was that neither ucsrvcp nor ucsrvjp.jar is capable of creating new CP tables & sequences in a non-standard tablespace.) I have configured each of the two nodes to run a single JCP. Regarding this, you wrote:

For high availability, you should run a [second] AE server and have a second JCP there. In this setup, it is important to have a load balancer which will route the requests to the correct JCP. 

When the AWI makes REST calls to the AE, how does the AWI know which node to contact? There is no JCP entry in uc4config.xml as far as I know.

Hi Michael, 

when a JCP is starting up, it will register itself in one of the MQ tables (I do not know the exact one right now). This table contains all running AWA components. 

When a user logs in to the AWI, the CP will ask for available JCP endpoints and return it to the AWI. The information is retrieved from the above mentioned database table. In case you are running 2 JCP on different servers, there is no clear definition which of the JCP entries is returned first. Whatever the database returns as first entry will be sent to the AWI. 

When you use a loadbalancer, you have to configure the "host" value in the REST section of the ucsrv.ini. The host value would be the name of your load balancer. The load balancer can use the healthcheck endpoint of the REST API to decide to which JCP it will route the request. E.g. always to the first one. If the healthcheck returns that the first JCP is not available it can route the request to the second JCP. 

Please also see these links to the documentation 

https://docs.automic.com/documentation/webhelp/english/AWA/12.2/DOCU/12.2/AWA%20Guides/help.htm#AWA/Admin/inifiles/admin… 

https://docs.automic.com/documentation/webhelp/english/AWA/12.2/DOCU/12.2/AWA%20Guides/help.htm#Installation_Manual/AE/I… 

Best regards,

Robert

Carsten_Schmitz:

See also this discussion: AE & AWI sizing 

Michael_Lowry

Cheers.

I'm aware of that thread, but I'm not sure what to make of it.

Automic support told me based on my logs that I should have at least three DWP, but the ideal number is not known to us. That goes for worker processes, too. Experimentation is difficult, because we can only really experiment on the production system (due to test systems not having the same load characteristics) and there is a lack of solid measurables.

Fourtyfive (even for a large installation) seems really excessive to me. A while ago we doubled WP (not DWP mind you, but WP) based on support advise, from a single digit count to doubly so but the result was inferior performance and seemingly (even) more database deadlocks, and a health check consultant who told us to reduce the number of WP back to single digits.

Finding the right amount of the various processes would be a lot easier if there were some documented, atomic (no pun intended) performance value for each process to look at. E.g. Automic could log and average how long object searches take and display it in the product (logs or administrative view), or include a benchmark tool that measures various operations, so you can hone in on the ideal values.

As matters stand, there's several screws to turn simultaneously, and few of them are properly labeled (as evident by your efforts to document the internal workings and responsibilities of the components).

*scnr*