Automic Workload Automation

We are installing workload automation v12 environment and have the database and windows 2012 r2 server installed, configured and working.

I am installing and configuring the second windows server (for redundancy).  I have installed the local windows agent but when I try and start I receive the following error.

Connect-Error: 10061 (No connection could be made because the target machine actively refused it.)

We are trouble shooting but thought I would ping folks.  Has anyone run into this and it is firewall/porting or UAC issue?

It seems the UAC causes multiple problems.

Hi Fred,

That looks more like a network issue than a UAC problem. Any Firewall in between the 2 servers? Is the windows firewall active?

The servers are in the same internal DMZ so no network firewalls.  I have built server firewall rules for both (primary and now secondary)  to allow port ranges defined in AE 'UCSrv.ini' file (both WP & CP ports) and still receive the same error.  That points me to UAC but not sure at this point.  The UAC setting (prompt) impacted the starting of WP & CP the first time, that is what has me thinking that.

Do you have a Windows environment?  If so did your setup require UAC to be disabled?

from the bottom of the heart of documentation...
:-)

https://docs.automic.com/documentation/WEBHELP/English/AWA/11.2/AE/11.2/All%20Guides/help.htm#ucacpq.htm?Highlight=UAC

here are some further hints

https://community.automic.com/discussion/comment/28655#Comment_28655

regarding 2)
The amount of S class agents depends on your license model (see System overview/licenses or the license PDF from Sales Operations for more details) - and Yes, they are intended for use on the physical AE server only.

here are two very good threads with firther information about license types and usage:

https://community.automic.com/discussion/6840/licenses-faq

https://community.automic.com/discussion/comment/30905#Comment_30905

Side tracked on this.

Viad,

The first/initial server has been installed and configured for all agents and is working as expected.  The second server was installing ok until the local windows agent.

You are correct on the license issue, either way you can only have one 'S' and one 'V' license for the automation engine side.  That is the same way our v9 environment is.  I mirrored the licensing and splitting of WP/CP port ranges and received the error.  I have tried combinations of the license either way, same error. (appreciate the insight though).

 Looking at Wolfgang's post/documents.

I have read the license documents and was good information but I think I have these correct.

Scratching my head though...which ever server I start WP & CP processes on I can start that local windows agent, either class=S or V.  When I try to start the 'secondary/backup' Windows agent only it receives the connection refused error.

I only have 1 "S" license (expected) and multiple 'V' licenses which is what I trying to start the secondary server/windows agent under....either server when acting as the secondary gets connection refused.

I am not so sure this is network or UAC but a configuration item I have wrong, ideas?

Has been a long time since v9 install with two AE servers.  For the secondary server (v12) I installed utilities, automation engine, service manager, service manager dialog and UI, just like the first server only excluding the DB load, is that correct?

Hi Fred

if you are still talking from
Connect-Error: 10061 (No connection could be made because the target machine actively refused it.)

this is a network error coming from your OS.

Out of experience I would say on your physical AE Server is a firewall running that prevents the agent'a IP connection to it.

You could test by performing a telnet from your agent machine to your ae server on the CP port.
if everythins fine just one line with CP bla bla bla is returned.

Hi Fred,

That is not a license issue (as long as you have enouph of them). If you set the agents license to 'S' you would get these messages:

20170922/125940.905 - U2000012 Connection to Server 'localhost' denied.
20170922/125940.905 - U2000010 Connection to Server 'localhost:xxxx' terminated.
20170922/125940.905 - U2000074 Connecting to system 'UC4' is not possible.

And the agent would always want to connect to localhost. No matter whats in the ini file. We had that issue lately.

Regarding the error message I would still say it's a firewall, IDS ... issue as Wolfgang says. Try if you can telnet the primary server from the second machine on the cp port (should be 2217).

Built-in Windows firewall or fancy "Anti Virus Network Protection" possibly?

"because the target machine actively refused it" means that the target machine sent a TCP-RST package in response to a TCP-SYN.

Yeah ... check the local Windows firewall on that machine.

The default for incoming connections with Windows 2012 r2 is to block them, at least as per this:
https://technet.microsoft.com/en-us/library/cc753002(v=ws.11).aspx

Appreciate everyone's feedback, good stuff.

UAC is finally approved to disable, no change in behavior, was never convinced that was it...check that off the list.

I have tried about every combination of license variations and the error never changes that, in the past the error has (if I can remember) always had license verbiage and/or like the message Matthias displays.  I would say this does not apply.

We were/are working on the telnet connection and firewall ports and have strange connection ports.  The server and agent ini files have the default port ranges like 2217 - 222#.  The service manager is port 8871.  When connections start we are seeing traffic on ports in the 50### range.  These seem to be ok except for the secondary AE server trying to connect.

Not sure if anyone has insight into the 50### ports?

Its definitely no license issue (as of the error message).

****, I bet it was UAC (Did you deactivate it in Registry (and Server reboot) as well?)

Next cause could be windows firewall - can you turn it off - just for testin purposes?

You can doublecheck the entries in CP logs - there should be visible if the agent connection atempt was logged from CP.

Do other (running)  agents have any connection-issue-entries in their logs?

Regarding your 50### ports issue - honestly: No idea!
Basically the Connection is opened from Agent to AE via CP.

If you start an agent via User Interface or Script AE connects to SMGR and SMGR performs the agent start.

Yes the UAC is finally resolved (internal approval), changed via registry and not gui, server restarted.

We have been testing firewall rules with no luck, next is to remove from server baseline and disable firewall.  If that resolves then the mystery to figure out how to configure.

No other connectivity issues.  Our system based on licenses and per uc4 v9 design we are replicating, we have 2 AE servers (windows).  One is active, running all WP, CP and agents, the secondary server only runs it's local windows agent.  That is the only agent or process having issues.

With the agent linked to service manager the initial connection is coming through on port 8871 (ucybsmgr.ini).

Well, we disabled the firewalls on both servers and still the same error.

Now I am really scratching my head.

Not sure if any install malfunction could cause this or not.

Either that or I have a WP, CP port loop somehow.

I am still digging but if any ideas let me know.

Not sure about uninstall then install the local agent...but happens on either based on switching 'primary' control so I do not think that will resolve anything.

Hmmm some further attempts would be:

* pls post the latest agent logfile to get a full overview
* did you define the IP+Port or DNS Servername+Port as Cp server in INI file of the agent?
* you can define as CP server your second AE server with another CP Port (just for test)
* remove the agent and copy/paste it from scratch (or install, if you prefer)

if nothing helps I would open a ticket @Automic support.

Hi,

the telnet test would be the ultimate proof. Therefore

1. Install telnet on the server the agent is running on
2. Open a cmd and write this "telnet <ae primary servername or ip> 2217"
3. Post what you got in return

You may also post a screenshot of your server processes you can find in the "Systemoverview --> Autmation Engine" Menu. Thanks

I personally don't know anything about 50xx-style ports. 8871 is for sending control commands to the agent, 23xx and onward is usually for agent-to-agent file transfers.

In addition to what Wolfgang suggested, you could also get a (free) packet sniffer and figure out what's going on. On Windows, something like these I suppose:

https://www.wireshark.org/download.html
https://www.winpcap.org/windump/

I believe the 50,### ports are windows start up ports, then the configured Automic ports apply, not sure yet on that.  For other items here is what I have, sorry for long winded info and appreciate you replying and information.

 

 

The telnet command works, issued from server port 2217 the following is returned.

 

00000147UC4:global001NAT                        A0000091UC4_TEST#CP0

 

        UC4_TEST                        12.0 CP 0012SERVERA:2218

 

 

 

AE and agent ini files, I mirrored our current v9 environment.

 

Basically serverA (say acting primary) for the local windows agent, the ini “cp” points to itself and one of the local AE cp ports –

 

ServerB does the same pointing to itself and local port.

 

Example / listing below with agent log.

 

It always seemed odd to point to “itself” but I think that is required for the UI to work.

 

I just tested “cp” definition for serverB pointing to itself with ‘serverA’ port and the agent connects and stays connected.

 

The UI now starts but squawks about server or xml file, I need to check that.

 

 

 

Do you folks define your server ini and agent any for configuration like this?  “To itself”, make sense?

 

 

 

SERVERA

 

<UCSrv.ini>

 

PWP -2270

 

 

 

[PORTS]

 

cp1=2217

 

cp2=2218

 

cp3=2219

 

 

 

wp1=2271

 

wp2=2272

 

wp3=2273

 

wp4=2274

 

wp5=2275

 

wp6=2276

 

wp7=2277

 

wp8=2278

 

wp9=2279

 

 

 

Agent <ucxjwx6.ini>

 

licence_class=S

 

[TCP/IP]

 

port=2300

 

cp=SERVERA:2217 (points to itself & to local AE CP port)

 

 

 

 

 

 

 

 

 

SERVERB

 

<UCSrv.ini>

 

PWP -2270

 

 

 

cp3=2220

 

cp4=2221

 

cp5=2222

 

 

 

wp10=2280

 

wp11=2281

 

wp12=2282

 

wp13=2283

 

wp14=2284

 

wp15=2285

 

wp16=2286

 

wp17=2287

 

wp18=2288

 

 

 

Agent <ucxjwx6.ini>

 

licence_class=V

 

[TCP/IP]

 

port=2300

 

cp=SERVERB:2220 (points to itself & to local AE CP port)

 

 

 

SERVERB agent connection/failure log.

 

 

 

 

 

 

 

 

 

20170926/183535.913 - U02000000 Started program Agent 'WIN_STERLINGTEST', version '12.0.2+build.720', changelist '1493103340'.

 

20170926/183535.913 - U02000232 Build Date: '2017-05-24', '21:00:03'

 

20170926/183535.913 - U02000037 Started Agent with INI file 'D:\Automic\Agents\Windows\bin\UCXJWX6.ini'.

 

20170926/183535.913 -           -------------------------------------------------------------------------------------------------------------------------------

 

20170926/183535.913 -           [GLOBAL]

 

20170926/183535.913 -           name=WIN_SERVERB

 

20170926/183535.913 -           system=UC4_TEST

 

20170926/183535.913 -          

 

20170926/183535.913 -           logon=1

 

20170926/183535.913 -           language=(E,D)

 

20170926/183535.913 -           logging=..\TEMP\UCXJWX6_LOG_##.TXT

 

20170926/183535.913 -           logcount=10

 

20170926/183535.913 -           helplib=UC.MSL

 

20170926/183535.913 -           helpcache=ALL

 

20170926/183535.913 -           licence_class=V

 

20170926/183535.913 -          

 

20170926/183535.913 -           ; userid_type=INCL

 

20170926/183535.913 -           ; use Windows Job Objects 0=no/1=yes

 

20170926/183535.913 -           useJobObject=0

 

20170926/183535.913 -           ECPEXE=powershell.exe -NonInteractive -ExecutionPolicy bypass -NoLogo -file

 

20170926/183535.913 -           ECPEXT=ps1

 

20170926/183535.913 -           ft_temp_file=yes

 

20170926/183535.913 -           HomeDirCache=10

 

20170926/183535.913 -          

 

20170926/183535.913 -           ; FileProcessingMaxDepth = 0

 

20170926/183535.913 -           ; FileProcessingTimeout  = 0

 

20170926/183535.913 -          

 

20170926/183535.913 -           ; ft_omit_user_home = no

 

20170926/183535.913 -           ; ft_suppress_report= no

 

20170926/183535.913 -          

 

20170926/183535.913 -           [USERID]

 

20170926/183535.913 -           ; user/domain=START

 

20170926/183535.913 -          

 

20170926/183535.913 -           [STATUS_CHECK]

 

20170926/183535.913 -           time=1

 

20170926/183535.913 -          

 

20170926/183535.913 -           [VARIABLES]

 

20170926/183535.913 -           UC_HOST_CODE=UC_CODE

 

20170926/183535.913 -           UC_HOST_JCL_VAR=WINDOWS

 

20170926/183535.913 -           UC_EX_PATH_BIN=.

 

20170926/183535.913 -           UC_EX_PATH_TEMP=..\TEMP\

 

20170926/183535.913 -           UC_EX_PATH_JOBREPORT=..\TEMP\

 

20170926/183535.913 -           UC_EX_JOB_MD=.\UCXJWX6M.EXE

 

20170926/183535.913 -           UC_EX_PATH_BACKUP=..\BACKUP

 

20170926/183535.913 -          

 

20170926/183535.913 -           [TRACE]

 

20170926/183535.913 -           file=..\TEMP\UCXJWX6_TRACE_##.TXT

 

20170926/183535.913 -           trccount=10

 

20170926/183535.913 -           tcp/ip=0

 

20170926/183535.913 -           mail=0

 

20170926/183535.913 -           filetransfer=0

 

20170926/183535.913 -           event=0

 

20170926/183535.913 -           compress=0

 

20170926/183535.913 -           memory=0

 

20170926/183535.913 -           jcl=0

 

20170926/183535.913 -          

 

20170926/183535.913 -           [TCP/IP]

 

20170926/183535.913 -           port=2300

 

20170926/183535.913 -           bindaddr=

 

20170926/183535.913 -           bindlocal=0

 

20170926/183535.913 -           ; try all n seconds to connect to server

 

20170926/183535.913 -           connect=60

 

20170926/183535.913 -           report=60

 

20170926/183535.913 -           SendBufferSize=1048576

 

20170926/183535.913 -           RecvBufferSize=1048576

 

20170926/183535.913 -           cp=SERVERB:2220

 

20170926/183535.913 -          

 

20170926/183535.913 -           TcpKeepAliveTime=yes

 

20170926/183535.913 -           tcp_nodelay=yes

 

20170926/183535.913 -          

 

 20170926/183535.913 -           [HOSTS]

 

20170926/183535.913 -           ; Overwrite IP destination in case of IP NATing problems

 

20170926/183535.913 -           ;  (e.g. firewalls)

 

20170926/183535.913 -           ; <UC4-name>=<dns-name> or

 

20170926/183535.913 -           ; <UC4-name>=<ip-addr>

 

20170926/183535.913 -          

 

20170926/183535.913 -           [CP_LIST]

 

20170926/183535.913 -           2221=SERVERB

 

20170926/183535.913 -          

 

20170926/183535.913 -           [AUTHORIZATION]

 

20170926/183535.913 -           KeyStore=

 

20170926/183535.913 -           InitialPackage=

 

20170926/183535.913 -           -------------------------------------------------------------------------------------------------------------------------------

 

20170926/183535.913 - U02000071 Current directory: D:\Automic\Agents\Windows\bin

 

20170926/183535.913 - U02000066 Host information: Host name='SERVERB.idacorp.local', IP address='ip.1.2.3'

 

20170926/183535.928 - U02000072 Connection to system 'UC4_TEST' initiated.

 

20170926/183535.928 - U02000011 Connection to Server ' SERVERB:2220' initiated.

 

20170926/183536.944 - U02000012 Connection to Server ' SERVERB:2220' denied.

 

20170926/183536.944 -                   Connect-Error: 10061 (No connection could be made because the target machine actively refused it.)

 

20170926/183536.944 - U02000011 Connection to Server ' SERVERB:2221' initiated.

 

20170926/183537.960 - U02000012 Connection to Server ‘SERVERB:2221' denied.

 

20170926/183537.960 -                   Connect-Error: 10061 (No connection could be made because the target machine actively refused it.)

 

20170926/183537.960 - U02000074 Connecting to system 'UC4_TEST' is not possible.

 

20170926/183550.616 - U02000041 Shutdown Agent 'WIN_ SERVERB'.

 

20170926/183550.616 - U02000001 Agent 'WIN_ SERVERB' version '12.0.2+build.720' ended normally.

 

20170926/183550.616 -           Agent is disconnected, message 'EXSTOP  ' will be discarded.

 

 

 

thx for the infos!

 

cp=SERVERB:2220 (points to itself & to local AE CP port)

=> This is the class S Agent (maintenance Purposes) which may only run on the Server where AE is running so - so of course its the same name/IP/localhost and the CP Port.

cp=SERVERB:2220
=> you could try using "127.0.0.1:2220" instead (just for fun....)
=> cp=SERVERA:2217 or cp=SERVERA:2218 (for testing purpooses)

here is an example of my win agent (my Test system is running locally (AE + Agents)
name=V112_WIN01
system=V112
licence_class=V
port=11230 (local Agent Port range)
cp=127.0.0.1:11217 (one of 2 CP Ports, 11217, 11218)

UAC is off
windows Firewall on

hope this helps a bit.

Wolfgang,

Thanks for info.

Is your first item really “serverA” ?  You indicate license=S, so I think so.

Do you have an installed environment with 2 AE servers?  If so you how do you have the secondary defined how?  As referenced above to the ‘localhost’?

I will do some testing with the IP and 127.0.0.1.  I am not sure how but when pointed to ‘itself’ like it should be for failover type reasoning it would have to be configured this way, like we have our working v9 configured.

No, my Test System is a single box.

Here is my complete INI File (until HOSTS-section)

[GLOBAL]
name=V112_WIN01
system=V112

logon=1
language=(E,D)
logging=..\TEMP\V112_WIN01_LOG_##.TXT
logcount=10
helplib=UC.MSL
helpcache=ALL
licence_class=V

; userid_type=INCL
; use Windows Job Objects 0=no/1=yes
useJobObject=0
ECPEXE=powershell.exe -NonInteractive -ExecutionPolicy bypass -NoLogo -file
ECPEXT=ps1
ft_temp_file=yes
HomeDirCache=10

; FileProcessingMaxDepth = 0
; FileProcessingTimeout  = 0

; ft_omit_user_home = no
; ft_suppress_report= no

[USERID]
; user/domain=START


[STATUS_CHECK]
time=1


[VARIABLES]
UC_HOST_CODE=UC_CODE
UC_HOST_JCL_VAR=WINDOWS
UC_EX_PATH_BIN=.
UC_EX_PATH_TEMP=..\TEMP\
UC_EX_PATH_JOBREPORT=..\TEMP\
UC_EX_JOB_MD=.\UCXJWX6M.EXE
UC_EX_PATH_BACKUP=..\BACKUP


[TRACE]
file=..\TEMP\V112_WIN01_TRACE_##.TXT
trccount=10
tcp/ip=0
mail=0
filetransfer=0
event=0
compress=0
memory=0
jcl=0


[TCP/IP]
port=11230
bindaddr=
bindlocal=0
; try all n seconds to connect to server
connect=60
report=60
SendBufferSize=1048576
RecvBufferSize=1048576
cp=127.0.0.1:11217

TcpKeepAliveTime=yes
tcp_nodelay=yes


[HOSTS]
; Overwrite IP destination in case of IP NATing problems
;  (e.g. firewalls)
; <UC4-name>=<dns-name> or
; <UC4-name>=<ip-addr>


I just took the Template INI file and modified the relevant values.

For now I would suggest using the V Class until the agent runs properly.

I have been out of office for a few days, thought I would check back in with what I have tested.

 

Automic v12, Automation Engine and local Windows agent configuration, this is where my testing has led me with both Application Engine local Windows agents working, including failover to either server.

 

 

 

Environment setup is as below:

 

• SQL      2016 R2 “Always On” database.
• Automation      Engine – installed on two Windows 2012 R2 servers for redundancy.
• AE ‘UCSrv.ini’       file is configured identically on both servers for WP and CP names (wp1,       wp2, wp3… & cp1, cp2, cp3…) with mirrored port numbers also.
• Only        1 AE server is active at a given time, therefore no reason to split WP &        CP configuration.
• 1       server is ‘primary’ with active WP, CP and local Windows agent, license       class either ‘S’ or ‘V’ works.
• 1       server is ‘backup’ with NO active WPs, CPs or any agents other than the local       server Windows agent. 
• Local        Windows agent is active and can run with ONLY license class ‘V’ (license        ‘S’ receives only denied connections).
• REQUIRED        is secondary Windows agent is started AFTER primary in license class ‘S’        AND after CP is started that is defined in ‘UCJWX6.ini’, under [TCP/IP]        section (‘servername:port’).
• From      testing appears that the Windows server that is to be PRIMARY must have ‘UCJWX6.ini’      file set to “licence_class=S” which ends up being like a ‘renew transfer      key’ in a sense.

 

 

 

 

 

 

 

 

 

Automic documentation and reference to license classes.

 

licence_class=

 

The license class that corresponds to the acquired license and the hardware and software that is used.

Allowed values: "1" to "9", "S", "V"
"1" to "9" = License class of the agent.
"S" = Agent that is used for the administration of tasks on the Server computer.
"V" = Virtual license class for the agent.

 

 

 

I am still trying to determine all the facts behind the class ‘S’ license and “administration of tasks”.

 

Not sure if anyone can shed any knowledge of this as to exact functions.