Automic Workload Automation

Hello,

Since we migrated to v12.1 from version 9.13 we have noticed one particular AIX agent seems to be shutting down with the below error.

20180810/064544.625 - U02000102 Error when calling the function 'bind(:2308)', error code '67(Address already in use)

This doesnt happen very frequently, may be thrice in five months,  but it is still bothering us as lot of jobs run on this agent an even if the agent is down for 15 minutes there is business impact.

Now we noticed that with version 12.1 there is a new Listener process that starts when we start the agent. Could this be causing the issue? Is there any way we can disable the listener process? We have noticed that agent usually shuts down after a FTP step.

Any help would be appreciated.

What's the name of the new process?

The new process is the listener process ucxja64-listener.

KumarRahul605332 wrote:

The new process is the listener process ucxja64-listener.

The process name is a bit different on Linux: ucxjlx6-listener. I see that this process is a child process of the main agent process. I did a little investigation into this process. (On this system, the agent port number is 22150.)

# lsof | grep 22150
ucxjlx6   33948     nobody    7u     IPv4          146567167      0t0        TCP *:22150 (LISTEN)

# fuser 22150/tcp
22150/tcp:           33948

# ss -netopt state listening | grep 22150
0      10                             *:22150                         *:*      users:(("ucxjlx6",33948,7)) uid:65534 ino:146567167 sk:ffff88080e1f0240

# ps -ef | grep 33948
nobody   33948 33943  0 13:59 ?        00:00:00 ucxjlx6-listener
root     36169 35930  0 14:05 pts/2    00:00:00 grep 33948

# ls -l /proc/33948/exe
lrwxrwxrwx 1 root root 0 Aug 16 14:00 /proc/33948/exe -> /opt/uc4/agent_v12.2/ucxjlx6

It is clear that this process is being spawned by the agent, and is listening on the port specified in the [TCP/IP] section of the agent’s INI file. I assume you’re using port 2308.

Does the problem happen only when the agent is first started up, or does it happen when the agent has already been running for a while?

Yes port 2308 is specified in the agent ini. This issue happens after the agent has been running for a while. And it is not very frequent but in the past three months it has happened three times.

If it were happening only at initial start-up, I would have recommended making sure no other process is listening to the port.

But since it's happening at random times when the agent has already been running for a while, I would say that it’s likely a bug.

I recommend that you:

• enable tracing, to capture more information the next time the problem happens; and
• open a new case with CA to report the problem.

Yeah I have opened a case with CA. They are saying it could be because of a network issue. We seen this issue with only one AIX agent. All our other AIX agents are still running on version 9 as version 12 is compatible with AIX 7 or above. In any case we will see what CA has to say on that. Thank you for your help.