SungHoon_Kim

Agent to Policy Server connectivity

Blog Post created by SungHoon_Kim Employee on Apr 15, 2017

This is to understand how the Web Agent connects to the Policy Server.

 

Following test environment is setup.

 

SMPS1

hostname: smps1.shared.lab

Version: R12.6 SP2

 

SMPS2

hostname: smps2.shared.lab

Version: R12.6 SP2

 

WA1

Web Server: IIS

Version: R12.52SP1CR6

SmHost.conf: policyserver=smps2.shared.lab

HCO: hco.smps1 (Only has smps1.shared.lab)

 

Use Case: Normal Web Server startup

Wireshark was run on WA1 machine.

Start up IIS and open a browser to access wa1.shared.lab

Once the IIS page fully loaded, stop capturing wireshark.

 

 

I will explain about the Agent Connectivity in 3 parts.

 

Part1: Bootstrap

This is where the Web Agent reads SmHost.conf and until it downloads HCO.

 

Part2: HCO

This is where Web Agent contacts Policy Servers listed in HCO and download ACO.

 

 

 

 

When you look at the packet trace, it has lots of entries which are not relevant.

Also, the time does not look meaningful(at least for this use case).

The IP Address also maybe better if it displayed hostnames so it is much easier to focus on the traffic of interest.

 

You can press CTRL+ALT+1 to change the time display format.

Or, you can goto "View", "Time Display Format" and "Date and Time of Day".

You can see the time now displays the actual date and time.

 

Now moving on to the IP Addresses.

You can see there is "View", "Name Resolution" and by default "Resolve Physical Address" is selected.

 

Unselect the "Resolve Physical Address" and select "Resolve Network Address" only.

 

 

Now you can see the IP addresses are mostly switched to hostnames.

 

If you are not getting the same result, then please check your DNS entries and ensure the Reverse DNS is setup correctly and these machines has their entries.

 

 

 

Now, back at the packet trace, we need to filter the entries that we would like to see.

Following filter helps.

 

ip.src_host == wa1.shared.lab && ip.dst_host == smps1.shared.lab || ip.dst_host == smps2.shared.lab

 

You can use this filter only when the IP is resolved to FQHN.

!!! You must ensure you enter the correct hostname in the filter as they may at times appear in upper case or lower case.

 

This will filter the communication where the client(source) hostname is "wa1.shared.lab" and the server(destination) hostname is either "smps1.shared.lab" or "smps2.shared.lab"

 

You can see here that wa1.shared.lab opens its port 50134 and connects to smps2.shared.lab:44443.

WA1 only knows about smps2.shared.lab from SmHost.conf file.

 

So, when the Web Server initializes, it would load its registered modules and Web Agent is one of them.

 

WebAgent Module will look for WebAgent.conf file.

In case of IIS, it would be in the registry. (w3wp.exe would be doing all these)

In case of other web servers like Apache, it would appear in the web server config file.

For example, in case of Apache, it would have following entry in the httpd.conf file.

 

Once the WebAgent.conf file is read, it would provide information about SmHost.conf file location and the Agent Config Object name.

 

 

Web Agent will load SmHost.conf file to contact the Policy Server.

Web Agent would identify the Policy Server to connect to, to download the Host Config Object.

In this case, it would contact "SMPS2.SHARED.LAB" and port 44443.

It would submit Trusted Host Name "trust_wa1" and also the Shared Secret so Policy Server can identify this Web Agent as trusted.

Web Agent would request for Host Config Object named "hco.smps1".

Once this HCO is downloaded, Web Agent will terminate the connection.

 

!!! You will observe Web Agent promptly resetting connection for bootstrap only.

 

Let's look at the packet trace.

From the filtered list, right click on the first entry where wa1 is contacting smps2.

Then select "Follow" and "TCP Stream"

 

You can close the popup window.

One thing you can see from there in clear-text is the Trusted Host Name "trust_wa1" which was found in the SmHost.conf file.

 

You can see the filter has changed to "tcp.stream eq 11".

The first 3 entries are the TCP handshake.

WA1 opened its tcp port 50134 and established connection to SMPS2 at port 44443.

 

You can then see the WA1 is sending Trusted Host Name to SMPS2.

You will observe this each time Web Agent creates a new connection to Policy Server.

 

We won't be able to decode what is being sent and received but this is where the WA1 is fetching the HCO "hco.smps1".

 

 

This is end of Boot Strap stage.

==================================================

 

Now, go back to the original filter we used.

ip.src_host == wa1.shared.lab && ip.dst_host == smps1.shared.lab || ip.dst_host == smps2.shared.lab

 

You would notice from above that, if you look from after the WA1 have reset the tcp port 50134 to SMPS2 at 44443, Web Agent is now contacting SMPS1.SHARED.LAB.

 

You would also notice that it is opening 2 TCP ports.

 

This is based on the MinSocketsPerPort configuration in the HCO which is set to 2.

This is for the initial connections to the policy servers registered in the HCO.

 

Subsequently, whenever Web Agent creates new additional connections to the Policy Server, it will create x number of connections defined in the NewSocketStep.

 

And Web Agent can create additional connections up to MaxSocketsPerPort.

HCO Sockets Parameters
[1972/2856][Tue Apr 18 2017 23:23:35] hostname='trust_wa1'.
[1972/2856][Tue Apr 18 2017 23:23:35] maxsocketsperport='20'.
[1972/2856][Tue Apr 18 2017 23:23:35] minsocketsperport='2'.
[1972/2856][Tue Apr 18 2017 23:23:35] newsocketstep='2'.
[1972/2856][Tue Apr 18 2017 23:23:35] requesttimeout='60'.

 

As this is a new connection, it will also follow the normal procedure of reading SmHost.conf, sending TrustedHost and SharedSecret.

 

These 2 connections will be used for sending normal HTTP requests Web Agent has intercepted for processing.

This is end of 2nd stage where Web Agent contacts HCO listed Policy Server to establish connection and it is also where it downloads ACO.

 

========================================================

 

But this is not the end.

You will see from wireshark that the above BootStrap connection and HCO connection repeats one more time.

 

It is because the first round (Bootstrap and HCO connection) was performed by the w3wp.exe process.

The second round (Bootstrap and HCO connection) is then performed again as LLAWP.exe gets spawned by the w3wp.exe process.

 

LLAWP.exe process will go through the same exact steps mentioned above.

 

I am attaching sample3.zip file which contains all the relevant data demonstrating the Agent to Policy Server connection.

 

There is also nother tool that would make it very clear who(w3wp.exe or LLAWP.exe) is connecting to the Policy Server.

It is Process Monitor. If your Web Agent is on Windows, I would highly recommend this tool.

It is a very powerful tool for debugging what the process does.

 

As there will be too much data, my advise is to enable only "Network Activity".

Unselect the "Registry Activity", "File System Activity" and "Process and Thread" activity.

Then add following filters.

 

Then let's look at the same sequence from Process Monitor log.

Step1. w3wp.exe connects to smps2.shared.lab:44443 using tcp port 49268 and disconnect.

You can tell that this is from reading SmHost.conf

Step2: w3wp.exe connects to smps1.shared.lab:44443 by creating 2 connections, port 49269 and 49270.

You can tell this is after reading HCO.

You would also notice there is no "TCP Disconnect" for these connections.

And if you look closely, after establishing 2 connections, w3wp.exe uses the first connection(port 49269) to receive(TCP Receive) something a few times. This is because it is downloading ACO.

Then the same thing happens again but this time it is LLAWP.exe process.

Step3: LLAWP.exe connects to smps2.shared.lab:44443 using tcp port 49271 and disconnect.

Step4: LLAWP.exe connects to smps1.shared.lab:44443 by creating 2 connections, port 49269 and 49270.

And again it is using the first port 49272 to download the ACO.

At this point, the LLAWP would have started logging WA.log

 

=====================================================

 

Then, which process(High Level Agent or Low Level Agent) would contact the Policy Server to send IsProtected/IsAuthenticated/IsAuthorized calls?

 

It is the HLA(High Level Agent, w3wp.exe).

 

To prove this, I have disabled Agent Cache so every refresh of browser would send IsProtected calls to Policy Server.

 

You can see numerous conversation between w3wp.exe process to smps1.shared.lab:44443

 

And you will find LLAWP.exe occasionally use its HCO connection to contact Policy Server as well.

It is for management calls.

You will find that LLAWP.exe would contact HCO listed Policy Server every 30 seconds.

 

This is a simplified use case for demonstration purpose.

If you are capturing the same packet trace from a linux machine for Apache Web Agent, you would find all the child httpd process contacting Policy Server. But it would still be same as what is described here for w3wp.exe process.

And you would also find 1 LLAWP process establishing its own connections going through the same procedure.

 

Hope this answers many questions you might have had relating to this topic.

 

Following article is written by Yu Nishzaki.

The sequence of communication between WebAgent and Policy Server. 

 

Next, I will share what happens when there are failovers and cluster is used.

Attachments

Outcomes