Walk through of setting up Apache httpd on windows for use as a reverse proxy/load balancer for a PIM 12.8 ENTM and load balancing ENTM.

Introduction

In this document we are using Apache 2.4.17 x64 from Apache Haus. Please note that CA does not officially endorse Apache Haus or this specific version of Apache httpd for windows over any other httpd distribution/version, it is just the one I am using for this document.

There will be three servers involved in this scenario:

1. <LBSERVER> is the load balancing server. This is the server where we are installing Apache httpd.

2. <ENTM> is the primary Enterprise Management server

3. <LBENTM> is the load balancing Enterprise Management server

The assumption has been made that both <ENTM> and <LBENTM> are installed and working and serving SSL from port 18443. It is very important that this is the case. There is no point in implementing a reverse proxy to servers that do not work themselves, it just adds an additional layer to debug.

The aim is to have Apache httpd serving SSL on only port 8443 on <LBSERVER> acting as a reverse proxy to <ENTM> and <LBENTM>. No other ports will be served by Apache httpd.

I have also added a rewrite to that users who go to https://<LBSERVER>:8443/ will automatically be redirected to https://<LBSERVER>:8443/iam/ac.

***** YOU MUST MAKE SURE THAT NOTHING IS RUNNING ON PORT 8443 on <LBSERVER> BEFORE CONTINUING *****

Use the following command to determine if anything is running on port 8443:

netstat -an | find ":8443 "

If it returns nothing then nothing is running on port 8443 and you are set to go.

Section 1 - Download and Install Apache 2.4.17 x64:

1. On <LBSERVER> go to the following url:

http://www.apachehaus.com/cgi-bin/download.plx

Download:

A) Apache 2.4.x VC11 -> Apache 2.4.17 x64

B) Microsoft Visual C++ 2012 Redistributable

I will refer to the folder you have downloaded these files to as <download> from here on.

2. Unzip httpd-2.4.17-x64-vc11.zip to <download>\httpd-2.4.17-x64-vc11.

3. Open <download>\httpd-2.4.17-x64-vc11\readme_first.html and review the installation instructions. I have included installation instructions below, but they may need to be adapted for your system or due to and changes made by Apache Haus if you are using a different version of Apache httpd 2.4.

4. Copy the <download>\httpd-2.4.17-x64-vc11\Apache24 folder to c:\, so you have a c:\Apache24 folder.

Section 2 - Generate the SSL keys and certificate

1. On <LBSERVER> open a command prompt and navigate to c:\Apache24\bin:

cd c:\Apache24\bin

2. On the command prompt, run the following command:

openssl req -config c:\Apache24\conf\openssl.cnf -new -out c:\Apache24\conf\ssl\my-server.csr

Follow the prompts as requested (***** remember any pass phrases etc that you use! *****) When finished this will have created two files:

c:\Apache24\conf\ssl\my-server.csr c:\Apache24\bin\privkey.pem

3. On the command prompt, run the following command:

openssl rsa -in c:\Apache24\bin\privkey.pem -out c:\Apache24\conf\ssl\my-server.key

Follow the prompts as requested. This will create the following file:

c:\Apache24\conf\ssl\my-server.key

4. On the command prompt, run the following command:

Openssl x509 -req -in c:\Apache24\conf\ssl\my-server.csr -out c:\Apache24\conf\ssl\my-server.cert -signkey c:\Apache24\conf\ssl\my-server.key -days 365

This will create the following file:

c:\Apache24\conf\ssl\my-server.cert

5. The following files should now exist:

c:\Apache24\bin\privkey.pem
c:\Apache24\conf\ssl\my-server.cert
c:\Apache24\conf\ssl\my-server.key
c:\Apache24\conf\ssl\my-server.csr

Section 3 - Configure Apache httpd

1. on <LBSERVER> edit C:\Apache24\conf\httpd.conf.

To comment out a line in httpd.conf place a # symbol at the beginning of the line.

To uncomment a line in httpd.conf remove the # symbol at the beginning of the line.

Comment out:

Listen 80

Find the following lines and uncomment them. These lines are not contiguous so will need to be found and uncommented one by one:

#LoadModule headers_module modules/mod_headers.so 
#LoadModule lbmethod_byrequests_module modules/mod_lbmethod_byrequests.so 
#LoadModule proxy_module modules/mod_proxy.so 
#LoadModule proxy_balancer_module modules/mod_proxy_balancer.so 
#LoadModule proxy_http_module modules/mod_proxy_http.so 
#LoadModule rewrite_module modules/mod_rewrite.so 
#LoadModule slotmem_shm_module modules/mod_slotmem_shm.so

Find:

ServerName localhost:80 

And change it to something appropriate e.g. replacing <LBSERVER> with the FQDN of your this server:

ServerName <LBSERVER>:8443

Add at the end of the file add the following lines, replacing <ENTM> and <LBENTM> with the hostname or FQDNs of the ENTM and load balancing ENTM respectively, and <LBSERVER> with the FQDN of this server:

Listen 8443 https 
Header add Set-Cookie "ROUTEID=.%{BALANCER_WORKER_ROUTE}e; path=/" env=BALANCER_ROUTE_CHANGED
 
<Proxy balancer://ac-cluster>
   BalancerMember https://<ENTM>:18443 route=1
   BalancerMember https://<LBENTM>:18443 route=2
   ProxySet stickysession=ROUTEID
</Proxy>

<VirtualHost <LBSERVER>:8443>
   ServerName <LBSERVER>
   SSLEngine on
   SSLProxyEngine on
   SSLCertificateFile "C:\Apache24\conf\ssl\my-server.cert"
   SSLCertificateKeyFile "C:\Apache24\conf\ssl\my-server.key"
   SSLProxyVerify none
   SSLProxyCheckPeerCN off
   SSLProxyCheckPeerName off
   SSLProxyCheckPeerExpire off

   ProxyPass /app balancer://ac-cluster/app
   ProxyPassReverse /app https://<ENTM>:18443/app
   ProxyPassReverse /app https://<LBENTM>:18443/app
  
   ProxyPass /iam balancer://ac-cluster/iam
   ProxyPassReverse /iam https://<ENTM>:18443/iam
   ProxyPassReverse /iam https://<LBENTM>:18443/iam

   ProxyPass /idmmanage balancer://ac-cluster/idmmanage
   ProxyPassReverse /idmmanage https://<ENTM>:18443/idmmanage
   ProxyPassReverse /idmmanage https://<LBENTM>:18443/idmmanage

   ProxyPass /castylesr5.1.1 balancer://ac-cluster/castylesr5.1.1
   ProxyPassReverse /castylesr5.1.1 https://<ENTM>:18443/castylesr5.1.1
   ProxyPassReverse /castylesr5.1.1 https://<LBENTM>:18443/castylesr5.1.1

   LogFormat "%h \"%{BALANCER_WORKER_NAME}e\" %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\"" custom
   CustomLog logs/proxy-access.log custom

   RewriteEngine  on
   RewriteRule    "^/$"  "/iam/ac/"  [R]
</VirtualHost>

2. Edit c:\Apache24\conf\extra\httpd-ahssl.conf:

Comment out:

Listen 443 https

Comment out all of the following lines:

<VirtualHost _default_:443>
  SSLEngine on
  ServerName localhost:443
  SSLCertificateFile "${SRVROOT}/conf/ssl/server.crt"
  SSLCertificateKeyFile "${SRVROOT}/conf/ssl/server.key"
  DocumentRoot "${SRVROOT}/htdocs"
# DocumentRoot access handled globally in httpd.conf
     CustomLog "${SRVROOT}/logs/ssl_request.log" \
          "%t %h %{SSL_PROTOCOL}x %{SSL_CIPHER}x \"%r\" %b"
     <Directory "${SRVROOT}/htdocs">
          Options Indexes Includes FollowSymLinks
          AllowOverride AuthConfig Limit FileInfo
    Require all granted
     </Directory>
</virtualhost>
<VirtualHost *:443>
  SSLEngine on
  ServerName serverone.tld:443
  SSLCertificateFile "${SRVROOT}/conf/ssl/serverone.crt"
  SSLCertificateKeyFile "${SRVROOT}/conf/ssl/serverone.key"
  DocumentRoot "${SRVROOT}/htdocs"
     CustomLog "${SRVROOT}/logs/ssl_request.log" \
          "%t %h %{SSL_PROTOCOL}x %{SSL_CIPHER}x \"%r\" %b"
     <Directory "${SRVROOT}/htdocs">
          Options Indexes Includes FollowSymLinks
          AllowOverride AuthConfig Limit FileInfo
    Require all granted
     </Directory>
</virtualhost>

<VirtualHost *:443>
  SSLEngine on
  ServerName servertwo.tld:443
  SSLCertificateFile "${SRVROOT}/conf/ssl/servertwo.crt"
  SSLCertificateKeyFile "${SRVROOT}/conf/ssl/servertwo.key"
  DocumentRoot "${SRVROOT}/htdocs"
     CustomLog "${SRVROOT}/logs/ssl_request.log" \
          "%t %h %{SSL_PROTOCOL}x %{SSL_CIPHER}x \"%r\" %b"
     <Directory "${SRVROOT}/htdocs">
          Options Indexes Includes FollowSymLinks
          AllowOverride AuthConfig Limit FileInfo
    Require all granted
     </Directory>
</virtualhost>

Section 4 - Starting And Testing Apache httpd, And Making It A Service

It is possible and easy to setup Apache httpd to run as a windows service, but we will test it first by running in a command prompt to see if there are any errors. Note that debugging errors for Apache httpd is beyond the scope of this document - any errors I came across I fixed in the configuration above - and CA Technical Support. However, Google is your friend. Apache httpd is the most widely used web server on the internet so if you encounter a problem, someone probably already has and has a solution, at least that was my experience when writing this document.

1. On <LBSERVER> open a command prompt and navigate to c:\Apache24\bin:

cd c:\Apache24\bin

2. Start httpd:

httpd

After entering the command wait a few seconds. httpd should stay running and not return to a command prompt. If it does and/or any errors are displayed they will need to be investigated and resolved before continuing. Looking in c:\Apache24\logs\error.log can also be helpful. To stop it just press CTRL + c in the command prompt. After a second or two it will stop.

3. Make sure httpd is running as per step 2, and log in and test by pointing a browser to (where <LBSERVER> is the hostname or FDQN of the server we have installed Apache httpd to):

https://<LBSERVER>:8443/iam/ac

You may be prompted with some SSL warnings due to using a self signed certificate as per Section 2 - Generate the SSL keys and certificate. Once past these you should be presented with the ControlMinder/PIM login. Test a few things.

4. Assuming everything appears to be working, run a few different browser sessions (that is different session, not just different tabs or windows of the same browser session) on different client machines.

Open c:\Apache24\logs\proxy-access.log and you should see entries like the below:

123.123.123.123 "https://<ENTM>:18443" - [26/Nov/2015:09:35:01 +1100] "POST /iam/ac/ca12/index.jsp?facesViewId=/app/page/screen/standard_search.jsp HTTP/1.1" 200 166071 "https://<LBSERVER>:8443/iam/ac/ca12/index.jsp?facesViewId=/app/page/screen/standard_search.jsp" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:42.0) Gecko/20100101 Firefox/42.0"

123.123.123.123 is the IP Address of the client where the browser is running.

https://<ENTM>:18443 will be either the hostname/FQDN of the ENTM or LBENTM. There should be a good mix of <ENTM> and <LBENTM> to show that the load balancing is working - if you have used different browser sessions, opening new tabs and/or windows of existing browser sessions does not work to test this.

https://<LBSERVER>:8443 is the hostname/FQDN of the server we installed Apache httpd on.

5. Once you are satisfied that Apache httpd is running as a reverse proxy correctly, we can set it up as a service. Open a command prompt and navigate to c:\Apache24\bin

cd c:\Apache24\bin

6. Execute the following command:

httpd -k install

This will return something like:

Installing the 'Apache2.4' service
The 'Apache2.4' service is successfully installed.
Testing httpd.conf....
Errors reported here must be corrected before the service can be started.

There should be no errors as we have already fixed them before progressing from step 2.

7. In Windows Services, there should now be an Apache2.4 service. This is stopped and started like any other service. By default this is set to start automatically when the server is started, you may or may not want to change this.

You have now finished installing the reverse proxy/load balancer.