Creating an ALL-IN-ONE VM Image - Part 6

Blog Post created by SungHoon_Kim Employee on Nov 5, 2015

This is something I like to do once in a while. It takes long time to setup everything but to me it is a hobby. It is like putting zigsaw puzzles.




Following components will be installed.


01. Install OS (Windows 2008 R2 - English)

02. Microsoft Loopback Adapter

03. Active Directory

04. DNS

05. IIS

06. Certificate Authority

07. MSSQL 2012

08. JDK 1.7.0_80 (32bit and 64bit)

09. NewAtlanta ServletExec 6.0

10. ASF Apache

11. CA Directory

12. Oracle Directory Server 11g

13. CA Single Sign-On Policy Server

14. CA Single Sign-On AdminUI

15. CA Single Sign-On Web Agent/Option Pack

16. CA Single Sign-On Secure Proxy Server

17. CABI 3.3


Some trivial steps are skipped such as installing the OS and promoting to a Domain Controller.



13. CA Single Sign-On Policy Server


Downloaded ""

In order to enable most available features of Policy Server, I will be installing some additional Windows services.


Load "Server Manager" and goto "Features".


Click on "Add Features" and select "SNMP Services" and install.


This will install "SNMP Service".

Double click on the "SNMP Service" and configure the "Traps" and "Security" tab.



  • Install Policy Server

Extract the zip file and run the "ca-ps-12.52-sp01-cr02-win32.exe"


Policy Store cannot be configured right now because I will be using CA Directory as Policy Store.


Advanced Auth Service is being configured but this is not a valid configuration because there is no policy store.

So, this need to be configured again later.


Don't restart the machine yet.


Create a Policy Server user account.

Under "Managed Service Accounts" (actually it doesn't really matter where), create a "User".

Load the System Properties and check the Environment Variables for "System".

You will find "ARCOT_HOME=C:\Program Files (x86)\CA\aas".

This is from the RiskMinder component introduced to Policy Server.

Later when SPS is installed, it also needs to set this same ARCOT_HOME but with different value.

So, in order to prevent this conflict, I am moving this to "policyserver" user's local user environment.


But first, you need to remove this "ARCOT_HOME" from here.

And in order to logon to this Domain Controller as "policyserver" user, I need to add this user to "Domain Admins" group.

Now, Switch User (Start Menu ==> Log off ==> Switch user) to policyserver user.


Now, set the ARCOT_HOME environment variable as user variable as above.

Next is to make the Policy Server run as "policyserver" service account.


You will get a notification that policyserver account is granted service rights.


You should see on the service entry that "Log On As" is set to "policyserver".


You MUST do the same for the "CA RiskMinder Service" as well.

These 2 services must be started as "policyserver" user.

After this, if the RiskMinder service starts up, you should be able to confirm it is functional by looking at its cariskminderstartup.log located in "C:\Program Files (x86)\CA\aas\logs" folder.


Now, logout policy server user and logon as "Administrator".

Because the Policy Server's environment variables are set at System Level, and only ARCOT_HOME was transferred to user variable, Administrator can run all the Policy Server commands.


  • smreg

It is officially mentioned in the documentation not to leave a copy of smreg.exe and XPSSecurity.exe on the policy server machine but as this is a sandbox I am placing it in the "<policyserver>/bin" folder.



  • Preparing CA Directory as Policy Store

Policy Server cannot automatically configure CA Directory as Policy Store.

It has to be performed manually.



Copy the following policy store schema files over to "C:\Program Files\CA\Directory\dxserver\config\schema\" folder.

     C:\Program Files (x86)\CA\siteminder\eTrust\netegrity.dxc

     C:\Program Files (x86)\CA\siteminder\xps\db\etrust.dxc


Copy "C:\Program Files\CA\Directory\dxserver\config\schema\default.dxg" to "SiteMinder.dxg"

Modify "SiteMinder.dxg" file to add the policy store schema files as below and save.


source "netegrity.dxc";

source "etrust.dxc";


Copy "C:\Program Files\CA\Directory\dxserver\config\limits\default.dxc" to "SiteMinder.dxc"

Remove the "Read-Only" attribute.

Update the tuning parameters as below.


# size limits

#set max-users = 255;

set max-op-size = 20000; #This determines max entries to be returned at a time. This value has to be high enough if you have a large policy store.

#set multi-write-queue = 20000;

# size limits

set max-users = 1000;

set credits = 5;

set max-local-ops = 1000;

# set max-op-size = 4000;

set multi-write-queue = 20000;



Update "C:\Program Files (x86)\CA\Directory\dxserver\config\servers\PRIMARY.dxi" for schema dxc

Update references


# schema

# source "../schema/default.dxg";

source "../schema/SiteMinder.dxg";

# service limits

# source "../limits/default.dxc";

source "../limits/SiteMinder.dxc";


Update Tuning parameter for Policy Store (CADir r12 SP1 or later).


set ignore-name-bindings = true;



Restart DSA instance


dxserver stop PRIMARY

dxserver start PRIMARY



Connect to Policy Store instance using JXPlorer


If everything went well, when you navigate to the "Schema" tab and look at the "objectclass" branch, you should see all the SiteMinder objectclasses.

Those with "sm*" are for the legacy and "xps*" are for the XPS Policy Store.


The work is not complete yet, you will need to manually create the OU structure.


At O=SM brance, right click and select "New" to create "OU=Netegrity".

You need to repeat the steps to create the following.





Now this PRIMARY instance is ready to be setup as policy store.


  • Configuring Policy Store

Load "smconsole" (aka, Policy Server Management Console) and enter the PRIMARY DSA instance details and click "Test LDAP Connection" to confirm it is connected successfully.



Also, configure the Key Store settings to point to the Policy Store. Click "Apply"



Open a command-line and change directory to "C:\Program Files (x86)\ca\siteminder\xps\dd"

Run the command "XPSDDInstall SmMaster.xdd"


You will find objects are created under the OU=XPS.


Now change directory to "C:\Program Files (x86)\ca\siteminder\db".

There are several "xml" files but all xml files are incorporated into the "smpolicy.xml".

So, you only need to import the "smpolicy.xml"


xpsimport smpolicy.xml -npass

You will find objects are created under OU=PolicySvr4.


Now, you need to create the "SiteMinder" super user.

Run "smreg -su <password>" to create and set the password.


Reboot the machine as it has not been done yet after the installation.


  • Reconfigure the RiskMinder component.

During the installation, Policy Store was not configured by the Wizard and the RiskMinder component was configured.

RiskMinder component cannot be configured properly without the policy store so this need to be run again.


Run the "Policy Server Configuration Wizard".


No need to choose any features to configure. Just click "Next".


You will be asked to enter the "Master Key".  This only accepts alphanumeric!!!

This is not Policy Store encryption Key. This is a key used by RiskMinder component.

You must keep a record of this key as you will need it in the future.


It asks again to set a password for the "SiteMinder" super user.

You cannot skip this part without entering a value so enter whatever password suits you. You will use that to administer the policy server.


Note: As the configuration wizard will set the "ARCOT_HOME" in the system variables, please remove it again.


Also, later when you have configured SiteMinder AdminUI, you will see "Default_<PolicyServerMachineName>_AAS" HostConfigObject would have been created as below.

AdminUI is not installed yet so you won't be able to see this but you can check from XPSExplorer.


If you have 2 Policy Server, then you are expected to see 2 HCO named "Default_<PolicyServerMachineName>_AAS"

If you don't have the matching number of HCO objects, then you will need to check the Policy Server hostnames to determine which one need to run the configuration wizard to register this.



  • OneView Monitor

At this point, you will be able to access http://www.sso.lab/sitemindermonitor/ to view the OneView Monitor.



  • SSL communication to LDAP backends

It is common to communicate to backend ldap servers using ssl communication.


Navigate to http:.//www.sso.lab/certsrv/ and download the "CA Certificate"


Create "C:\Program Files (x86)\ca\siteminder\certs" folder.

Rename the newcer.cer to ROOTCA.cer and copy it to "C:\Program Files (x86)\ca\siteminder\certs"


Open a command-line and change directory to "C:\Program Files (x86)\ca\siteminder\certs"

Run the following command to create cert8.db file.


"certutil -A -n "ALLINONE Root CA" -t "C,," -i ROOTCA.cer -d ."


This command will create cert8.db, key3.db and secmod.db files if it did not exist, and also add the certificate as trusted Root CA certificate.

If you already have the cert8.db, key3.db and secmod.db files, then it will only add.


Now, load the smconsole and enter the configuration at the "Data" tab.

Select "cert8.db" file and click "Apply"



Now, Policy Server should be able to connect to any LDAP server via ssl as long as its certificate was issued by this RootCA that was imported into cert8.db.


  • SNMP monitoring


Navigate to "C:\Program Files (x86)\CA\siteminder\bin" and run "SnmpWalkRun.bat

Then in the "OID" type "products" (or select an entry from dropdown menu) and click on "Walk".

If it is configured correctly, you should see the following. (Ignore the popup message about "End of MIB")

In this case, you can see that the port used was "161" which is going through the OS's master SNMP agent.

If you want to test the "Netegrity SNMP Agent" directly, change the port to "8001" and try.

Configuration is in "C:\Program Files (x86)\CA\siteminder\config\snmp.conf" file.

Update the file as below.


LOG_FILE=C:\Program Files (x86)\CA\siteminder\log\SNMP.log



And for snmp trap messages, modify "C:\Program Files (x86)\CA\siteminder\config\snmptrap.conf" file as below.



  • Adding Event Handler



Execute the following commands.


Follow these steps:

  1. Open a command line on the Policy Server, and enter the following command: xpsconfig

    The tool starts and displays the name of the log file for this session, and a menu of choices opens.

  2. Enter the following: xps

    A list of options appears.

  3. Enter the following: 5(AuditSMHandlers)

    The settings for the event handler libraries appear.

  4. Type C, and then enter the path and file name of the event handler library ("C:\Program Files (x86)\CA\siteminder\bin\eventsnmp.dll"). The settings for the event handler libraries appear. The value you added is shown at the bottom of the settings as a "pending value."

    5. Enter Q and [ENTER] three times to end your XPS session.


    6. Policy Server need to be restarted.


In the smps.log, you should find the following entry.


     [5032/5036][Thu Nov 05 2015 15:47:20][SmEventWedge.cpp:321][LateInit][INFO][sm-xpsxps-06860] Event handler library loaded: "C:\Program Files (x86)\CA\siteminder\bin\eventsnmp.dll"




This concludes Part 6 of ALL IN ONE Image.