Aamir,
You may want to get a support issue open. What version of the JDK is being used? I know right now there are some problems with JDK 1.8 updates 161 and 162, but for now those have been isolated to issues with the SDM custom operators. This is due to an update in Java for vulnerabilities for 1.9 update 4, 1.8 updates 161 and 162, and 1.7 update 171.
Also, when you started the installation, did you select Run As Administrator? Any UAC restrictions on the server? If UAC is set during install, we have seen problems. We advise to set UAC to never for the installation period.
Is the Java control panel set in the network section to use direct connection? If not, this should be changed.
If none of the above apply or resolve the install issue, please open a support issue so we can help you directly to successfully install Process Automation.