Automic Workload Automation

  • 1.  How to install Event Engine manually on Windows?

    Posted Apr 18, 2019 02:04 PM

    Has anyone been able to manually install Event Engine manually on Windows? 

    I understand it can be installed on Windows if you use the OneInstaller, but I do not want to use that because I want to learn how it's done manually.

     

    I already have the Backend, IA agent, and Datastore installed.  The only missing pieces are the Rule Engine and Streaming Platform.

     

    I understand the documentation states, "The Rule Engine and Streaming Platform is not supported on Windows OS", but I got confirmation today that it works just fine on Windows.  



  • 2.  Re: How to install Event Engine manually on Windows?
    Best Answer

    Posted Apr 23, 2019 04:58 PM

    Hi Harriet,
    I just finished up manual installations of all the components (Postgres, IA, flink, kafka) on Windows; and everything does appear to be up and working.

     

    What I think the main challenge is that the ONE Installer appears to do a lot of backend configuration that the "out of the box" setup does not do.

     

    The main things I had to modify to get everything working correctly (and please note, this was my first time through so I may be missing things or made it harder than it needed to be):
    Postgres:
    The psql file complained about some of the syntax. I had to modify it directly changing (adding quotes):

    CREATE USER :username WITH PASSWORD :password SUPERUSER;

    to
    CREATE USER :username WITH PASSWORD ':password' SUPERUSER;

     

    That worked, but it still gave an error on the:
    SELECT trim(:password) as password_cleaned \gset

     

    That at that point I didn't run the test to see if it also needed the quotes or what it was unhappy about. All that affected was the final out of the three datasource lines for the applications.properties file.
    Flink & Kafka:

    So, I 'cheated' a little bit. I looked at the finished installs from a ONE Installer installation, and they included a few files NOT present in the default installation: launcher.exe, start.bat, stop.bat. Which, amazingly enough, corresponds exactly to our SMD:
    E:\Automic\Analytics\kafka\bin\windows\launcher.exe "start.bat" "stop.bat"
    E:\Automic\Analytics\flink\bin\launcher.exe "start.bat" "stop.bat"

     

    Now, trying to start them after modifying the paths in the SMD was not successful. Both of them needed modifications:

     

    Kafka:

    Start.bat (modified lines in bold)

    rem Starts both Zookeeper and Kafka
    @echo off
    setlocal
    set "JAVA_HOME=C:\Program Files\Java\jdk1.8.0_201\"
    set "PATH=%JAVA_HOME%\bin;%path%"
    rem replace the current (absolute) directory variable with a relative one to avoid running into the command length limit
    rem of Windows commands in kafka-run-class.bat during startup, otherwise the classpath of Zookeeper/Kafka might get too long.
    set "CD=..\.."

    @echo Starting up Zookeeper> streaming_start.out

    start /b %~dp0zookeeper-server-start.bat ..\..\config\zookeeper.properties ^1^> ..\zookeeper.out ^2^>^&^1

    :BeginPollZookeeper

    @echo Waiting for Zookeeper>> streaming_start.out

    rem wmic spills out exactly 5 lines if there is no process with the desired properties,
    rem this is exploited here to check if there already is a zookeeper java process available.
    SET NO_PROCESS_LINE_COUNT=5

    SET count=1
    FOR /F "tokens=*" %%F IN ('wmic process where "commandline like '%%C:\\Program Files\\Java\\jdk1.8.0[_]201/bin/java%%quorum.QuorumPeerMain%%' and not name='wmic.exe'" get Processid') DO ( SET /a count+=1
    )
    echo %count%

    IF "%count%" == "%NO_PROCESS_LINE_COUNT%" (
    @echo Waiting for Zookeeper>> streaming_start.out
    GOTO :BeginPollZookeeper
    )

    @echo Removing broker information from Zookeeper>> streaming_start.out
    cmd /c "%~dp0zookeeper-shell.bat localhost:2181 rmr /brokers"
    @echo Starting up Kafka>> streaming_start.out
    %~dp0kafka-server-start.bat ..\..\config\server.properties ^1^> ..\kafka.out ^2^>^&^1
    endlocal
    exit

     

    Stop.bat (modified lines in bold)

    rem Stops both Zookeeper and Kafka

    setlocal

    @echo Stopping Kafka> streaming_stop.out

    wmic process where (commandline like "%%C:\\Program Files\\Java\\jdk1.8.0[_]201/bin/java%%kafka.Kafka%%" and not name="wmic.exe") delete
    wmic process where (commandline like "%%kafka-server-start.bat%%" and not name="wmic.exe") delete

    @echo Stopping Zookeeper>> streaming_stop.out

    wmic process where (commandline like "%%C:\\Program Files\\Java\\jdk1.8.0[_]201/bin/java%%quorum.QuorumPeerMain%%" and not name="wmic.exe") delete
    wmic process where (commandline like "%%zookeeper-server-start.bat%%" and not name="wmic.exe") delete

    endlocal

     

    Flink:
    Start.bat (modified lines in bold)

    rem Starts Flink
    @echo off
    setlocal
    set "JAVA_HOME=C:\Program Files\Java\jdk1.8.0_201\"
    set "PATH=%JAVA_HOME%\bin;%path%"

    @echo Starting up Flink> flink_start.out

    %~dp0start-local.bat ^1^> ..\flink.out ^2^>^&^1

    endlocal
    exit
    Stop.bat (modified lines in bold)

    rem Stops Flink

    setlocal

    @echo Stopping Flink> flink_stop.out
    wmic process where (commandline like "%%java%%jobmanager.JobManager%%E:\\Automic\\Analytics\\flink\\bin%%" and not name="wmic.exe") delete

    endlocal

    At that point, everything else was configuration as per the documentation - everything appears to be up and running. But, it definitely wasn't next-next-finish.

    - Darren



  • 3.  Re: How to install Event Engine manually on Windows?

    Posted Apr 25, 2019 10:55 AM

    Hey Darren (snida02),

     

    I finally got around to reviewing your solution.  I have a few questions...

     

    Where did you get the launcher.exe since it's not included in the included in the default installation?  Did you just go to the ONE Installer installation and copied that file to your manual installation?  

     

    Also, if I am understanding your solution correctly, you just took the start.bat and stop.bat files from the ONE Installer installation, made the above modifications, and drop them in their corresponding bin folders?  Is that correct?

     

    Thanks,

    Harriet



  • 4.  Re: How to install Event Engine manually on Windows?

    Posted Apr 28, 2019 07:23 PM

    Hi Harriet,

    Yes, I copied the launcher and the start and stop.bat's from the ONE installed installation; I don't think they'd actually be needed, but that was a lot faster than working entirely within the third party products. I'm almost positive all the launcher.exe does is allow for the service manager to stop and start them, and the start and stop.bat files are just to make that "easier" natively on windows.