Hi Harriet,
I just finished up manual installations of all the components (Postgres, IA, flink, kafka) on Windows; and everything does appear to be up and working.
What I think the main challenge is that the ONE Installer appears to do a lot of backend configuration that the "out of the box" setup does not do.
The main things I had to modify to get everything working correctly (and please note, this was my first time through so I may be missing things or made it harder than it needed to be):
Postgres:
The psql file complained about some of the syntax. I had to modify it directly changing (adding quotes):
CREATE USER :username WITH PASSWORD :password SUPERUSER;
to
CREATE USER :username WITH PASSWORD ':password' SUPERUSER;
That worked, but it still gave an error on the:
SELECT trim(:password) as password_cleaned \gset
That at that point I didn't run the test to see if it also needed the quotes or what it was unhappy about. All that affected was the final out of the three datasource lines for the applications.properties file.
Flink & Kafka:
So, I 'cheated' a little bit. I looked at the finished installs from a ONE Installer installation, and they included a few files NOT present in the default installation: launcher.exe, start.bat, stop.bat. Which, amazingly enough, corresponds exactly to our SMD:
E:\Automic\Analytics\kafka\bin\windows\launcher.exe "start.bat" "stop.bat"
E:\Automic\Analytics\flink\bin\launcher.exe "start.bat" "stop.bat"
Now, trying to start them after modifying the paths in the SMD was not successful. Both of them needed modifications:
Kafka:
Start.bat (modified lines in bold)
rem Starts both Zookeeper and Kafka
@echo off
setlocal
set "JAVA_HOME=C:\Program Files\Java\jdk1.8.0_201\"
set "PATH=%JAVA_HOME%\bin;%path%"
rem replace the current (absolute) directory variable with a relative one to avoid running into the command length limit
rem of Windows commands in kafka-run-class.bat during startup, otherwise the classpath of Zookeeper/Kafka might get too long.
set "CD=..\.."
@echo Starting up Zookeeper> streaming_start.out
start /b %~dp0zookeeper-server-start.bat ..\..\config\zookeeper.properties ^1^> ..\zookeeper.out ^2^>^&^1
:BeginPollZookeeper
@echo Waiting for Zookeeper>> streaming_start.out
rem wmic spills out exactly 5 lines if there is no process with the desired properties,
rem this is exploited here to check if there already is a zookeeper java process available.
SET NO_PROCESS_LINE_COUNT=5
SET count=1
FOR /F "tokens=*" %%F IN ('wmic process where "commandline like '%%C:\\Program Files\\Java\\jdk1.8.0[_]201/bin/java%%quorum.QuorumPeerMain%%' and not name='wmic.exe'" get Processid') DO ( SET /a count+=1
)
echo %count%
IF "%count%" == "%NO_PROCESS_LINE_COUNT%" (
@echo Waiting for Zookeeper>> streaming_start.out
GOTO :BeginPollZookeeper
)
@echo Removing broker information from Zookeeper>> streaming_start.out
cmd /c "%~dp0zookeeper-shell.bat localhost:2181 rmr /brokers"
@echo Starting up Kafka>> streaming_start.out
%~dp0kafka-server-start.bat ..\..\config\server.properties ^1^> ..\kafka.out ^2^>^&^1
endlocal
exit
Stop.bat (modified lines in bold)
rem Stops both Zookeeper and Kafka
setlocal
@echo Stopping Kafka> streaming_stop.out
wmic process where (commandline like "%%C:\\Program Files\\Java\\jdk1.8.0[_]201/bin/java%%kafka.Kafka%%" and not name="wmic.exe") delete
wmic process where (commandline like "%%kafka-server-start.bat%%" and not name="wmic.exe") delete
@echo Stopping Zookeeper>> streaming_stop.out
wmic process where (commandline like "%%C:\\Program Files\\Java\\jdk1.8.0[_]201/bin/java%%quorum.QuorumPeerMain%%" and not name="wmic.exe") delete
wmic process where (commandline like "%%zookeeper-server-start.bat%%" and not name="wmic.exe") delete
endlocal
Flink:
Start.bat (modified lines in bold)
rem Starts Flink
@echo off
setlocal
set "JAVA_HOME=C:\Program Files\Java\jdk1.8.0_201\"
set "PATH=%JAVA_HOME%\bin;%path%"
@echo Starting up Flink> flink_start.out
%~dp0start-local.bat ^1^> ..\flink.out ^2^>^&^1
endlocal
exit
Stop.bat (modified lines in bold)
rem Stops Flink
setlocal
@echo Stopping Flink> flink_stop.out
wmic process where (commandline like "%%java%%jobmanager.JobManager%%E:\\Automic\\Analytics\\flink\\bin%%" and not name="wmic.exe") delete
endlocal
At that point, everything else was configuration as per the documentation - everything appears to be up and running. But, it definitely wasn't next-next-finish.
- Darren