Please find below the high-level integration approach for AA log file analysis:
- User FluentD or “Apache Nifi” tail file processor to produce Kafka messages.
- User Kafka Server as a centralized unit to provide standard messaging producer and consumer service for third-party modules
- Use Graylag server as Kafka consumer.
- Once log message reach to gray log server you can use regex extractor to extract log message data.
- Use Graylag functionality to do the followings
- Generate report on any search criteria
- Analyze log data
- Generate Graph and chart
- Real-time monitoring, etc.