kumsa29

AA Log file Analysis : High Level Framework Design

Blog Post created by kumsa29 Employee on Apr 27, 2018

Please find below the high-level integration approach for AA log file analysis:

  1. User FluentD or “Apache Nifi”  tail file processor to produce Kafka messages.
  2. User Kafka Server as a centralized unit to provide standard messaging producer and consumer service for third-party modules
  3. Use Graylag server as Kafka consumer.
  4. Once log message reach to gray log server you can use regex extractor to extract log message data.
  5. Use Graylag functionality to do the followings
    1. Generate report on any search criteria
    2. Analyze log data
    3. Generate Graph and chart
    4. Real-time monitoring, etc.

 

High-Level-Architecture

 

NifiTemplate-KafkaProducer-for-Logfile

 

GraylogServer-KafkaConsumer-for-LogfileAnalysis

Outcomes