架設Redis作為蒐集log的緩衝

 

Logstash 建議用Redis作為蒐集log資料的中介及緩衝，主要有以下原因

• 安裝簡單而迅速
• 效率佳
• 經過多數Logstash社群驗證過

 

開始安裝redis

yum install redis

修改設定檔 /etc/redis.conf，找到以下設定值

bind 127.0.0.1

加上註解符號#，讓所有IP來源都可以連接Redis (當然安全性不佳，後續再來解決)

#bind 127.0.0.1

啟動Redis

service redis start

測試Redis

#redis-cli -h localhost
redis localhost:6379> ping
PONG
#
這樣就表示服務有正常啟動。

新增從Redis將資料送至Elasticsearch的設定值

• 在/etc/logstash/conf.d/目錄下新增 central.conf
  input {
          redis {
                  host => "localhost"
                  type => "redis-input"
                  data_type => "list"
                  key => "logstash"
          }
  }
  output {
          elasticsearch {
                  host => "localhost"
                  cluster => "LogCluster"
                  node_name => "LogMaster"
          }
  }
  
  




• 修正先前apache access log的設定檔，將最後輸出的部分改為redis
  input {
    file {
      path => "/var/log/httpd/access_log"
      type => "apache"  # a type to identify those logs (will need this later)
    }
  }
  
  filter {
    if [type] == "apache" {   # this is where we use the type from the input section
      grok {
        match => [ "message", "%{COMBINEDAPACHELOG}" ]
      }
    }
  }
  
  output {
          redis {
                  host => "localhost"
                  data_type => "list"
                  key => "logstash"
          }
  }
  
  




• 重新啟動logstash服務
  service logstash restart
  

這樣就可以使用了喔…