Redhat linux 下 fluentd+fluentd 實現橋接


實驗環境為2台測試機分別為Server201(192.168.10.201)和Server202(192.168.19.202),分別安裝好fluentd,具體安裝步驟不再累述。

重點描述2台機器上的配置,目的是將Server202上的日志,通過fluentd橋接的方式,放置到Server201下的某個目錄。

1、Server202上fluentd.conf:

<source>
 type tail 

#需要轉儲的日志1
 path /home/aircom/mongodb-linux-i686-2.0.6/bin/logs/1001.log
format /^*(?<message>.*)$/
 tag mongo.apache
</source>
<source>
 type tail

#需要轉儲的日志2
 path /home/aircom/mongodb-linux-i686-2.0.6/bin/logs/1002.log
format /^*(?<message>.*)$/
 tag mongo.apache
</source>

源日志為2個日志文件。分別為path /home/aircom/mongodb-linux-i686-2.0.6/bin/logs/1001.log和path /home/aircom/mongodb-linux-i686-2.0.6/bin/logs/1002.log

標簽為tag mongo.apache,方便match段匹配。

<match mongo.**>
  type forward

#轉儲服務器
  host 192.168.19.201
  <secondary>
    host 192.168.19.201
  </secondary>
</match>

分別表示了2台機器,可以添加備用機,這里的主、備機IP均相同,生產環境可以添加不同的IP。


2、Server201上fluent.conf配置

為了方便測試,我們直接修改,將注釋直接打開,其含義為匹配沒有match的日志,寫入到文件。

<match **>
  type file

#存儲路徑
  path /var/log/fluent/else
  compress gz
</match>

啟動fluentd

201上查看配置的路徑

[root@Server201 fluentd-0.10.6]# ll  /var/log/fluent/
total 68
-rw-r--r-- 1 root root 61675 Sep 28 10:20 else.20120928.b4cab973e8d931ee9

產生了文件else.20120928.b4cab973e8d931ee9

3、啟動Server202下的fluentd:fluentd -c fluent.conf -deamon -o log/log.log

tail 輸入日志,得到后幾行輸出

2012-09-28 09:58:45 +0800: adding match pattern="mongo.**" type="forward"
2012-09-28 09:58:45 +0800: 'host' option in forward output is obsoleted. Use '<server> host xxx </server>' instead.
2012-09-28 09:58:45 +0800: adding forwarding server '192.168.19.201:24224' host="192.168.19.201" port=24224 weight=60
2012-09-28 09:58:45 +0800: 'host' option in forward output is obsoleted. Use '<server> host xxx </server>' instead.
2012-09-28 09:58:45 +0800: adding forwarding server '192.168.19.201:24224' host="192.168.19.201" port=24224 weight=60
2012-09-28 09:58:45 +0800: listening fluent socket on 0.0.0.0:24224
2012-09-28 09:58:45 +0800: following tail of /home/aircom/mongodb-linux-i686-2.0.6/bin/logs/1001.log
2012-09-28 09:58:45 +0800: following tail of /home/aircom/mongodb-linux-i686-2.0.6/bin/logs/1002.log

標記顏色的2行表示2台接受數據的機子,端口為24224,

開始發送數據的時候這個日志還會得到如下輸出

2012-09-28 10:02:07 +0800: detached forwarding server '192.168.19.201:24224' host="192.168.19.201" port=24224 phi=8.280932634573869
2012-09-28 10:02:07 +0800: detached forwarding server '192.168.19.201:24224' host="192.168.19.201" port=24224 phi=8.280785193737733
2012-09-28 10:02:42 +0800: recovered forwarding server '192.168.19.201:24224' host="192.168.19.201" port=24224
2012-09-28 10:02:42 +0800: recovered forwarding server '192.168.19.201:24224' host="192.168.19.201" port=24224

4、查看Server201上橋接轉儲后的日志

tail -f /var/log/fluent/else.20120928.b4cab973e8d931ee9

大概顯示如下輸出,日志刷新頻次為1分鍾

2012-09-28 10:23:09 +0800: plugin/in_forward.rb:140:initialize: accepted fluent socket object_id=242397940
2012-09-28 10:23:09 +0800: plugin/in_forward.rb:181:on_close: closed fluent socket object_id=242397940
2012-09-28T10:23:09+08:00       fluent.trace    {"message":"accepted fluent socket object_id=242397940"}
2012-09-28T10:22:23+08:00       mongo.apache    {"message":"Fri Sep 28 10:22:23 [conn369] end connection 192.168.19.201:51962"}
2012-09-28T10:22:23+08:00       mongo.apache    {"message":"Fri Sep 28 10:22:23 [initandlisten] connection accepted from 192.168.19.201:51968 #372"}
2012-09-28T10:22:27+08:00       mongo.apache    {"message":"Fri Sep 28 10:22:27 [conn370] end connection 192.168.19.201:51963"}
2012-09-28T10:22:27+08:00       mongo.apache    {"message":"Fri Sep 28 10:22:27 [initandlisten] connection accepted from 192.168.19.201:51969 #373"}
2012-09-28T10:22:33+08:00       mongo.apache    {"message":"Fri Sep 28 10:22:33 [conn371] end connection 192.168.19.202:39038"}
2012-09-28T10:22:33+08:00       mongo.apache    {"message":"Fri Sep 28 10:22:33 [initandlisten] connection accepted from 192.168.19.202:39045 #374"}
2012-09-28T10:22:34+08:00       mongo.apache    {"message":"Fri Sep 28 10:22:34 [conn378] end connection 192.168.19.202:55169"}
2012-09-28T10:22:34+08:00       mongo.apache    {"message":"Fri Sep 28 10:22:34 [initandlisten] connection accepted from 192.168.19.202:55176 #381"}
2012-09-28T10:22:35+08:00       mongo.apache    {"message":"Fri Sep 28 10:22:35 [conn379] end connection 192.168.19.201:41497"}
2012-09-28T10:22:35+08:00       mongo.apache    {"message":"Fri Sep 28 10:22:35 [initandlisten] connection accepted from 192.168.19.201:41503 #382"}
2012-09-28T10:22:37+08:00       mongo.apache    {"message":"Fri Sep 28 10:22:37 [conn380] end connection 192.168.19.201:41498"}
2012-09-28T10:22:37+08:00       mongo.apache    {"message":"Fri Sep 28 10:22:37 [initandlisten] connection accepted from 192.168.19.201:41504 #383"}
2012-09-28T10:22:53+08:00       mongo.apache    {"message":"Fri Sep 28 10:22:53 [conn372] end connection 192.168.19.201:51968"}
2012-09-28T10:22:53+08:00       mongo.apache    {"message":"Fri Sep 28 10:22:53 [initandlisten] connection accepted from 192.168.19.201:51974 #375"}
2012-09-28T10:22:56+08:00       mongo.apache    {"message":"Fri Sep 28 10:22:56 [clientcursormon] mem (MB) res:76 virt:419 mapped:224"}
2012-09-28T10:22:57+08:00       mongo.apache    {"message":"Fri Sep 28 10:22:57 [conn373] end connection 192.168.19.201:51969"}
2012-09-28T10:22:57+08:00       mongo.apache    {"message":"Fri Sep 28 10:22:57 [initandlisten] connection accepted from 192.168.19.201:51975 #376"}
2012-09-28T10:23:01+08:00       mongo.apache    {"message":"Fri Sep 28 10:23:01 [clientcursormon] mem (MB) res:44 virt:329 mapped:160"}
2012-09-28T10:23:03+08:00       mongo.apache    {"message":"Fri Sep 28 10:23:03 [conn374] end connection 192.168.19.202:39045"}
2012-09-28T10:23:03+08:00       mongo.apache    {"message":"Fri Sep 28 10:23:03 [initandlisten] connection accepted from 192.168.19.202:39051 #377"}
2012-09-28T10:23:04+08:00       mongo.apache    {"message":"Fri Sep 28 10:23:04 [conn381] end connection 192.168.19.202:55176"}
2012-09-28T10:23:04+08:00       mongo.apache    {"message":"Fri Sep 28 10:23:04 [initandlisten] connection accepted from 192.168.19.202:55182 #384"}
2012-09-28T10:23:05+08:00       mongo.apache    {"message":"Fri Sep 28 10:23:05 [conn382] end connection 192.168.19.201:41503"}
2012-09-28T10:23:05+08:00       mongo.apache    {"message":"Fri Sep 28 10:23:05 [initandlisten] connection accepted from 192.168.19.201:41509 #385"}
2012-09-28T10:23:07+08:00       mongo.apache    {"message":"Fri Sep 28 10:23:07 [conn383] end connection 192.168.19.201:41504"}
2012-09-28T10:23:07+08:00       mongo.apache    {"message":"Fri Sep 28 10:23:07 [initandlisten] connection accepted from 192.168.19.201:41510 #386"}
2012-09-28T10:23:09+08:00       fluent.trace    {"message":"closed fluent socket object_id=242397940"}


至此已經完全實現2台服務器上2個fluentd的橋接。


注意!

本站转载的文章为个人学习借鉴使用,本站对版权不负任何法律责任。如果侵犯了您的隐私权益,请联系我们删除。



 
粤ICP备14056181号  © 2014-2020 ITdaan.com