如何在Logstash中處理JSON嵌套數組

[英]How to process JSON nested array in Logstash


I have a nested field with arrays in array in JSON like the following:

我有一個嵌套字段,在JSON數組中有數組,如下所示:

{
    "foo": {
        "bar": [
            [
                "a",
                "b"
            ],
            [
                "c",
                "d"
            ]
        ]
    }
}

The following is my config file:

以下是我的配置文件:

input {
    file {
        codec => "json"
        path => "pathtofile"
        type => "footype"
        start_position => "beginning"
    }
}
filter {
    json {
        source => "message"
        remove_field => [ "host", "message", "path" ]
    }
}
output {
    elasticsearch {
        action => "index"
        index => "bar"
        hosts => [ "http://localhost:9200" ]
    }
}

I got the following error:

我收到以下錯誤:

09:40:47.725 [[main]>worker0] WARN logstash.outputs.elasticsearch - Failed action. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"bar", :_type=>"footype", :_routing=>nil}, 2017-02-13T01:40:30.387Z myconnection %{message}], :response=>{"index"=>{"_index"=>"bar", "_type"=>"footype", "_id"=>"AVo1IN0vK2jgwdCXqZ-q", "status"=>400, "error"=>{"type"=>"illegal_argument_exception", "reason"=>"mapper [foo.bar] of different type, current_type [long], merged_type [text]"}}}}

09:40:47.725 [[main]> worker0] WARN logstash.outputs.elasticsearch - 失敗的操作。 {:status => 400,:action => [“index”,{:_id => nil,:_index =>“bar”,:_ type =>“footype”,:_routing => nil},2017-02- 13T01:40:30.387Z myconnection%{message}],:response => {“index”=> {“_ index”=>“bar”,“_ type”=>“footype”,“_ id”=>“AVo1IN0vK2jgwdCXqZ- q“,”status“=> 400,”error“=> {”type“=>”illegal_argument_exception“,”reason“=>”不同類型的mapper [foo.bar],current_type [long],merged_type [text] “}}}}

I have a feeling that it's the array problem. I have done some research and know that array is not well supported. But I need to ingest the array in elasticsearch. Is there a way to actually do that?

我有一種感覺,這是陣列問題。我做了一些研究,並且知道陣列沒有得到很好的支持。但我需要在elasticsearch中攝取數組。有沒有辦法真正做到這一點?

Any helps will be appreciated.

任何幫助將不勝感激。

1 个解决方案

#1


0  

I solved this by using a ruby filter:

我通過使用紅寶石過濾器解決了這個問題:

ruby {
        code => '
            j = 0
            for i in event.get("[foo][bar]") do
                #i is an array element in the big array
                l = 0
                for k in i do
                    event.set("item#" + j.to_s + "#" + l.to_s, k)
                    l = l + 1
                end
                j = j + 1
            end
        '
    }

This will eventually produce fields

這最終會產生字段

item#0#0 = "a"
item#0#1 = "b"
item#1#0 = "c"
item#1#1 = "d"

注意!

本站翻译的文章,版权归属于本站,未经许可禁止转摘,转摘请注明本文地址:https://www.itdaan.com/blog/2017/02/14/72f1533ee09cb2f4e1d2bab17a887c80.html



 
粤ICP备14056181号  © 2014-2021 ITdaan.com