[Elasticsearch] 數據建模 - 處理關聯關系(2)


字段折疊(Field Collapsing)

一個常見的需求是通過對某個特定的字段分組來展現搜索結果。我們或許希望通過對用戶名分組來返回最相關的博文。對用戶名分組意味着我們需要使用到terms聚合。為了對用戶的全名進行分組,name字段需要有not_analyzed的原始值,如聚合和分析中解釋的那樣。

<code class="language-json hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background: transparent;">PUT /my_index/_mapping/blogpost
{
"<span class="hljs-attribute" style="box-sizing: border-box;">properties</span>": <span class="hljs-value" style="box-sizing: border-box;">{
"<span class="hljs-attribute" style="box-sizing: border-box;">user</span>": <span class="hljs-value" style="box-sizing: border-box;">{
"<span class="hljs-attribute" style="box-sizing: border-box;">properties</span>": <span class="hljs-value" style="box-sizing: border-box;">{
"<span class="hljs-attribute" style="box-sizing: border-box;">name</span>": <span class="hljs-value" style="box-sizing: border-box;">{ (1)
"<span class="hljs-attribute" style="box-sizing: border-box;">type</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"string"</span></span>,
"<span class="hljs-attribute" style="box-sizing: border-box;">fields</span>": <span class="hljs-value" style="box-sizing: border-box;">{
"<span class="hljs-attribute" style="box-sizing: border-box;">raw</span>": <span class="hljs-value" style="box-sizing: border-box;">{ (2)
"<span class="hljs-attribute" style="box-sizing: border-box;">type</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"string"</span></span>,
"<span class="hljs-attribute" style="box-sizing: border-box;">index</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"not_analyzed"</span>
</span>}
</span>}
</span>}
</span>}
</span>}
</span>}
</span>}</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li></ul>

(1) 字段用來支持全文搜索。 
(2) 字段用來支持terms聚合來完成分組。

然后添加一些數據:

<code class="language-json hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background: transparent;">PUT /my_index/user/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>
{
"<span class="hljs-attribute" style="box-sizing: border-box;">name</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"John Smith"</span></span>,
"<span class="hljs-attribute" style="box-sizing: border-box;">email</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"john@smith.com"</span></span>,
"<span class="hljs-attribute" style="box-sizing: border-box;">dob</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"1970/10/24"</span>
</span>}

PUT /my_index/blogpost/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>
{
"<span class="hljs-attribute" style="box-sizing: border-box;">title</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"Relationships"</span></span>,
"<span class="hljs-attribute" style="box-sizing: border-box;">body</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"It's complicated..."</span></span>,
"<span class="hljs-attribute" style="box-sizing: border-box;">user</span>": <span class="hljs-value" style="box-sizing: border-box;">{
"<span class="hljs-attribute" style="box-sizing: border-box;">id</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span></span>,
"<span class="hljs-attribute" style="box-sizing: border-box;">name</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"John Smith"</span>
</span>}
</span>}

PUT /my_index/user/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span>
{
"<span class="hljs-attribute" style="box-sizing: border-box;">name</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"Alice John"</span></span>,
"<span class="hljs-attribute" style="box-sizing: border-box;">email</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"alice@john.com"</span></span>,
"<span class="hljs-attribute" style="box-sizing: border-box;">dob</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"1979/01/04"</span>
</span>}

PUT /my_index/blogpost/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">4</span>
{
"<span class="hljs-attribute" style="box-sizing: border-box;">title</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"Relationships are cool"</span></span>,
"<span class="hljs-attribute" style="box-sizing: border-box;">body</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"It's not complicated at all..."</span></span>,
"<span class="hljs-attribute" style="box-sizing: border-box;">user</span>": <span class="hljs-value" style="box-sizing: border-box;">{
"<span class="hljs-attribute" style="box-sizing: border-box;">id</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span></span>,
"<span class="hljs-attribute" style="box-sizing: border-box;">name</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"Alice John"</span>
</span>}
</span>}</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li></ul>

現在我們可以運行一個查詢來獲取關於relationships的博文,通過用戶名為John對結果進行分組。這都要感謝top_hits聚合:

<code class="language-json hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background: transparent;">GET /my_index/blogpost/_search?search_type=count (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>)
{
"<span class="hljs-attribute" style="box-sizing: border-box;">query</span>": <span class="hljs-value" style="box-sizing: border-box;">{ (2)
"<span class="hljs-attribute" style="box-sizing: border-box;">bool</span>": <span class="hljs-value" style="box-sizing: border-box;">{
"<span class="hljs-attribute" style="box-sizing: border-box;">must</span>": <span class="hljs-value" style="box-sizing: border-box;">[
{ "<span class="hljs-attribute" style="box-sizing: border-box;">match</span>": <span class="hljs-value" style="box-sizing: border-box;">{ "<span class="hljs-attribute" style="box-sizing: border-box;">title</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"relationships"</span> </span>}</span>},
{ "<span class="hljs-attribute" style="box-sizing: border-box;">match</span>": <span class="hljs-value" style="box-sizing: border-box;">{ "<span class="hljs-attribute" style="box-sizing: border-box;">user.name</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"John"</span> </span>}</span>}
]
</span>}
</span>}</span>,
"<span class="hljs-attribute" style="box-sizing: border-box;">aggs</span>": <span class="hljs-value" style="box-sizing: border-box;">{
"<span class="hljs-attribute" style="box-sizing: border-box;">users</span>": <span class="hljs-value" style="box-sizing: border-box;">{
"<span class="hljs-attribute" style="box-sizing: border-box;">terms</span>": <span class="hljs-value" style="box-sizing: border-box;">{
"<span class="hljs-attribute" style="box-sizing: border-box;">field</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"user.name.raw"</span></span>, (3)
"<span class="hljs-attribute" style="box-sizing: border-box;">order</span>": <span class="hljs-value" style="box-sizing: border-box;">{ "<span class="hljs-attribute" style="box-sizing: border-box;">top_score</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"desc"</span> </span>} (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">4</span>)
</span>}</span>,
"<span class="hljs-attribute" style="box-sizing: border-box;">aggs</span>": <span class="hljs-value" style="box-sizing: border-box;">{
"<span class="hljs-attribute" style="box-sizing: border-box;">top_score</span>": <span class="hljs-value" style="box-sizing: border-box;">{ "<span class="hljs-attribute" style="box-sizing: border-box;">max</span>": <span class="hljs-value" style="box-sizing: border-box;">{ "<span class="hljs-attribute" style="box-sizing: border-box;">script</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"_score"</span> </span>}</span>}</span>, (5)
"<span class="hljs-attribute" style="box-sizing: border-box;">blogposts</span>": <span class="hljs-value" style="box-sizing: border-box;">{ "<span class="hljs-attribute" style="box-sizing: border-box;">top_hits</span>": <span class="hljs-value" style="box-sizing: border-box;">{ "<span class="hljs-attribute" style="box-sizing: border-box;">_source</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"title"</span></span>, "<span class="hljs-attribute" style="box-sizing: border-box;">size</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">5</span> </span>}</span>} (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">6</span>)
</span>}
</span>}
</span>}
</span>}</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li></ul>

(1) 我們感興趣的博文在blogposts聚合中被返回了,因此我們可以通過設置search_type=count來禁用通常的搜索結果。

(2) 該查詢返回用戶名為John,title匹配relationships的博文。

(3) terms聚合為每個user.name.raw值創建一個桶。

(4)(5) 在users聚合中,使用top_score聚合通過每個桶中擁有最高分值的文檔進行排序。

(6) top_hits聚合只返回每個用戶的5篇最相關博文的title字段。

部分響應如下所示:

<code class="language-json hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background: transparent;">...
<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"hits"</span>: {
"<span class="hljs-attribute" style="box-sizing: border-box;">total</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span></span>,
"<span class="hljs-attribute" style="box-sizing: border-box;">max_score</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span></span>,
"<span class="hljs-attribute" style="box-sizing: border-box;">hits</span>": <span class="hljs-value" style="box-sizing: border-box;">[] (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>)
</span>},
<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"aggregations"</span>: {
"<span class="hljs-attribute" style="box-sizing: border-box;">users</span>": <span class="hljs-value" style="box-sizing: border-box;">{
"<span class="hljs-attribute" style="box-sizing: border-box;">buckets</span>": <span class="hljs-value" style="box-sizing: border-box;">[
{
"<span class="hljs-attribute" style="box-sizing: border-box;">key</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"John Smith"</span></span>, (2)
"<span class="hljs-attribute" style="box-sizing: border-box;">doc_count</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span></span>,
"<span class="hljs-attribute" style="box-sizing: border-box;">blogposts</span>": <span class="hljs-value" style="box-sizing: border-box;">{
"<span class="hljs-attribute" style="box-sizing: border-box;">hits</span>": <span class="hljs-value" style="box-sizing: border-box;">{ (3)
"<span class="hljs-attribute" style="box-sizing: border-box;">total</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span></span>,
"<span class="hljs-attribute" style="box-sizing: border-box;">max_score</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.35258877</span></span>,
"<span class="hljs-attribute" style="box-sizing: border-box;">hits</span>": <span class="hljs-value" style="box-sizing: border-box;">[
{
"<span class="hljs-attribute" style="box-sizing: border-box;">_index</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"my_index"</span></span>,
"<span class="hljs-attribute" style="box-sizing: border-box;">_type</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"blogpost"</span></span>,
"<span class="hljs-attribute" style="box-sizing: border-box;">_id</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"2"</span></span>,
"<span class="hljs-attribute" style="box-sizing: border-box;">_score</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.35258877</span></span>,
"<span class="hljs-attribute" style="box-sizing: border-box;">_source</span>": <span class="hljs-value" style="box-sizing: border-box;">{
"<span class="hljs-attribute" style="box-sizing: border-box;">title</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"Relationships"</span>
</span>}
</span>}
]
</span>}
</span>}</span>,
"<span class="hljs-attribute" style="box-sizing: border-box;">top_score</span>": <span class="hljs-value" style="box-sizing: border-box;">{ (4)
"<span class="hljs-attribute" style="box-sizing: border-box;">value</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.3525887727737427</span>
</span>}
</span>},
...</span></span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li><li style="box-sizing: border-box; padding: 0px 5px;">34</li></ul>

(1) hits數組為空因為我們設置了search_type=count

(2) 針對每個用戶都有一個對應的桶。

(3) 在每個用戶的桶中,有一個blogposts.hits數組,它包含了該用戶的相關度最高的搜索結果。

(4) 用戶桶通過用戶最相關的博文進行排序。

使用top_hits聚合等效於運行獲取最相關博文及其對應用戶的查詢,然后針對每個用戶運行同樣的查詢,來得到每個用戶最相關的博文。可見使用top_hits更高效。

每個桶中返回的top hits是通過運行一個基於原始主查詢的迷你查詢而來。該迷你查詢也同樣支持高亮(Highlighting)以及分頁(Pagination)。


反規范化和並發(Denormalization and Concurrency)

當然,數據反規范化也有弊端。首先,它會讓索引變大,因為每篇博文的_source都變大了,與此同時需要索引的字段也變多了。通常這並不是一個大問題。寫入到磁盤的數據會被高度壓縮,而且磁盤存儲空間也不貴。ES能夠很從容地處理這些多出來的數據。

更重要的弊端在於,如果用戶修改了他的名字,他名下的所有博文都需要被更新。即使用戶真的這么做了,一位用戶也不太可能寫了上千篇博文,因此通過scrollbulk APIs,更新也不會超過1秒。

然而,讓我們來考慮一個變化更常見,影響更深遠和重要的復雜情景 - 並發。

在這個例子中,我們通過ES來模擬一個擁有目錄樹的文件系統,就像Linux上的文件系統那樣:根目錄是/,每個目錄都能包含文件和子目錄。

我們希望能夠搜索某個目錄下的文件,就像下面這樣:

grep "some text" /clinton/projects/elasticsearch/*

它需要我們對文件的路徑進行索引:

<code class="language-json hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background: transparent;">PUT /fs/file/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>
{
"<span class="hljs-attribute" style="box-sizing: border-box;">name</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"README.txt"</span></span>,
"<span class="hljs-attribute" style="box-sizing: border-box;">path</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"/clinton/projects/elasticsearch"</span></span>,
"<span class="hljs-attribute" style="box-sizing: border-box;">contents</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"Starting a new Elasticsearch project is easy..."</span>
</span>}</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li></ul>

NOTE

說實在的,我們同時也應該一個目錄下所有的文件和子目錄進行索引,但是為了簡潔起見,這里忽略了這一需求。

我們還需要能夠搜索某個目錄下任意深度的文件,就像下面這樣:

grep -r "some text" /clinton

為了支持這一需求,需要對路徑層次進行索引:

  • /clinton
  • /clinton/projects
  • /clinton/projects/elasticsearch

該層次結構可以通過對path字段使用path_hierarchy tokenizer來自動生成:

<code class="language-json hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background: transparent;">PUT /fs
{
"<span class="hljs-attribute" style="box-sizing: border-box;">settings</span>": <span class="hljs-value" style="box-sizing: border-box;">{
"<span class="hljs-attribute" style="box-sizing: border-box;">analysis</span>": <span class="hljs-value" style="box-sizing: border-box;">{
"<span class="hljs-attribute" style="box-sizing: border-box;">analyzer</span>": <span class="hljs-value" style="box-sizing: border-box;">{
"<span class="hljs-attribute" style="box-sizing: border-box;">paths</span>": <span class="hljs-value" style="box-sizing: border-box;">{ (1)
"<span class="hljs-attribute" style="box-sizing: border-box;">tokenizer</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"path_hierarchy"</span>
</span>}
</span>}
</span>}
</span>}
</span>}</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li></ul>

(1) 以上的自定義的分析器使用了path_hierarchy分詞器,使用其默認設置。參見path_hierarchy tokenizer

文件類型的映射則像下面這樣:

<code class="language-json hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background: transparent;">PUT /fs/_mapping/file
{
"<span class="hljs-attribute" style="box-sizing: border-box;">properties</span>": <span class="hljs-value" style="box-sizing: border-box;">{
"<span class="hljs-attribute" style="box-sizing: border-box;">name</span>": <span class="hljs-value" style="box-sizing: border-box;">{ (1)
"<span class="hljs-attribute" style="box-sizing: border-box;">type</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"string"</span></span>,
"<span class="hljs-attribute" style="box-sizing: border-box;">index</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"not_analyzed"</span>
</span>}</span>,
"<span class="hljs-attribute" style="box-sizing: border-box;">path</span>": <span class="hljs-value" style="box-sizing: border-box;">{ (2)
"<span class="hljs-attribute" style="box-sizing: border-box;">type</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"string"</span></span>,
"<span class="hljs-attribute" style="box-sizing: border-box;">index</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"not_analyzed"</span></span>,
"<span class="hljs-attribute" style="box-sizing: border-box;">fields</span>": <span class="hljs-value" style="box-sizing: border-box;">{
"<span class="hljs-attribute" style="box-sizing: border-box;">tree</span>": <span class="hljs-value" style="box-sizing: border-box;">{ (3)
"<span class="hljs-attribute" style="box-sizing: border-box;">type</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"string"</span></span>,
"<span class="hljs-attribute" style="box-sizing: border-box;">analyzer</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"paths"</span>
</span>}
</span>}
</span>}
</span>}
</span>}</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li></ul>

(1) name字段會包含完整的名字。

(2)(3) path字段會包含完整的目錄名,而path.tree字段會包含路徑層次結構。

一旦建立了該索引並完成了文件的索引,我們就能夠執行如下查詢,它搜索/client/projects/elasticsearch目錄下包含有elasticsearch的文件:

<code class="language-json hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background: transparent;">GET /fs/file/_search
{
"<span class="hljs-attribute" style="box-sizing: border-box;">query</span>": <span class="hljs-value" style="box-sizing: border-box;">{
"<span class="hljs-attribute" style="box-sizing: border-box;">filtered</span>": <span class="hljs-value" style="box-sizing: border-box;">{
"<span class="hljs-attribute" style="box-sizing: border-box;">query</span>": <span class="hljs-value" style="box-sizing: border-box;">{
"<span class="hljs-attribute" style="box-sizing: border-box;">match</span>": <span class="hljs-value" style="box-sizing: border-box;">{
"<span class="hljs-attribute" style="box-sizing: border-box;">contents</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"elasticsearch"</span>
</span>}
</span>}</span>,
"<span class="hljs-attribute" style="box-sizing: border-box;">filter</span>": <span class="hljs-value" style="box-sizing: border-box;">{
"<span class="hljs-attribute" style="box-sizing: border-box;">term</span>": <span class="hljs-value" style="box-sizing: border-box;">{ (1)
"<span class="hljs-attribute" style="box-sizing: border-box;">path</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"/clinton/projects/elasticsearch"</span>
</span>}
</span>}
</span>}
</span>}
</span>}</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li></ul>

(1) 只在該目錄下搜索。

任何存儲於/clinton目錄下的文件都會在path.tree字段中包含/clinton。因此我們可以通過下面的搜索來得到/clinton目錄下的所有文件:

<code class="language-json hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background: transparent;">GET /fs/file/_search
{
"<span class="hljs-attribute" style="box-sizing: border-box;">query</span>": <span class="hljs-value" style="box-sizing: border-box;">{
"<span class="hljs-attribute" style="box-sizing: border-box;">filtered</span>": <span class="hljs-value" style="box-sizing: border-box;">{
"<span class="hljs-attribute" style="box-sizing: border-box;">query</span>": <span class="hljs-value" style="box-sizing: border-box;">{
"<span class="hljs-attribute" style="box-sizing: border-box;">match</span>": <span class="hljs-value" style="box-sizing: border-box;">{
"<span class="hljs-attribute" style="box-sizing: border-box;">contents</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"elasticsearch"</span>
</span>}
</span>}</span>,
"<span class="hljs-attribute" style="box-sizing: border-box;">filter</span>": <span class="hljs-value" style="box-sizing: border-box;">{
"<span class="hljs-attribute" style="box-sizing: border-box;">term</span>": <span class="hljs-value" style="box-sizing: border-box;">{ (1)
"<span class="hljs-attribute" style="box-sizing: border-box;">path.tree</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"/clinton"</span>
</span>}
</span>}
</span>}
</span>}
</span>}</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li></ul>

(1) 尋找該目錄或其下任意子目錄中的文件。

重命名文件和目錄(Renaming Files and Directories)

到目前為止還不錯。文件重命挺簡單 - 只需要一個簡單的更新或者索引請求就行了。你甚至可以使用樂觀並發控制(Optimistic Concurrency Control)來確保你的更改不會和另一個用戶的更改發生沖突。

<code class="language-json hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background: transparent;">PUT /fs/file/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>?version=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>    (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>)
{
"<span class="hljs-attribute" style="box-sizing: border-box;">name</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"README.asciidoc"</span></span>,
"<span class="hljs-attribute" style="box-sizing: border-box;">path</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"/clinton/projects/elasticsearch"</span></span>,
"<span class="hljs-attribute" style="box-sizing: border-box;">contents</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"Starting a new Elasticsearch project is easy..."</span>
</span>}</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li></ul>

(1) version值能夠確保只有當索引中的文檔擁有相同的version值時,更改才會生效。

我們還可以對目錄重命名,但是這意味着對該目錄下的所有文件執行更新。這個操作或快或慢,取決於有多少文件需要被更新。我們需要做的是使用scan-and-scroll來獲取所有的文件,然后使用bulk API完成更新。這個過程並不是原子性的,但是所有的文件都能夠被迅速地更新。


解決並發問題(Solving Concurrency Issues)

當我們允許多個用戶同時對文件和目錄進行重命名時,問題就來了。假設你對/clinton目錄進行重命名,它包含了成百上千個文件。同時,另外一個用戶重命名了/clinton/projects/elasticsearch/README.txt這個文件。該用戶的更改雖然在你的操作之后,但是它完成的也許更快。

下面兩種情況中的一種會發生:

  • 你決定使用version值,這意味着你對文件README.txt的重命名操作會因為version沖突而失敗。
  • 你不使用versioning,那么你的更改會直接覆蓋掉另一用戶的更改。

問題的根源在ES並不支持ACID事務。對單個文檔的更新雖然是符合ACID原則的,但是對多個文檔的更新則不然。

如果你使用的主要數據源是關系行數據庫,而ES只是簡單地被用來當作搜索引擎或者提升性能的方法,那么首先更新數據庫,然后待這些更新操作成功后再將這些變更復制到ES中。這樣的話,你就能受益於數據庫對於ACID事務的支持了,它能保證ES中的所有更新順序都是正確的。並發的問題在關系型數據庫中被處理了。

如果你沒有使用關系型數據源,那么這些並發問題就需要在ES中完成。下面有三種可行的方案,這些方案都涉及到了某種形式的鎖:

  • 全局鎖(Global Locking)
  • 文檔鎖(Document Locking)
  • 樹鎖(Tree Locking)

TIP

以上提到的方案都可以通過應用了相同原則的外部系統實現。

全局鎖(Global Locking)

我們可以通過在任何時候只允許一個進程執行更新操作來完全避免並發性的問題。大多數的修改只設計很少的幾個文件,完成的也相當快。對於頂層目錄的重命名也許會阻塞其它變更操作更久一點,但是這種情況發生的頻率也會更低一些。

因為在ES中文檔級別的變更是滿足ACID的,我們可以將一份文檔是否存在作為一個全局鎖。為了獲取一個鎖,我們嘗試去創建一個全局鎖文檔:

<code class="language-json hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background: transparent;">PUT /fs/lock/global/_create
{}</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li></ul>

如果上述的創建請求由於發生了沖突而失敗了,就表明另一個進程已經被授權得到了全局鎖,我們只能稍后重試。如果上述請求成功了,我們就擁有了全局鎖從而可以進行后續的變更操作。一旦這些操作完成了,必須通過刪除全局鎖文檔來釋放它:

<code class="language-json hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background: transparent;">DELETE /fs/lock/global</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>

取決於變更的頻繁程度和它們需要消耗的時間,全局鎖對系統的性能或許會有相當程度的限制。可以通過實現更加細粒度的鎖來增加並行性。

文檔鎖(Document Locking)

相比於對整個文件系統上鎖,我們可以通過上面提到的技術來對單個文檔完成鎖定。一個進程可以使用scan-and-scroll請求來獲取到變更會影響到的所有文檔的IDs,然后對每份文檔創建一個鎖文件:

<code class="language-json hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background: transparent;">PUT /fs/lock/_bulk
{ "<span class="hljs-attribute" style="box-sizing: border-box;">create</span>": <span class="hljs-value" style="box-sizing: border-box;">{ "<span class="hljs-attribute" style="box-sizing: border-box;">_id</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span></span>}</span>} (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>)
{ "<span class="hljs-attribute" style="box-sizing: border-box;">process_id</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">123</span> </span>} (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>)
{ "<span class="hljs-attribute" style="box-sizing: border-box;">create</span>": <span class="hljs-value" style="box-sizing: border-box;">{ "<span class="hljs-attribute" style="box-sizing: border-box;">_id</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span></span>}</span>}
{ "<span class="hljs-attribute" style="box-sizing: border-box;">process_id</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">123</span> </span>}
...</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li></ul>

(1) 鎖文檔的ID需要和被鎖定的文檔的ID一致。

(2) process_id是將要執行變更操作進程的唯一ID。

如果某些文檔已經被鎖了,那么部分bulk請求會失敗,只好重試。

當然,如果我們試圖再次去鎖定所有的文檔,對於那些已經被我們鎖定的文檔,前面使用的創建語句會失敗!相比一個簡單的創建語句,我們需要使用帶有upsert參數的update請求:

<code class="language-json hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background: transparent;">if ( ctx._source.process_id != process_id ) {   (1)
assert false; (2)
}
ctx.op = 'noop'; (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span>)</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li></ul>

(1) process_id是我們傳入到腳本中的參數。

(2) assert false會拋出一個異常,它導致更新失敗。

(3) 將op從update修改為noop能夠防止update請求真的執行變更操作,而仍然返回成功。

完整的update請求如下所示:

<code class="language-json hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background: transparent;">POST /fs/lock/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>/_update
{
"<span class="hljs-attribute" style="box-sizing: border-box;">upsert</span>": <span class="hljs-value" style="box-sizing: border-box;">{ "<span class="hljs-attribute" style="box-sizing: border-box;">process_id</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">123</span> </span>}</span>,
"<span class="hljs-attribute" style="box-sizing: border-box;">script</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"if ( ctx._source.process_id != process_id )
{ assert false }; ctx.op = 'noop';"</span>
<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"params"</span>: {
"<span class="hljs-attribute" style="box-sizing: border-box;">process_id</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">123</span>
</span>}
</span>}</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li></ul>

如果文檔並不存在,upsert會執行insert操作 - 就和之前使用的create請求一樣。但是,如果文檔存在,腳本會查看文檔中保存的process_id。如果它和我們的相同,就會放棄update(noop)並返回成功。如果它和我們的不同,那么assert false就會拋出一個異常告訴我們鎖定失敗了。

一旦所有的鎖都別成功創建了,重命名操作就開始了。在這之后,我們必須釋放所有的鎖,通過delete-by-query請求來完成:

<code class="language-json hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background: transparent;">POST /fs/_refresh   (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>)

DELETE /fs/lock/_query
{
"<span class="hljs-attribute" style="box-sizing: border-box;">query</span>": <span class="hljs-value" style="box-sizing: border-box;">{
"<span class="hljs-attribute" style="box-sizing: border-box;">term</span>": <span class="hljs-value" style="box-sizing: border-box;">{
"<span class="hljs-attribute" style="box-sizing: border-box;">process_id</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">123</span>
</span>}
</span>}
</span>}</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li></ul>

(1) refresh調用保證了所有的鎖文檔對delete-by-query請求是可見的。

文檔級別的鎖擁有更加細粒度的訪問控制,但是為百萬計的文檔創建鎖是非常昂貴的。在某些場合下,比如前面例子中出現的目錄樹,可以通過更少的工作來達到細粒度的鎖定。

樹鎖(ree Locking)

相比像前面那樣對每份文檔上鎖,也可以支隊目錄樹的部分上鎖。我們需要對重命名的文檔或者目錄擁有獨占性訪問,可以通過獨占性鎖文檔實現:

<code class="language-json hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background: transparent;">{ "<span class="hljs-attribute" style="box-sizing: border-box;">lock_type</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"exclusive"</span> </span>}</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>

同時我們也需要對上層目錄使用分享鎖:

<code class="language-json hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background: transparent;">{
"<span class="hljs-attribute" style="box-sizing: border-box;">lock_type</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"shared"</span></span>,
"<span class="hljs-attribute" style="box-sizing: border-box;">lock_count</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span> (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>)
</span>}</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li></ul>

(1) lock_count記錄了擁有該分享鎖的進程數量。

對/clinton/projects/elasticsearch/README.txt重命名的進程需要該文件的獨占鎖,以及針對目錄/clinton,/clinton/projects和/clinton/projects/elasticsearch的分享鎖。

對於獨占鎖,可以通過簡單的創建請求來實現,但是分享鎖需要帶有腳本的update請求來實現:

<code class="language-json hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background: transparent;">if (ctx._source.lock_type == 'exclusive') {
assert false; (1)
}
ctx._source.lock_count++ (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>)</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li></ul>

(1) 如果lock_type是獨占性的,那么assert語句會拋出一個異常,導致update請求失敗。

(2) 否則,增加lock_count。

該腳本能夠處理當鎖文檔已經存在的情況,但是我們仍然需要使用upsert來處理它不存在時的情況。完整的update請求如下所示:

<code class="language-json hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background: transparent;">POST /fs/lock/%<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>Fclinton/_update   (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>)
{
"<span class="hljs-attribute" style="box-sizing: border-box;">upsert</span>": <span class="hljs-value" style="box-sizing: border-box;">{ (2)
"<span class="hljs-attribute" style="box-sizing: border-box;">lock_type</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"shared"</span></span>,
"<span class="hljs-attribute" style="box-sizing: border-box;">lock_count</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>
</span>}</span>,
"<span class="hljs-attribute" style="box-sizing: border-box;">script</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"if (ctx._source.lock_type == 'exclusive')
{ assert false }; ctx._source.lock_count++"</span>
</span>}</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li></ul>

(1) 文檔的ID是/clinton,再進行URL編碼后變成了%2fclinton。

(2) 當文檔不存在時,upsert代表的文檔會被插入。

一旦我們成功獲取了文件所在目錄的所有上級目錄的分享鎖后,就可以嘗試去創建一個針對該文件的獨占鎖了:

<code class="language-json hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background: transparent;">PUT /fs/lock/%<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>Fclinton%<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>fprojects%<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>felasticsearch%<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>fREADME.txt/_create
{ "<span class="hljs-attribute" style="box-sizing: border-box;">lock_type</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"exclusive"</span> </span>}</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li></ul>

現在,如果另外的某個人想要對/clinton目錄重命名,他們就需要獲取該目錄的獨占鎖:

<code class="language-json hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background: transparent;">PUT /fs/lock/%<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>Fclinton/_create
{ "<span class="hljs-attribute" style="box-sizing: border-box;">lock_type</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"exclusive"</span> </span>}</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li></ul>

該請求會失敗,因為一個擁有相同ID的鎖文檔已經存在了。該用戶只好等待我們的操作完成並釋放鎖。獨占鎖可以被刪除:

<code class="language-json hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background: transparent;">DELETE /fs/lock/%<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>Fclinton%<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>fprojects%<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>felasticsearch%<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>fREADME.txt</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>

而分享鎖需要另一個腳本來減少lock_count的計數,如果該計數減少到了0,就刪除它:

<code class="language-json hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background: transparent;">if (--ctx._source.lock_count == <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>) {
ctx.op = 'delete' (1)
}</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li></ul>

(1) 一旦lock_count為0,ctx.op就從update變成delete。

對每個上級目錄都需要以相反的順序執行update請求,從最長的目錄到最短的目錄:

<code class="language-json hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background: transparent;">POST /fs/lock/%<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>Fclinton%<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>fprojects%<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>felasticsearch/_update
{
"<span class="hljs-attribute" style="box-sizing: border-box;">script</span>": <span class="hljs-value" style="box-sizing: border-box;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"if (--ctx._source.lock_count == 0) { ctx.op = 'delete' } "</span>
</span>}</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li></ul>

樹鎖通過最少的代價實現了細粒度的並發控制。當然,它並不能適用於每個場景 - 數據模型必須要有類似目錄樹這種結構才行。

NOTE

以上的三種方案 - 全局鎖,文檔鎖和樹鎖 - 都沒有處理關於鎖的最棘手的問題:如果持有鎖的進程掛了怎么辦?

一個進程的意外掛掉給我們留下了兩個問題:

  • 我們如何知道我們可以釋放被掛掉的進程持有的鎖?
  • 我們如何清理掛掉的進程沒有完成的工作?

這些話題都超出了本書的范疇,但是如果你決定使用鎖,就需要考慮一下它們。

盡管反規范化對於很多項目而言都是一個好的選擇,由於需要鎖的支持,會導致較為復雜的實現。相比之下,ES提供了另外兩種模型來處理關聯的實體:嵌套對象(Nested Objects)和父子關系(Parent-Child Relationship)。


注意!

本站转载的文章为个人学习借鉴使用,本站对版权不负任何法律责任。如果侵犯了您的隐私权益,请联系我们删除。



 
粤ICP备14056181号  © 2014-2021 ITdaan.com