Elasticsearch控制相关度分数API

旧城等待， 2023-06-27 03:45 74阅读 0赞

> 本文基于Elasticsearch7.x

Elasticsearch可以控制相关度分数计算的, 让满足一些条件的分数更高.

在学习本篇博客前先了解下[Elasticsearch全文搜索之基础语法API][Elasticsearch_API]

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdGlhbnhpYW5nX2thb2xh_size_16_color_FFFFFF_t_70]

### Rest API ###

Elasticsearch有三种控制相关度分数的方法:

*  boost
 *  boosting
 *  function\_score

**添加搜索实例数据**

POST /blogs/_bulk
    {"index": {}}
    {"post_date": "2020-01-01", "title": "Quick brown rabbits", "content": "Brown rabbits are commonly seen.", "author_id": 11401}
    {"index": {}}
    {"post_date": "2020-01-02", "title": "Keeping pets healthy", "content": "My quick brown fox eats rabbits on a regular basis.", "author_id": 11402}
    {"index": {}}
    {"post_date": "2020-01-03", "title": "My dog barks", "content": "I see a lot of barking dogs on the road.", "author_id": 11403}

**boost**

(1) 语法

*  当 boost > 1 时, 打分的权重相对性提升
 *  当 0 < boost < 1 时, 打分的权重相对性降低

(2) 实例

GET /blogs/_search
    {
      "query": {
        "bool": {
          "should": [
            {
             "match": {
               "content": {
                  "query": "rabbits",
                  "boost": 2
                }
             } 
            },
            {
              "match": {
                "title": {
                  "query": "brown",
                  "boost": 0.1
                }
              }
            }
          ]
        }
      }
    }

**boosting**

(1) 语法

*  positive  
    必须匹配的条件
 *  negative  
    匹配该条件的文档要减低相关度分数, 最终分数为匹配positive条件的分数 \* negative\_boost
 *  negative\_boost  
    降低匹配negative条件相关度分数的系数, 取值范围为0~1.0

(2) 实例

GET /blogs/_search
    {
      "query": {
        "boosting": {
          "positive": {
            "match": {
              "title": "rabbits"
            }
          },
          "negative": {
            "term": {
              "post_date": {
                "value": "2020-01-02"
              }
            }
          },
          "negative_boost": 0.2
        }
      }
    }

**function\_score**

function\_score可以在搜索结束后, 对每一个匹配的文档进行一系列的重新算分, 根据新生成的分数进行排序.

(1) 语法

GET /blogs/_search
    {
      "query": {
        "function_score": {
          "query": {
            全局匹配条件
          },
          "functions": [
            {
              "filter": {
                函数生效的匹配条件
              },
              function
            },
            {
              "filter": {
                函数生效的匹配条件
              },
              function
            }
          ],
          "boost": , 
          "score_mode": "",
          "boost_mode": "",
          "min_score": ,
          "max_boost": 
        }
      }
    }

a. functions

*  Weight  
    为函数算分设置一个权重.
 *  Random Score  
    生成一个\[0,1)的函数分数.
 *  Field Value Factor  
    使用字段值来计算函数得分.
 *  Decay Functions  
    使用衰减函数对文档进行评分, 设置一个字段值为中心点, 匹配文档的字段值离这个中心点越近, 分数越高. 字段类型只能是数值、时间以及地理坐标.
 *  Script Score  
    自定义脚本.

b. score\_mode

存在多个函数时, 计算最终函数分数的方式.

*  multiply  
    多个函数分数相乘, 默认.
 *  sum  
    多个函数分数相加.
 *  avg  
    取平均值.
 *  first  
    匹配条件的第一个函数分数.
 *  max  
    取最大值.
 *  min  
    取最小值.

c. boost\_mode

计算函数分数与查询分数的综合得分的方式.

*  multiply  
    相乘, 默认.
 *  replace  
    取函数分数.
 *  sum  
    相加.
 *  avg  
    取平均值.
 *  max  
    取最大值.
 *  min  
    取最小值.

d. max\_boost

最大的函数分数, 函数分数不能超过max\_boost.

e. min\_score

最小综合分数, 最终综合分数小于min\_score的文档将会被过滤掉.

f. boost

设置一个基础分数.

(2) 实例

a. Weight和Random Score

GET /blogs/_search
    {
      "query": {
        "function_score": {
          "query": {
            "match": {
              "content": "rabbits"
            }
          },
          "functions": [
            {
              "filter": {
                "term": {
                  "post_date": "2020-01-01"
                }
              },
              "random_score": {
                "seed": 10,
                "field": "_seq_no"
              },
              "weight": 23
            },
            {
              "filter": {
                "term": {
                  "author_id": "11402"
                }
              },
              "weight": 42
            }
          ],
          "boost": 5, 
          "score_mode": "multiply",
          "boost_mode": "multiply",
          "min_score": 42,
          "max_boost": 42
        }
      }
    }

b. Field Value Factor

*  field  
    字段.
 *  factor  
    相乘因子.
 *  modifier  
    数学函数, 如none, log, log1p, log2p, ln, ln1p, ln2p, square, sqrt, or reciprocal.
 *  missing  
    字段不存在时的默认值.

函数得分 = modifier(factor \* field.value).

GET /blogs/_search
    {
      "query": {
        "function_score": {
          "query": {
            "match": {
              "content": "rabbits"
            }
          },
          "functions": [
            {
              "field_value_factor": {
                "field": "author_id",
                "factor": 1.2,
                "modifier": "sqrt",
                "missing": 1
              }
            }
          ]
        }
      }
    }

c. Decay Functions

衰减函数有linear, exp和gauss三种(线性、指数和高斯函数), 我们以gauss为例.

*  origin  
    设置中心点, 落在中心点上的文档评分 \_score 为满分1.0
 *  scale  
    在此位置计算的分数将等于衰减参数.
 *  decay  
    从中心点 origin 衰减到 scale 所得的评分 \_score, 默认值为 0.5
 *  offset  
    在范围 -offset <= origin <= +offset 内的所有评分 \_score 都是 1.0

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdGlhbnhpYW5nX2thb2xh_size_16_color_FFFFFF_t_70 1]

> 具体计算公式看官网: https://www.elastic.co/guide/en/elasticsearch/reference/7.5/query-dsl-function-score-query.html\#function-weight

GET /blogs/_search
    {
      "query": {
        "function_score": {
          "query": {
            "match": {
              "content": "rabbits"
            }
          },
          "functions": [
            {
              "gauss": {
                "post_date": {
                  "origin": "2020-01-03",
                  "scale": "10d",
                  "offset": "5d",
                  "decay": 0.5
                }
              }
            }
          ]
        }
      }
    }

### Java API ###

下面介绍Elasticsearch Java Client 的使用, 我们来将上文的实例转化为 Java Client.

(1) main方法

public static void main(String[] args) throws IOException {
        RestHighLevelClient client = new RestHighLevelClient(
                RestClient.builder(
                        new HttpHost("localhost", 9200, "http")));
    
        bulkIndex(client);
    
        boost(client);
    
        boosting(client);
    
        functionScore(client);
    
        fieldValueFactor(client);
    
        decayFunction(client);
        
        client.close();
    }

新增文档和查询文档请求不要一起执行, 这样会查不到文档, 因为新增文档后需要1s时间进行倒排索引创建, 这也是ES被称为近实时的原因.

(2) 添加搜索数据

private static void bulkIndex(RestHighLevelClient client) throws IOException {
        BulkRequest bulkRequest = new BulkRequest();
    
        bulkRequest.add(new IndexRequest("blogs").id("1")
                .source(XContentType.JSON, "post_date", "2020-01-01", "title", "Quick brown rabbits", "content", "Brown rabbits are commonly seen.", "author_id", 11401));
        bulkRequest.add(new IndexRequest("blogs").id("2")
                .source(XContentType.JSON, "post_date", "2020-01-02", "title", "Keeping pets healthy", "content", "My quick brown fox eats rabbits on a regular basis.", "author_id", 11402));
        bulkRequest.add(new IndexRequest("blogs").id("3")
                .source(XContentType.JSON, "post_date", "2020-01-03", "title", "My dog barks", "content", "I see a lot of barking dogs on the road.", "author_id", 11403));
    
        client.bulk(bulkRequest, RequestOptions.DEFAULT);
    }

(3) boost

private static void boost(RestHighLevelClient client) throws IOException {
        SearchRequest searchRequest = new SearchRequest("blogs");
    
        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
        BoolQueryBuilder boolQueryBuilder = new BoolQueryBuilder();
        boolQueryBuilder.should(new MatchQueryBuilder("content", "rabbits").boost(2f));
        boolQueryBuilder.should(new MatchQueryBuilder("title", "brown").boost(0.1f));
    
        searchSourceBuilder.query(boolQueryBuilder);
        searchRequest.source(searchSourceBuilder);
    
        SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
        SearchHit[] hits = searchResponse.getHits().getHits();
        for (SearchHit hit : hits) {
            System.out.println(hit.getSourceAsString());
        }
    }

(4) boosting

private static void boosting(RestHighLevelClient client) throws IOException {
        SearchRequest searchRequest = new SearchRequest("blogs");
    
        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
    
        MatchQueryBuilder matchQueryBuilder = new MatchQueryBuilder("title", "rabbits");
        TermQueryBuilder termQueryBuilder = new TermQueryBuilder("post_date", "2020-01-02");
        BoostingQueryBuilder boostingQueryBuilder = new BoostingQueryBuilder(matchQueryBuilder, termQueryBuilder);
        boostingQueryBuilder.negativeBoost(0.2f);
    
        searchSourceBuilder.query(boostingQueryBuilder);
        searchRequest.source(searchSourceBuilder);
    
        SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
        SearchHit[] hits = searchResponse.getHits().getHits();
        for (SearchHit hit : hits) {
            System.out.println(hit.getSourceAsString());
        }
    }

(5) Weight和Random Score

private static void functionScore(RestHighLevelClient client) throws IOException {
        SearchRequest searchRequest = new SearchRequest("blogs");
    
        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
    
        //function1
        RandomScoreFunctionBuilder randomScoreFunctionBuilder = ScoreFunctionBuilders.randomFunction().seed(10).setField("_seq_no").setWeight(23);
        TermQueryBuilder termQueryBuilder1 = new TermQueryBuilder("post_date", "2020-01-01");
        FunctionScoreQueryBuilder.FilterFunctionBuilder filterFunctionBuilder = new FunctionScoreQueryBuilder.FilterFunctionBuilder(termQueryBuilder1, randomScoreFunctionBuilder);
    
        //function2
        TermQueryBuilder termQueryBuilder2 = new TermQueryBuilder("author_id", 11402);
        WeightBuilder weightBuilder = ScoreFunctionBuilders.weightFactorFunction(42);
        FunctionScoreQueryBuilder.FilterFunctionBuilder filterFunctionBuilder2 = new FunctionScoreQueryBuilder.FilterFunctionBuilder(termQueryBuilder2, weightBuilder);
    
        //query and functions
        MatchQueryBuilder matchQueryBuilder = new MatchQueryBuilder("content", "rabbits");
        FunctionScoreQueryBuilder.FilterFunctionBuilder[] filterFunctionBuilders = new FunctionScoreQueryBuilder.FilterFunctionBuilder[]{filterFunctionBuilder, filterFunctionBuilder2};
    
        FunctionScoreQueryBuilder functionScoreQueryBuilder = new FunctionScoreQueryBuilder(matchQueryBuilder, filterFunctionBuilders);
        functionScoreQueryBuilder.boost(5);
        functionScoreQueryBuilder.scoreMode(org.elasticsearch.common.lucene.search.function.FunctionScoreQuery.ScoreMode.MULTIPLY);
        functionScoreQueryBuilder.boostMode(CombineFunction.MULTIPLY);
        functionScoreQueryBuilder.setMinScore(42);
        functionScoreQueryBuilder.maxBoost(42);
    
        searchSourceBuilder.query(functionScoreQueryBuilder);
        searchRequest.source(searchSourceBuilder);
    
        SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
        SearchHit[] hits = searchResponse.getHits().getHits();
        for (SearchHit hit : hits) {
            System.out.println(hit.getSourceAsString());
        }
    }

(6) Field Value Factor

private static void fieldValueFactor(RestHighLevelClient client) throws IOException {
        SearchRequest searchRequest = new SearchRequest("blogs");
    
        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
    
        MatchQueryBuilder matchQueryBuilder = new MatchQueryBuilder("content", "rabbits");
        FieldValueFactorFunctionBuilder fieldValueFactorFunctionBuilder = ScoreFunctionBuilders.fieldValueFactorFunction("author_id").factor(1.2f).modifier(FieldValueFactorFunction.Modifier.SQRT).missing(1);
        FunctionScoreQueryBuilder functionScoreQueryBuilder = new FunctionScoreQueryBuilder(matchQueryBuilder, fieldValueFactorFunctionBuilder);
    
        searchSourceBuilder.query(functionScoreQueryBuilder);
        searchRequest.source(searchSourceBuilder);
    
        SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
        SearchHit[] hits = searchResponse.getHits().getHits();
        for (SearchHit hit : hits) {
            System.out.println(hit.getSourceAsString());
        }
    }

(7) Decay Functions

private static void decayFunction(RestHighLevelClient client) throws IOException {
        SearchRequest searchRequest = new SearchRequest("blogs");
    
        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
    
        MatchQueryBuilder matchQueryBuilder = new MatchQueryBuilder("content", "rabbits");
        GaussDecayFunctionBuilder gaussDecayFunctionBuilder = ScoreFunctionBuilders.gaussDecayFunction("post_date", "2020-01-03", "10d", "5d", 0.5);
        FunctionScoreQueryBuilder functionScoreQueryBuilder = new FunctionScoreQueryBuilder(matchQueryBuilder, gaussDecayFunctionBuilder);
    
        searchSourceBuilder.query(functionScoreQueryBuilder);
        searchRequest.source(searchSourceBuilder);
    
        SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
        SearchHit[] hits = searchResponse.getHits().getHits();
        for (SearchHit hit : hits) {
            System.out.println(hit.getSourceAsString());
        }
    }

[Elasticsearch_API]: https://blog.csdn.net/litianxiang_kaola/article/details/103804382
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdGlhbnhpYW5nX2thb2xh_size_16_color_FFFFFF_t_70]: https://img-blog.csdnimg.cn/20200102145725993.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdGlhbnhpYW5nX2thb2xh,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdGlhbnhpYW5nX2thb2xh_size_16_color_FFFFFF_t_70 1]: https://img-blog.csdnimg.cn/20191230103358520.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpdGlhbnhpYW5nX2thb2xh,size_16,color_FFFFFF,t_70