Codebook

Appearance

第 203 / 250 章中间件⏱ 12 分钟阅读

第 203 章:ES 高级搜索与优化 ​

学习目标

  • 掌握复杂查询(Query DSL)
  • 学会相关性打分优化
  • 理解分片与索引优化
  • 学会 ES 集群监控

一、复合查询 ​

1.1 multi_match(多字段) ​

bash
POST /articles/_search
{
  "query": {
    "multi_match": {
      "query": "NestJS 教程",
      "fields": ["title^3", "content", "tags^2"],
      "type": "best_fields"
    }
  }
}

^3 = 字段权重 3 倍。

类型:

  • best_fields:取单个字段最高分
  • most_fields:多个字段分数相加
  • cross_fields:跨字段匹配
  • phrase:短语匹配
  • phrase_prefix:短语前缀

1.2 短语匹配 ​

bash
POST /articles/_search
{
  "query": {
    "match_phrase": {
      "title": "NestJS 教程"
    }
  }
}

slop 允许间隔:

bash
{
  "match_phrase": {
    "title": { "query": "NestJS 教程", "slop": 2 }
  }
}

1.3 prefix / wildcard / fuzzy ​

bash
# 前缀
{ "prefix": { "title": "nest" } }

# 通配符(* 多个, ? 一个)
{ "wildcard": { "title": "nes*" } }

# 模糊(容错)
{ "fuzzy": { "title": { "value": "nedtjs", "fuzziness": 2 } } }

1.4 function_score(自定义打分) ​

bash
POST /articles/_search
{
  "query": {
    "function_score": {
      "query": { "match": { "title": "NestJS" } },
      "functions": [
        {
          "filter": { "term": { "author": "Tom" } },
          "weight": 2
        },
        {
          "field_value_factor": {
            "field": "viewCount",
            "factor": 0.1,
            "modifier": "log1p",
            "missing": 0
          }
        }
      ],
      "score_mode": "sum",
      "boost_mode": "multiply"
    }
  }
}

二、深分页 ​

2.1 search_after ​

bash
# 第一页
POST /articles/_search
{
  "size": 10,
  "sort": [
    { "createdAt": "desc" },
    { "_id": "asc" }
  ]
}

# 后续页(用上次的 sort 值)
POST /articles/_search
{
  "size": 10,
  "sort": [
    { "createdAt": "desc" },
    { "_id": "asc" }
  ],
  "search_after": [1700000000000, "abc123"]
}

2.2 PIT(Point In Time) ​

bash
# 创建 PIT
POST /articles/_pit?keep_alive=1m
# 返回 {"id": "..."}

# 用 PIT 搜索
POST /_search
{
  "pit": { "id": "...", "keep_alive": "1m" },
  "size": 10,
  "sort": [{ "createdAt": "desc" }]
}

三、相关性打分(BM25) ​

3.1 评分要素 ​

python
_score(q, d) = boost * TF * IDF * fieldLength
# TF:词频
# IDF:逆文档频率
# fieldLength:字段越短分数越高

3.2 explain 调试 ​

bash
GET /articles/_explain/1
{
  "query": { "match": { "title": "NestJS" } }
}

四、聚合深入 ​

4.1 nested 聚合 ​

bash
{
  "aggs": {
    "by_comment": {
      "nested": { "path": "comments" },
      "aggs": {
        "top_users": {
          "terms": { "field": "comments.user", "size": 5 }
        }
      }
    }
  }
}

4.2 pipeline 聚合 ​

bash
{
  "aggs": {
    "sales": {
      "date_histogram": { "field": "date", "calendar_interval": "month" },
      "aggs": {
        "monthly_sales": { "sum": { "field": "amount" } }
      }
    },
    "total": {
      "sum_bucket": {
        "buckets_path": "sales>monthly_sales"
      }
    }
  }
}

五、Mapping 优化 ​

5.1 字段类型选型 ​

类型场景例子
text全文搜索title, content
keyword精确 / 聚合author, tags, status
integer整数age, viewCount
date日期createdAt
boolean布尔isActive
nested嵌套数组comments
geo_point经纬度location

5.2 禁止 _source ​

json
{
  "mappings": {
    "_source": { "enabled": false }
  }
}

节省存储,但不能 reindex / update。

5.3 字段动态模板 ​

json
{
  "mappings": {
    "dynamic_templates": [
      {
        "strings_as_keyword": {
          "match_mapping_type": "string",
          "mapping": { "type": "keyword", "ignore_above": 256 }
        }
      }
    ]
  }
}

六、索引优化 ​

6.1 分片设计 ​

单分片: 10-50GB
分片数 = 期望吞吐 / 单分片吞吐
节点数 = 分片数 * (副本数 + 1)

6.2 Routing 减少搜索范围 ​

bash
PUT /orders/_doc/1?routing=user:123

查询时:

bash
POST /orders/_search?routing=user:123
{
  "query": { "match_all": {} }
}

6.3 Index Aliases(零停机切换) ​

bash
# 创建 v1 索引
PUT /articles_v1

# 创建别名
POST /_aliases
{
  "actions": [
    { "add": { "index": "articles_v1", "alias": "articles" } }
  ]
}

# 创建 v2 索引
PUT /articles_v2

# 切换别名(零停机)
POST /_aliases
{
  "actions": [
    { "remove": { "index": "articles_v1", "alias": "articles" } },
    { "add": { "index": "articles_v2", "alias": "articles" } }
  ]
}

七、性能优化 ​

7.1 查询优化 ​

优化说明
避免 wildcard 前缀*foo 会扫全索引
用 keyword 不分词聚合 / 排序
限制 size深分页用 search_after
filter 不算分不用算分的放 filter
关闭 _source 字段只取需要的字段

7.2 索引优化 ​

bash
# 强制合并(减少 segment)
POST /articles/_forcemerge?max_num_segments=1

# 关闭不必要的索引
"index.refresh_interval": "30s"   # 默认 1s,批量导入可设 30s

# 索引重建
POST /_reindex
{
  "source": { "index": "articles_v1" },
  "dest": { "index": "articles_v2" }
}

7.3 JVM 调优 ​

bash
# heap 不超过物理内存 50%,最大 32GB
ES_JAVA_OPTS="-Xms16g -Xmx16g"

# 关闭 swap
swapoff -a

# 内存锁定
bootstrap.memory_lock=true

八、ES 监控 ​

8.1 关键指标 ​

指标含义告警阈值
cluster_status集群状态黄色>1h
jvm_heap_used_percentJVM 堆使用率>80%
search_rate查询速率-
index_rate写入速率-
query_latency查询延迟>1s

8.2 慢查询 ​

yaml
# elasticsearch.yml
index.search.slowlog.threshold.query.warn: 10s
index.search.slowlog.threshold.fetch.warn: 1s

8.3 Profile API ​

bash
POST /articles/_search
{
  "profile": true,
  "query": { "match": { "title": "NestJS" } }
}

返回每个阶段耗时。

九、ELK 日志聚合 ​

yaml
# logstash.conf
input {
  beats { port => 5044 }
}

filter {
  grok {
    match => { "message" => "%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:level} %{GREEDYDATA:msg}" }
  }
  date {
    match => [ "timestamp", "ISO8601" ]
  }
}

output {
  elasticsearch {
    hosts => ["localhost:9200"]
    index => "app-logs-%{+YYYY.MM.dd}"
  }
}

十、本章小结 ​

优化收益
合理分片提高并行度
routing减少搜索范围
alias零停机切换
filter context跳过打分
force merge减少 segment
关闭 _source节省存储

动手练习 ​

  1. 实现 multi_match + 打分权重
  2. 用 search_after 实现深分页
  3. 用聚合统计作者文章数 + 阅读量
  4. 用 IK 分词器优化中文搜索

推荐阅读 ​

下一章:第 204 章:MongoDB 入门与建模 →

本站基于 VitePress 构建 · 由 Codebook 团队维护

Copyright © 2026-present Codebook