原英文版地址: https://www.elastic.co/guide/en/elasticsearch/reference/7.7/analysis-chargroup-tokenizer.html, 原文档版权归 www.elastic.co 所有
本地英文版地址: ../en/analysis-chargroup-tokenizer.html
本地英文版地址: ../en/analysis-chargroup-tokenizer.html
重要: 此版本不会发布额外的bug修复或文档更新。最新信息请参考 当前版本文档。
Char Group Tokenizeredit
The char_group tokenizer breaks text into terms whenever it encounters a
character which is in a defined set. It is mostly useful for cases where a simple
custom tokenization is desired, and the overhead of use of the pattern tokenizer
is not acceptable.
Configurationedit
The char_group tokenizer accepts one parameter:
|
|
A list containing a list of characters to tokenize the string on. Whenever a character
from this list is encountered, a new token is started. This accepts either single
characters like e.g. |
Example outputedit
POST _analyze
{
"tokenizer": {
"type": "char_group",
"tokenize_on_chars": [
"whitespace",
"-",
"\n"
]
},
"text": "The QUICK brown-fox"
}
returns
{
"tokens": [
{
"token": "The",
"start_offset": 0,
"end_offset": 3,
"type": "word",
"position": 0
},
{
"token": "QUICK",
"start_offset": 4,
"end_offset": 9,
"type": "word",
"position": 1
},
{
"token": "brown",
"start_offset": 10,
"end_offset": 15,
"type": "word",
"position": 2
},
{
"token": "fox",
"start_offset": 16,
"end_offset": 19,
"type": "word",
"position": 3
}
]
}