本地英文版地址: ../en/analysis-limit-token-count-tokenfilter.html
Limit token count token filteredit
Limits the number of output tokens. The limit filter is commonly used to limit
the size of document field values based on token count.
By default, the limit filter keeps only the first token in a stream. For
example, the filter can change the token stream [ one, two, three ] to
[ one ].
This filter uses Lucene’s LimitTokenCountFilter.
If you want to limit the size of field values based on _character length_, use the <<ignore-above,`ignore_above`>> mapping parameter.
Configurable parametersedit
-
max_token_count -
(Optional, integer)
Maximum number of tokens to keep. Once this limit is reached, any remaining
tokens are excluded from the output. Defaults to
1. -
consume_all_tokens -
(Optional, boolean)
If
true, thelimitfilter exhausts the token stream, even if themax_token_counthas already been reached. Defaults tofalse.
Exampleedit
The following analyze API request uses the limit
filter to keep only the first two tokens in quick fox jumps over lazy dog:
GET _analyze
{
"tokenizer": "standard",
"filter": [
{
"type": "limit",
"max_token_count": 2
}
],
"text": "quick fox jumps over lazy dog"
}
The filter produces the following tokens:
[ quick, fox ]
Add to an analyzeredit
The following create index API request uses the
limit filter to configure a new
custom analyzer.
PUT limit_example
{
"settings": {
"analysis": {
"analyzer": {
"standard_one_token_limit": {
"tokenizer": "standard",
"filter": [ "limit" ]
}
}
}
}
}
Customizeedit
To customize the limit filter, duplicate it to create the basis
for a new custom token filter. You can modify the filter using its configurable
parameters.
For example, the following request creates a custom limit filter that keeps
only the first five tokens of a stream:
PUT custom_limit_example
{
"settings": {
"analysis": {
"analyzer": {
"whitespace_five_token_limit": {
"tokenizer": "whitespace",
"filter": [ "five_token_limit" ]
}
},
"filter": {
"five_token_limit": {
"type": "limit",
"max_token_count": 5
}
}
}
}
}