WARNING: The 2.x versions of Elasticsearch have passed their EOL dates. If you are running a 2.x version, we strongly advise you to upgrade.
This documentation is no longer maintained and may be removed. For the latest information, see the current Elasticsearch documentation.
Fuzzinessedit
Fuzzy matching treats two words that are “fuzzily” similar as if they were the same word. First, we need to define what we mean by fuzziness.
In 1965, Vladimir Levenshtein developed the Levenshtein distance, which measures the number of single-character edits required to transform one word into the other. He proposed three types of one-character edits:
- Substitution of one character for another: _f_ox → _b_ox
- Insertion of a new character: sic → sic_k_
- Deletion of a character: b_l_ack → back
Frederick Damerau later expanded these operations to include one more:
- Transposition of two adjacent characters: _st_ar → _ts_ar
For example, to convert the word bieber
into beaver
requires the
following steps:
-
Substitute
v
forb
: bie_b_er → bie_v_er -
Substitute
a
fori
: b_i_ever → b_a_ever -
Transpose
a
ande
: b_ae_ver → b_ea_ver
These three steps represent a Damerau-Levenshtein edit distance of 3.
Clearly, bieber
is a long way from beaver
—they are too far apart to be
considered a simple misspelling. Damerau observed that 80% of human
misspellings have an edit distance of 1. In other words, 80% of misspellings
could be corrected with a single edit to the original string.
Elasticsearch supports a maximum edit distance, specified with the fuzziness
parameter, of 2.
Of course, the impact that a single edit has on a string depends on the
length of the string. Two edits to the word hat
can produce mad
, so
allowing two edits on a string of length 3 is overkill. The fuzziness
parameter can be set to AUTO
, which results in the following maximum edit distances:
-
0
for strings of one or two characters -
1
for strings of three, four, or five characters -
2
for strings of more than five characters
Of course, you may find that an edit distance of 2
is still overkill, and
returns results that don’t appear to be related. You may get better results,
and better performance, with a maximum fuzziness
of 1
.
- Elasticsearch - The Definitive Guide:
- Foreword
- Preface
- Getting Started
- You Know, for Search…
- Installing and Running Elasticsearch
- Talking to Elasticsearch
- Document Oriented
- Finding Your Feet
- Indexing Employee Documents
- Retrieving a Document
- Search Lite
- Search with Query DSL
- More-Complicated Searches
- Full-Text Search
- Phrase Search
- Highlighting Our Searches
- Analytics
- Tutorial Conclusion
- Distributed Nature
- Next Steps
- Life Inside a Cluster
- Data In, Data Out
- What Is a Document?
- Document Metadata
- Indexing a Document
- Retrieving a Document
- Checking Whether a Document Exists
- Updating a Whole Document
- Creating a New Document
- Deleting a Document
- Dealing with Conflicts
- Optimistic Concurrency Control
- Partial Updates to Documents
- Retrieving Multiple Documents
- Cheaper in Bulk
- Distributed Document Store
- Searching—The Basic Tools
- Mapping and Analysis
- Full-Body Search
- Sorting and Relevance
- Distributed Search Execution
- Index Management
- Inside a Shard
- You Know, for Search…
- Search in Depth
- Structured Search
- Full-Text Search
- Multifield Search
- Proximity Matching
- Partial Matching
- Controlling Relevance
- Theory Behind Relevance Scoring
- Lucene’s Practical Scoring Function
- Query-Time Boosting
- Manipulating Relevance with Query Structure
- Not Quite Not
- Ignoring TF/IDF
- function_score Query
- Boosting by Popularity
- Boosting Filtered Subsets
- Random Scoring
- The Closer, The Better
- Understanding the price Clause
- Scoring with Scripts
- Pluggable Similarity Algorithms
- Changing Similarities
- Relevance Tuning Is the Last 10%
- Dealing with Human Language
- Aggregations
- Geolocation
- Modeling Your Data
- Administration, Monitoring, and Deployment