WARNING: The 2.x versions of Elasticsearch have passed their EOL dates. If you are running a 2.x version, we strongly advise you to upgrade.
This documentation is no longer maintained and may be removed. For the latest information, see the current Elasticsearch documentation.
One Language per Fieldedit
For documents that represent entities like products, movies, or legal notices, it is common for the same text to be translated into several languages. Although each translation could be represented in a single document in an index per language, another reasonable approach is to keep all translations in the same document:
{ "title": "Fight club", "title_br": "Clube da Luta", "title_cz": "Klub rváčů", "title_en": "Fight club", "title_es": "El club de la lucha", ... }
Each translation is stored in a separate field, which is analyzed according to the language it contains:
PUT /movies { "mappings": { "movie": { "properties": { "title": { "type": "string" }, "title_br": { "type": "string", "analyzer": "brazilian" }, "title_cz": { "type": "string", "analyzer": "czech" }, "title_en": { "type": "string", "analyzer": "english" }, "title_es": { "type": "string", "analyzer": "spanish" } } } } }
The |
|
Each of the other fields uses the appropriate analyzer for that language. |
Like the index-per-language approach, the field-per-language approach
maintains clean term frequencies. It is not quite as flexible as having
separate indices. Although it is easy to add a new field by using the update-mapping
API, those new fields may require new
custom analyzers, which can only be set up at index creation time. As a
workaround, you can close the index, add the new
analyzers with the update-settings
API,
then reopen the index, but closing the index means that it will require some
downtime.
The documents of a single language can be queried independently, or queries can target multiple languages by querying multiple fields. We can even specify a preference for particular languages by boosting that field:
- Elasticsearch - The Definitive Guide:
- Foreword
- Preface
- Getting Started
- You Know, for Search…
- Installing and Running Elasticsearch
- Talking to Elasticsearch
- Document Oriented
- Finding Your Feet
- Indexing Employee Documents
- Retrieving a Document
- Search Lite
- Search with Query DSL
- More-Complicated Searches
- Full-Text Search
- Phrase Search
- Highlighting Our Searches
- Analytics
- Tutorial Conclusion
- Distributed Nature
- Next Steps
- Life Inside a Cluster
- Data In, Data Out
- What Is a Document?
- Document Metadata
- Indexing a Document
- Retrieving a Document
- Checking Whether a Document Exists
- Updating a Whole Document
- Creating a New Document
- Deleting a Document
- Dealing with Conflicts
- Optimistic Concurrency Control
- Partial Updates to Documents
- Retrieving Multiple Documents
- Cheaper in Bulk
- Distributed Document Store
- Searching—The Basic Tools
- Mapping and Analysis
- Full-Body Search
- Sorting and Relevance
- Distributed Search Execution
- Index Management
- Inside a Shard
- You Know, for Search…
- Search in Depth
- Structured Search
- Full-Text Search
- Multifield Search
- Proximity Matching
- Partial Matching
- Controlling Relevance
- Theory Behind Relevance Scoring
- Lucene’s Practical Scoring Function
- Query-Time Boosting
- Manipulating Relevance with Query Structure
- Not Quite Not
- Ignoring TF/IDF
- function_score Query
- Boosting by Popularity
- Boosting Filtered Subsets
- Random Scoring
- The Closer, The Better
- Understanding the price Clause
- Scoring with Scripts
- Pluggable Similarity Algorithms
- Changing Similarities
- Relevance Tuning Is the Last 10%
- Dealing with Human Language
- Aggregations
- Geolocation
- Modeling Your Data
- Administration, Monitoring, and Deployment