ElasticSearch Nested Mapping
Last updated on Nov 21, 2023
Mapping:
Mapping is used to describe the framework as well as field types depending on the responses to specific questions. As an example:
- Which string fields should be full text, and what should be numbers or dates (in what formats)?
- When should you use the _all field, which concatenates multiple fields into a single string to aid in analysis and indexing?
- What custom rules should be set up to automatically update new field types as they are added?
Now we will learn about the Elasticsearch Nested Mapping in depth.
Become a ElasticSearch Certified professional by learning ElasticSearch Certification Course from hkrtrainings!
Elasticsearch Nested Mapping:
ElasticSearch documents can have properties with arrays or other JSON objects as values. In most cases, this is sufficient. However, this is not always the case. Let's index a movie again, but this time we'll add an array of actors to it, with each actor being a JSON object:
Adding a ‘cast' property to a movie.
curl -XPUT “https://localhost:9200/movies/movie/7” -d’
{
“title”: “The Matrix”,
“cast”: [
{
“firstName”: “keerthi”,
“lastName”: “Ritesh”
},
{
“firstName”: “lawn”,
“lastName”: “fish”
}
]
}
Now that the movie has been indexed, we will get a hit for it if we search for movies starring an actor with the first name "keerthi." Unless we filter on a field mapped with the basic analyzer, we get the lowercase version, "keerthi."Searching for movies, filtering on the cast.firstName fields.
curl -XPOST “https://localhost:9200/movies/movie/_search” -d’
{
“query”: {
“filtered”: {
“query”: {
“match_all”: {}
},
“filter”: {
“term”: {
“cast.firstName”: “keerthi”
}
}
}
}
}
ElasticSearch Training
- Master Your Craft
- Lifetime LMS & Faculty Access
- 24/7 online expert support
- Real-world & Project Based Learning
Filtering on both the cast.firstName and cast.lastName fields.
curl -XPOST “https://localhost:9200/movies/movie/_search” -d’
{
“query”: {
“filtered”: {
“query”: {
“match_all”: {}
},
“filter”: {
“bool”: {
“must”: [
{
“term”: {
“cast.firstName”: “keerthi”
}
},
{
“term”: {
“cast.lastName”: “Ritesh”
}
}
]
}
}
}
}
}
The Matrix receives a hit as a result of the above request. Everything is fine. Is that correct? Let's see what happens if we search for movies starring an actor with the first name "keerthi" and the last name "fish."
Filtering on both cast fields again, but with a different value for cast.lastName this time.
curl -XPOST “https://localhost:9200/movies/movie/_search” -d’
{
“query”: {
“filtered”: {
“query”: {
“match_all”: {}
},
“filter”: {
“bool”: {
“must”: [
{
“term”: {
“cast.firstName”: “keerthi”
The nested type mapping 83
}
},
{
“term”: {
“cast.lastName”: “fish”
}
}
]
}
}
}
}
}
But on first brief look, the above request must not return The Matrix because there is no such actor between its cast. ElasticSearch, on the other hand, will return The Matrix for the above query. After all, the film features an actor with the first name "keerthi" and, albeit in a different role, an actor with the last name "fish." This has no means of knowing that if we want the two term filters to match the same unique object inside the list of actors based on the query above. Though it did, the way the information is indexed prevents it from meeting that requirement.
ElasticSearch loses the relationship of belonging to the same object when it indexes fields from JSON objects in an array. In other words, the document will include the following fields and values:
Field Value
Title “the” ” matrix”
cast.firstName “keerthi” “lawn”
Cast.last name “Ritesh” “fishburn”
We could prepare our index for this use case by adding a property to the actors that contains both the first and last name, as shown below:
{
“firstName”: “keerthi”,
“lastName”: “Ritesh”,
“fullName”: “keerthi Ritesh”
}
Subscribe to our YouTube channel to get new updates..!
By using method, we could simply filter on the fullName property to find only movies starring an actor named keerthi Ritesh. However, such a straightforward approach isn't always sufficient. Fortunately, ElasticSearch allows us to filter on multiple fields within the same objects in arrays; mapping such fields as the nested type. To test this, let's create a new index with the "actors" field mapped as nested.
Making a new index with the cast field nested.
curl -XPUT “https://localhost:9200/movies-2” -d’
{
“mappings”: {
“movie”: {
“properties”: {
“cast”: {
“type”: “nested”
}
}
}
}
}’
Upon indexing the same movie document into the new index, we can now use a nested filter to find movies based on multiple properties of each actor. Here's how we'd look for films starring the actor "keerthi fish":
To filter on both the cast, a nested filter is used.
firstName and role
In the cast field, there are lastName fields within the same objects.
curl -XPOST “https://localhost:9200/movies-2/movie/_search” -d’
{
“query”: {
“filtered”: {
“query”: {
“match_all”: {}
},
“filter”: {
“nested”: {
“path”: “cast”,
“filter”: {
“bool”: {
“must”: [
{
“term”: {
“firstName”: “keerthi”
}
},
{
“term”: {
“lastName”: “fish”
}
}
]
}
}
}
The nested type mapping 85
}
}
}
}’
We've covered our beginning bool filter in a nested filter, as you can see. The nested filter has a path property that specifies that the filter applies to the searched document's cast property. It also includes a filter (or query) that will be applied to each value contained within the nested property.
Running the above query as intended does not return The Matrix, but changing it to match “Ritesh” as the last name does return The Matrix. There is, however, one caveat.
Incorporating nested values within parent documents:
We won't get any hits if we return to our very first query, filtering only on actors' first names without using a nested filter, as shown in the request below.
curl -XPOST “https://localhost:9200/movies-2/movie/_search” -d’
{
“query”: {
“filtered”: {
“query”: {
“match_all”: {}
},
“filter”: {
“term”: {
“cast.firstName”: “keerthi”
}
}
}
}
}’
I'm looking for movies and narrowing it down based on the cast. firstName fields in the index, with the cast field nested.
curl -XPOST “https://localhost:9200/movies-2/movie/_search” -d’
{
“query”: {
“filtered”: {
“query”: {
“match_all”: {}
},
“filter”: {
“nested”: {
“path”: “cast”,
“filter”: {
“term”: {
“firstName”: “keerthi”
}
}
}
}
}
}
}’
The Matrix is returned by the request above. Even so, using nested filters or queries when everyone we need to do is filter on a single asset can be time consuming. To use the authority of nested filters for complicated criteria while being able to sort on values in arrays as if we had not even mapped such properties as nested, we could indeed adjust our mappings so the nested values were included in the parent document. This is accomplished by using the include in parent parameter, as shown below:
Making a new index with cast field mapped as nested and the include in parent property set to true.In an index created with the above request, we will be able to use nested filters to filter on pairings of values within the same complex objects in the actors array while also being able to filter on single fields without using nested filters.
Nevertheless, we must now consider carefully when and how to use, when to not use nested filters in our questions, as a query for “keerthi fish” will match The Matrix when wrapped in a regular bool filter but not when wrapped in a nested filter. In other words, if we forget to use nested filters when using include in parent, users might just get surprising consequences due to enquiring perfectly matched documents that they shouldn't.
Conclusion:
In the above blog post we had clearly explained about the elasticsearch nested mapping along with examples. If you have any doubts related to this topic,drop your queries in the comments section to get them answered quickly.
About Author
Upcoming ElasticSearch Training Online classes
| Batch starts on 21st Nov 2024 |
|
||
| Batch starts on 25th Nov 2024 |
|
||
| Batch starts on 29th Nov 2024 |
|