5
Spring Boot Data MongoDB: Projections and Aggregations Examples | JavaProgramTo....
source link: https://www.javaprogramto.com/2020/05/spring-boot-data-mongodb-projections-aggregations.html
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
An in-depth guide to Projections and Aggregations Examples in Spring Boot MongoDB API with MongoTemplae and MongoRepository for the group(), match(), and sort() operation with Examples.
1. Introduction
In this tutorial, You'll learn how to use Spring Data MongoDB Projections and Aggregation operations with examples.
Actually, Spring Boot Data MongoDB provides a plain and simple high-level abstraction layer to work with Mongo native queries indirectly.
If you are new to Spring Boot MongoDB, then refer the previous article on "MongoDB CRUD Operations in Spring Boot"
First, We'll see what is Projection and a few examples on this.
Next, What is an aggregation along with some examples such as to do grouping, sort, and limit operations.
We are going to use the Employee document in this article, showing the data already present in the table.
Find More article on Spring Boot + MongoDB
Create MongoDBConfig class to register MongoDB client to the MongoTemplate.
You need to pass the collection name to MongoTemplate and this collection will be used for storing the Employee data.
We are going to use the Employee document in this article, showing the data already present in the table.
Find More article on Spring Boot + MongoDB
2. MongoDB + Spring Boot Data Dependencies
<dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-data-mongodb</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency>
3. Configuring MongoTemplate in Spring Boot Data
Let us create an Employee class as below with the following attributes.package com.javaprogramto.springboot.MongoDBSpringBootCURD.model;
import lombok.Getter;
import lombok.Setter;
import lombok.ToString;
import org.springframework.data.annotation.Id;
import org.springframework.data.mongodb.core.mapping.Document;
import java.util.Date;
@Getter
@Setter
@ToString
@Document(collection = "Employee")
public class Employee {
@Id
private int id;
private String name;
private int age;
private long phoneNumber;
private Date dateOfJoin;
}
Create MongoDBConfig class to register MongoDB client to the MongoTemplate.
You need to pass the collection name to MongoTemplate and this collection will be used for storing the Employee data.
package com.javaprogramto.springboot.MongoDBSpringBootCURD.config;
import com.mongodb.MongoClient;
import com.mongodb.MongoClientOptions;
import com.mongodb.MongoCredential;
import com.mongodb.ServerAddress;
import com.mongodb.client.MongoClients;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.mongodb.MongoDbFactory;
import org.springframework.data.mongodb.core.MongoTemplate;
import org.springframework.data.mongodb.core.SimpleMongoDbFactory;
import org.springframework.data.mongodb.repository.config.EnableMongoRepositories;
@Configuration
//@EnableMongoRepositories(basePackages = "com.javaprogramto.springboot.MongoDBSpringBootCURD.repository")
public class MongoDBConfig {
@Bean
public MongoClient mongo() throws Exception {
return new MongoClient("localhost");
}
@Bean
public MongoTemplate mongoTemplate() throws Exception {
return new MongoTemplate(mongo(), "JavaProgramTo");
}
}
4. Projection (MongoDB + Spring Boot Data) Examples
In MongoDB, the Table is called a Document which stores the actual values in the collection.
By default, if you fetch the records from the mongo DB document then it will fetch all the columns. In most of the cases, you don't need all the columns but unnecessarily getting not related to the columns.
If you can limit the no of columns using some technique then you can see the area to improve the performance and memory utilization.
Because of these reasons, MongoDB comes up with "Projections" concept the way to fetch only the required or set of columns from a Document.
Obviously, It reduces the amount of data to be transferred between the database and the client. Hence, Performance will be improved wisely.
4.1 MongoTemplate Projections
package com.javaprogramto.springboot.MongoDBSpringBootCURD.controller;
import com.javaprogramto.springboot.MongoDBSpringBootCURD.model.Employee;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.data.mongodb.core.MongoTemplate;
import org.springframework.data.mongodb.core.query.Query;
import org.springframework.web.bind.annotation.*;
import java.util.List;
@RestController
@RequestMapping("/projection")
public class MongoTemplateProjectionController {
@Autowired
private MongoTemplate mongoTemplate;
@GetMapping("/allname")
public List<Employee> getOnlyName() {
Query query = new Query();
query.fields().include("name");
List<Employee> list = mongoTemplate.find(query, Employee.class);
return list;
}
@GetMapping("/allnameage")
public List<Employee> getOnlyNameAge() {
Query query = new Query();
query.fields().include("name").include("age").exclude("id");
List<Employee> list = mongoTemplate.find(query, Employee.class);
return list;
}
@GetMapping("/excludename")
public List<Employee> excludeName() {
Query query = new Query();
query.fields().exclude("name");
List<Employee> list = mongoTemplate.find(query, Employee.class);
return list;
}
@GetMapping("/excludeId")
public List<Employee> excludeId() {
Query query = new Query();
query.fields().exclude("id");
List<Employee> list = mongoTemplate.find(query, Employee.class);
return list;
}
}
In the above controller, We have @Autowired the MongoTemplate and call find() method by passing the Query() object.
Fields of include() and exclude() methods can be used to include or exclude columns from the Document.
The column annotated with @Id annotation will be fetched always unless excluded by using exclude() method.
Note: You can not use include(), exclude() methods at the same time. The combination these two methods will end up in the run time exception saying "MongoQueryException".
If you use exclude() method twice or more then that only will throw the error. But, if you once exclude() with multiple inclusions will not cause any error.
Fields of include() and exclude() methods can be used to include or exclude columns from the Document.
The column annotated with @Id annotation will be fetched always unless excluded by using exclude() method.
Note: You can not use include(), exclude() methods at the same time. The combination these two methods will end up in the run time exception saying "MongoQueryException".
If you use exclude() method twice or more then that only will throw the error. But, if you once exclude() with multiple inclusions will not cause any error.
Query query = new Query();
query.fields().include("name").include("age").exclude("id").exclude("phoneNumber");
List<Employee> list = mongoTemplate.find(query, Employee.class);
com.mongodb.MongoQueryException: Query failed with error code 2 and error message 'Projection cannot have a mix of inclusion and exclusion.' on server localhost:27017
at com.mongodb.operation.FindOperation$1.call(FindOperation.java:735) ~[mongodb-driver-core-3.11.2.jar:na]
at com.mongodb.operation.FindOperation$1.call(FindOperation.java:725) ~[mongodb-driver-core-3.11.2.jar:na]
at com.mongodb.operation.OperationHelper.withReadConnectionSource(OperationHelper.java:463) ~[mongodb-driver-core-3.11.2.jar:na]
at com.mongodb.operation.FindOperation.execute(FindOperation.java:725) ~[mongodb-driver-core-3.11.2.jar:na]
at com.mongodb.operation.FindOperation.execute(FindOperation.java:89) ~[mongodb-driver-core-3.11.2.jar:na]
Internally Field class uses criteria map to store the exclude and include column names as key. and value will be 1 for include and 0 will be for excluding.
Even though columns excluded with exclude() method and those properties will be present in the response json but it will have initialized with default values.
For wrapper types value will be null and for primitive types will its default values.
Such as "age" is a type of int so once excluded then it will be assigned with value 0. For boolean, it will be false.
@Query annotation will take values as "{}" because it not taking any filter so all the records will be fetched and values attribute is mandatory.
fields attribute is used for projections, 1 indicates include field, and 0 indicates to exclude the field.
For wrapper types value will be null and for primitive types will its default values.
Such as "age" is a type of int so once excluded then it will be assigned with value 0. For boolean, it will be false.
4.2 MongoRepository Projection
If you are using MongoRepository then you should use @Query annotation on the method level.@Query annotation will take values as "{}" because it not taking any filter so all the records will be fetched and values attribute is mandatory.
fields attribute is used for projections, 1 indicates include field, and 0 indicates to exclude the field.
package com.javaprogramto.springboot.MongoDBSpringBootCURD.repository;
import com.javaprogramto.springboot.MongoDBSpringBootCURD.model.Employee;
import org.springframework.data.mongodb.repository.MongoRepository;
import org.springframework.data.mongodb.repository.Query;
import org.springframework.stereotype.Component;
import java.util.List;
@Component
public interface EmployeeRepository extends MongoRepository<Employee, Integer> {
// Getting only name and excluding id field.
@Query(value = "{}", fields = "{name : 1,_id : 0}")
public List<Employee> findNameAndExcludeId();
// getting only name age fields but id will be fetched automatically because it is annotated with @Id.
@Query(value = "{}", fields = "{name : 1, age : 1}")
public List<Employee> nameAndAge();
// Fetches only Id.
@Query(value = "{}", fields = "{id : 1}")
public List<Employee> findOnlyIds();
// Fetches only id and age.
@Query(value = "{}", fields = "{id : 1, age : 1}")
public List<Employee> findByIdAge();
}
MonoRepository Controller:
package com.javaprogramto.springboot.MongoDBSpringBootCURD.controller.projection;
import com.javaprogramto.springboot.MongoDBSpringBootCURD.model.Employee;
import com.javaprogramto.springboot.MongoDBSpringBootCURD.repository.EmployeeRepository;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.*;
import java.util.List;
@RestController
@RequestMapping("/mongorepo/projection")
public class EmployeeMongoRepoController {
Logger logger = LoggerFactory.getLogger(getClass());
@Autowired
private EmployeeRepository employeeRepository;
@GetMapping("/nameExcludeId")
public List<Employee> getAll() {
List<Employee> list = employeeRepository.findNameAndExcludeId();
return list;
}
@GetMapping("/nameage")
public List<Employee> getNameAge() {
List<Employee> list = employeeRepository.nameAndAge();
return list;
}
@GetMapping("/idage")
public List<Employee> getIdAge() {
List<Employee> list = employeeRepository.findByIdAge();
return list;
}
@GetMapping("/ids")
public List<Employee> getOnlyIds() {
List<Employee> list = employeeRepository.findOnlyIds();
return list;
}
}
Output:
If you have only excluded column in the field attribute then all remaining columns will be added to the included list automatically.
Repository:
// Fetches only id and age.
@Query(value = "{}", fields = "{id : 0}")
public List<Employee> excludeId();
Controller:
@GetMapping("/excludeid")
public List<Employee> excludeId() {
List<Employee> list = employeeRepository.excludeId();
return list;
}
Output:
5. Aggregations (MongoDB + Spring Boot Data) Examples
MongoDB is built to perform the aggregation operations to simplify the process of the bulk data set.Basically, the Data set is processed in multiple steps, and the output of stage one is passed to the next step as input.
Along with the stages, you can add many transformations and filters as needed.
It supports grouping, filtering based on criteria or matching, And also provides the sorting at the end to see in the desired manner.
Input Employee class
package com.javaprogramto.springboot.MongoDBSpringBootCURD.model;
import lombok.Getter;
import lombok.Setter;
import lombok.ToString;
import org.springframework.data.annotation.Id;
import org.springframework.data.mongodb.core.mapping.Document;
import java.util.Date;
@Getter
@Setter
@ToString
@Document(collection = "Employee")
public class Employee {
@Id
private int id;
private String name;
private int age;
private long phoneNumber;
private Date dateOfJoin;
}
Output Type Class
We store the result in the below class after finding the count based on grouping the age.
package com.javaprogramto.springboot.MongoDBSpringBootCURD.model;
public class AgeCount {
private int id;
private int count;
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public int getCount() {
return count;
}
public void setCount(int count) {
this.count = count;
}
}
Aggregation Group, Match, Sort Example
The below example to1. group by age
2. Match count > 1
3. Sort the age
All these are executed sequentially.
package com.javaprogramto.springboot.MongoDBSpringBootCURD.controller.projection;
import com.javaprogramto.springboot.MongoDBSpringBootCURD.model.AgeCount;
import com.javaprogramto.springboot.MongoDBSpringBootCURD.model.Employee;
import com.javaprogramto.springboot.MongoDBSpringBootCURD.repository.EmployeeRepository;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.data.domain.Sort;
import org.springframework.data.mongodb.core.MongoTemplate;
import org.springframework.data.mongodb.core.aggregation.*;
import org.springframework.data.mongodb.core.query.Criteria;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
@RestController
@RequestMapping("/aggregation")
public class AggregationController {
@Autowired
private MongoTemplate mongoTemplate;
@GetMapping("/group")
public List<AgeCount> groupByAge() {
// grouping by age.
GroupOperation groupOperation = Aggregation.group("age").count().as("count");
// filtering same age count > 1
MatchOperation matchOperation = Aggregation.match(new Criteria("count").gt(1));
SortOperation sortOperation = Aggregation.sort(Sort.by(Sort.Direction.ASC, "count"));
Aggregation aggregation = Aggregation.newAggregation(groupOperation, matchOperation, sortOperation);
AggregationResults<AgeCount> result = mongoTemplate.aggregate(aggregation, "Employee", AgeCount.class);
return result.getMappedResults();
}
}
Aggregation class has lots of static methods to perform aggregation operations such as group(), sort(), skip(), match(), limit() operations.
In the above Aggregation example, First created three aggregation operations such as GroupOperation for grouping, MatchOperation for filtering records based on a condition, and last SortOperation for sorting in ASC or DESC order based a property.
Finally, At last, we need to pass all these operations to the newAggregation() method which will return Aggregation instance, and here the order is important how all these should be executed. The output of the grouping operation is supplied to the match operation and the next output of the Matching Operation to the Sorting operation.
Additionally, you need to make another call to execute this aggregation set of operations by using mongoTemplate.aggregate() method.
mongoTemplate.aggregate() method takes aggretation which is created with newAggretation() method, collection name as "Employee" and output type as AgeCount class.
As a result, mongoTemplate.aggregate() method returns a instnace of AggregationResults<AgeCount> and get the output list of AgeCount objects by invoking the result.getMappedResults().
Let us hit the endpoint now to see the output to get the same age count employees which is greater than 1.
localhost:9000/aggregation/group
[
{
"id": 25,
"count": 2 },
{
"id": 32,
"count": 2 },
{
"id": 35,
"count": 2 },
{
"id": 39,
"count": 2 },
{
"id": 42,
"count": 4 }
]
If you want to get the "age" in place of "id" property then you should change the property in the AgeCount model.
package com.javaprogramto.springboot.MongoDBSpringBootCURD.model;
public class AgeCount {
private int age;
private int count;
public int getAge() {
return age;
}
public void setAge(int age) {
this.age = age;
}
public int getCount() {
return count;
}
public void setCount(int count) {
this.count = count;
}
}
Modified Method
@GetMapping("/group")
public List<AgeCount> groupByAge() {
MatchOperation empIdgt10 = Aggregation.match(Criteria.where("_id").gt(10));
// grouping by age.
GroupOperation groupOperation = Aggregation.group("age").count().as("count");
ProjectionOperation projectionOperation = Aggregation.project("count").and("age").previousOperation();
// filtering same age count > 1
MatchOperation matchOperation = Aggregation.match(new Criteria("count").gt(1));
SortOperation sortOperation = Aggregation.sort(Sort.by(Sort.Direction.ASC, "count", "age"));
Aggregation aggregation = Aggregation.newAggregation(empIdgt10, groupOperation, projectionOperation, matchOperation, sortOperation);
AggregationResults<AgeCount> list = mongoTemplate.aggregate(aggregation, "Employee", AgeCount.class);
return list.getMappedResults();
}
Output:
Now id is replaced by age field.
[
{
"age": 25,
"count": 2 },
{
"age": 32,
"count": 2 },
{
"age": 35,
"count": 2 },
{
"age": 39,
"count": 2 },
{
"age": 42,
"count": 4 }
]
A simplified version of Grouping and Sorting Aggregation
The below produces the same result but we have taken output type is String.@GetMapping("/groupbysimples")
public List<String> groupByAge2() {
Aggregation agg = newAggregation(
match(Criteria.where("_id").gt(10)),
group("age").count().as("total"),
project("total").and("age").previousOperation(),
sort(Sort.Direction.ASC, "total")
);
//Convert the aggregation result into a List
AggregationResults<String> groupResults = mongoTemplate.aggregate(agg, Employee.class, String.class);
List<String> result = groupResults.getMappedResults();
return result;
}
Output:
localhost:9000/aggregation/groupbysimples
JSON fields are taken count as total and order of properties are different but the count is the same.
[
"{\"total\": 1, \"age\": 40}",
"{\"total\": 2, \"age\": 25}",
"{\"total\": 2, \"age\": 35}",
"{\"total\": 2, \"age\": 32}",
"{\"total\": 2, \"age\": 39}",
"{\"total\": 4, \"age\": 42}"
]
6. Conclusion
In this article, You've seen in-depth knowledge of MongoDB Projections and Aggregations in Spring Boot applications.Projections are the new way to fetch only required columns from the document which reduces the size of the data to be transferred and hence results in improving the response time to the client.
Projections are done using MongoTemplate or MongoRepository but MongoTemplate will through run time errors if you exclusion() method multiple times and MongoReposity will not produce any errors even if you any number exclusions in @Query(fields={}) annotation.
Aggretations can be done with MongoTemplate.aggregate() method by providing the various Aggregate Operations to perform match(), limit(), group() and sort() operations.
All are shown with the examples.
All the code is shown in this article is over GitHub.
You can download the project directly and can run in your local without any errors.
If you have any queries please post in the comment section.
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK