Using mongoTemplate with pipeline aggregation to deduplicate based on a certain condition

Oct 17, 202478

AI Translation

This post is translated from Chinese into English through AI.View Original

AI-generated summary

In daily operations, deduplication based on a specific database field can be achieved using the `mongotemplate.findDistinct` method. However, this method does not easily support additional functionalities like sorting, pagination, and counting. To better implement these business requirements, MongoDB's aggregation pipeline can be utilized. The provided code demonstrates how to combine filtering, deduplication, sorting, and pagination. It defines a method that sets up criteria for filtering, groups by a specified field to deduplicate, sorts the results, and paginates the output. Additionally, it includes a separate aggregation to count the total number of unique entries. The results are printed, showing the current page and the total count. The `User` class is defined with fields for `num`, `name`, and `age`, and the database is assumed to contain relevant data.

In daily business, you may encounter deduplication based on a certain field in the database. We can use the mongotemplate.findDistinct method. However, using this method does not match well with other business requirements such as sorting, pagination, and getting the total count. To better implement the business logic, we can use MongoDB's pipeline aggregation. Below is the code:

Condition + Deduplication + Sorting + Pagination

public void group() {
        int page = 1;  // Page number
        int size = 4;  // Page size

        Criteria criteria = Criteria.where("num").lte(10);
        Aggregation aggregation = Aggregation.newAggregation(
                Aggregation.match(criteria), // Filtering criteria
                Aggregation.group("age") // Deduplication based on specified field
                        .first("$$ROOT").as("document"), // Retain the entire document
                Aggregation.replaceRoot("document"), // Use the saved entire document object as the output document
                Aggregation.sort(Sort.by(Sort.Direction.ASC, "num")), // Sort based on specified field
                Aggregation.skip((page - 1) * size), // Skip the specified number of pages
                Aggregation.limit(size)
        );
        AggregationResults<User> results = mongoTemplate.aggregate(aggregation, "user", User.class);
        List<User> userList = results.getMappedResults();
        Aggregation countAgg = Aggregation.newAggregation(
                Aggregation.match(criteria),     // Filtering criteria
                Aggregation.group("age"),  // Deduplication based on specified field
                Aggregation.count().as("total") // Calculate the total count after deduplication
        );

        AggregationResults<org.bson.Document> countResults = mongoTemplate.aggregate(countAgg, "user", org.bson.Document.class);
        int total = 0;
        if (!countResults.getMappedResults().isEmpty()) {
            total = countResults.getMappedResults().get(0).getInteger("total");
        }

        System.out.println("Data retrieved");
        System.out.println("Currently on page " + page);
        userList.forEach(System.out::println);
        System.out.println("Total count: " + total);
    }

Entity Class

public class User {
    private Integer num;
    private String name;
    private Integer age;
}

Database Data

Execution Result