In this article, you’ll learn how to build a CRUD RESTful API server with Golang, Gin Gonic, MongoDB-Go-driver, Docker, and Docker-compose.
CRUD RESTful API with Golang + MongoDB Series:
- API with Golang + MongoDB + Redis + Gin Gonic: Project Setup
- Golang & MongoDB: JWT Authentication and Authorization
- API with Golang + MongoDB: Send HTML Emails with Gomail
- API with Golang, Gin Gonic & MongoDB: Forget/Reset Password
- Build Golang gRPC Server and Client: SignUp User & Verify Email
- Build Golang gRPC Server and Client: Access & Refresh Tokens
- Build CRUD RESTful API Server with Golang, Gin, and MongoDB
Golang, Gin Gonic, MongoDB CRUD RESTful API Overview
We’ll build a RESTful API where we can create, read, update and delete a post in the MongoDB database.
| RESOURCE | HTTP METHOD | ROUTE | DESCRIPTION |
|---|---|---|---|
| posts | GET | /api/posts | Retrieve all posts |
| posts | POST | /api/posts | Creates a new post |
| posts | GET | /api/posts/:postId | Retrieve a single post |
| posts | PATCH | /api/posts/:postId | Update a post |
| posts | DELETE | /api/posts/:postId | Delete a post |
Am going to use Postman to test the Golang API endpoints. You can use any API testing tool you are comfortable with.
-To create a new post, you need to make a POST request to /api/posts
-Update a single post by making a PATCH request to /api/posts/:postId
-Retrieve a single post by making a GET request to /api/posts/:postId
-Delete a post by making a DELETE request to /api/posts/:postId
-Retrieve all posts by making a GET request to /api/posts?page=1&limit=10
-You should see all the posts you created in the MongoDB database.
Create the Models with Structs
The most obvious way to begin is to create the Golang structs to represent the models we’ll be working with. These structs need to have the BSON annotation tags.
models/post.model.go
package models
import (
"time"
"go.mongodb.org/mongo-driver/bson/primitive"
)
type CreatePostRequest struct {
Title string `json:"title" bson:"title" binding:"required"`
Content string `json:"content" bson:"content" binding:"required"`
Image string `json:"image,omitempty" bson:"image,omitempty"`
User string `json:"user" bson:"user" binding:"required"`
CreateAt time.Time `json:"created_at,omitempty" bson:"created_at,omitempty"`
UpdatedAt time.Time `json:"updated_at,omitempty" bson:"updated_at,omitempty"`
}
type DBPost struct {
Id primitive.ObjectID `json:"id,omitempty" bson:"_id,omitempty"`
Title string `json:"title,omitempty" bson:"title,omitempty"`
Content string `json:"content,omitempty" bson:"content,omitempty"`
Image string `json:"image,omitempty" bson:"image,omitempty"`
User string `json:"user,omitempty" bson:"user,omitempty"`
CreateAt time.Time `json:"created_at,omitempty" bson:"created_at,omitempty"`
UpdatedAt time.Time `json:"updated_at,omitempty" bson:"updated_at,omitempty"`
}
type UpdatePost struct {
Id primitive.ObjectID `json:"id,omitempty" bson:"_id,omitempty"`
Title string `json:"title,omitempty" bson:"title,omitempty"`
Content string `json:"content,omitempty" bson:"content,omitempty"`
Image string `json:"image,omitempty" bson:"image,omitempty"`
User string `json:"user,omitempty" bson:"user,omitempty"`
CreateAt time.Time `json:"created_at,omitempty" bson:"created_at,omitempty"`
UpdatedAt time.Time `json:"updated_at,omitempty" bson:"updated_at,omitempty"`
}
In some of the structs, we provided the Id fields with a primitive.ObjectID type. This will inform the MongoDB server that these fields will be actual MongoDB ObjectIds.
In addition, the _id bson fields of the Structs will be filled with unique ObjectIds by the MongoDB Server.
You will notice that in the CreatePostRequest struct, we omitted the Id field. That is because when inserting a new document into the MongoDB database, we don’t need to provide the Id field since the MongoDB server will automatically generate and assign a unique ObjectId to the document.
Also, in the CreatePostRequest struct, we added the required annotation to some of the fields to help Gin Gonic validate the request body against the provided struct and return the appropriate validation errors if any of the required fields were not provided.
Finally, we used the time.Time data type for the created_at and updated_at fields. This will tell the MongoDB server that these fields should have the bson timestamp data type.
Create the Service Interface
One of the RESTful API architectures is to separate the business from the application logic.
We always push most of the business logic to the models or services so that we can end up with fat models or services and thin controllers.
To begin, let’s define an interface to list the methods we must implement to perform the basic CRUD operations.
services/post.service.go
type PostService interface {
CreatePost(*models.CreatePostRequest) (*models.DBPost, error)
UpdatePost(string, *models.UpdatePost) (*models.DBPost, error)
FindPostById(string) (*models.DBPost, error)
FindPosts(page int, limit int) ([]*models.DBPost, error)
DeletePost(string) error
}
Create Methods to Implement the Interface
Next, let’s create a struct to help us implement the PostService interface we defined above.
We’re going to need two key things – the MongoDB collection struct to help us interact with the MongoDB server and the context interface to perform some background cleanup.
Initialize the Service Struct
Since Golang does not support inheritance, we’ll use a popular programming paradigm called composition to inherit the features of the MongoDB collection struct and the context interface.
services/post.service.impl.go
type PostServiceImpl struct {
postCollection *mongo.Collection
ctx context.Context
}
func NewPostService(postCollection *mongo.Collection, ctx context.Context) PostService {
return &PostServiceImpl{postCollection, ctx}
}
Define a Service to Create a Post
Now let’s create a service to insert the new post into the MongoDB database. To do that, we’ll use a receiver function also known as a method to set the CreatePost method on the PostServiceImpl struct.
The MongoDB InsertOne() method accepts a context and an interface which in this case is the CreatePostRequest struct. The MongoDB InsertOne() function will automatically convert the struct into BSON using the tags we provided.
services/post.service.impl.go
func (p *PostServiceImpl) CreatePost(post *models.CreatePostRequest) (*models.DBPost, error) {
post.CreateAt = time.Now()
post.UpdatedAt = post.CreateAt
res, err := p.postCollection.InsertOne(p.ctx, post)
if err != nil {
if er, ok := err.(mongo.WriteException); ok && er.WriteErrors[0].Code == 11000 {
return nil, errors.New("post with that title already exists")
}
return nil, err
}
opt := options.Index()
opt.SetUnique(true)
index := mongo.IndexModel{Keys: bson.M{"title": 1}, Options: opt}
if _, err := p.postCollection.Indexes().CreateOne(p.ctx, index); err != nil {
return nil, errors.New("could not create index for title")
}
var newPost *models.DBPost
query := bson.M{"_id": res.InsertedID}
if err = p.postCollection.FindOne(p.ctx, query).Decode(&newPost); err != nil {
return nil, err
}
return newPost, nil
}
In the above code, I added the createdAt and updatedAt timestamps to the post before inserting it into the database.
Also, I added a unique constraint on the title field so that no two posts will have the same title.
Next, I used the ObjectId returned by the InsertOne() method to retrieve the newly created document.
Define a Service to Update Post
To perform an update operation, we use the MongoDB $set operator. If you will remember, we added an omitempty annotation to all the fields in the UpdatePost struct to help us update only the fields provided in the request body.
Another important thing is that the MongoDB FindOneAndUpdate() method returns the document as it appeared before updating, so we need to explicitly instruct it to return the newly updated document.
services/post.service.impl.go
func (p *PostServiceImpl) UpdatePost(id string, data *models.UpdatePost) (*models.DBPost, error) {
doc, err := utils.ToDoc(data)
if err != nil {
return nil, err
}
obId, _ := primitive.ObjectIDFromHex(id)
query := bson.D{{Key: "_id", Value: obId}}
update := bson.D{{Key: "$set", Value: doc}}
res := p.postCollection.FindOneAndUpdate(p.ctx, query, update, options.FindOneAndUpdate().SetReturnDocument(1))
var updatedPost *models.DBPost
if err := res.Decode(&updatedPost); err != nil {
return nil, errors.New("no post with that Id exists")
}
return updatedPost, nil
}
Define a Service to Delete Post
Now let’s perform the simplest CRUD operation. To delete a document in the collection, we use the MongoDB DeleteOne() or FindOneAndDelete() methods.
The FindOneAndDelete() method returns the document as it appeared before deletion whilst the DeleteOne() method returns a DeletedCount value.
services/post.service.impl.go
func (p *PostServiceImpl) DeletePost(id string) error {
obId, _ := primitive.ObjectIDFromHex(id)
query := bson.M{"_id": obId}
res, err := p.postCollection.DeleteOne(p.ctx, query)
if err != nil {
return err
}
if res.DeletedCount == 0 {
return errors.New("no document with that Id exists")
}
return nil
}
Define a Service to Get Single Post
Next, let’s define a service to return a single document from the collection. The MongoDB FindOne() method accepts a context and a filter. The filter can be an unordered or ordered BSON document with key/value pairs.
In addition, we need to decode the found document to a regular Golang struct.
services/post.service.impl.go
func (p *PostServiceImpl) FindPostById(id string) (*models.DBPost, error) {
obId, _ := primitive.ObjectIDFromHex(id)
query := bson.M{"_id": obId}
var post *models.DBPost
if err := p.postCollection.FindOne(p.ctx, query).Decode(&post); err != nil {
if err == mongo.ErrNoDocuments {
return nil, errors.New("no document with that Id exists")
}
return nil, err
}
return post, nil
}
Define a Service to Get All Posts
The last CRUD operation is a little challenging so pay attention. MongoDB also has a cursor feature which is simply a pointer to the results of a query rather than all the results in memory.
With the help of a for loop and the cursor.Next() method, we’ll iterate through the cursor to retrieve the documents and append each document to a defined slice.
Also, remember to close the cursor with the cursor.Close() method before returning from the FindPosts() function.
services/post.service.impl.go
func (p *PostServiceImpl) FindPosts(page int, limit int) ([]*models.DBPost, error) {
if page == 0 {
page = 1
}
if limit == 0 {
limit = 10
}
skip := (page - 1) * limit
opt := options.FindOptions{}
opt.SetLimit(int64(limit))
opt.SetSkip(int64(skip))
query := bson.M{}
cursor, err := p.postCollection.Find(p.ctx, query, &opt)
if err != nil {
return nil, err
}
defer cursor.Close(p.ctx)
var posts []*models.DBPost
for cursor.Next(p.ctx) {
post := &models.DBPost{}
err := cursor.Decode(post)
if err != nil {
return nil, err
}
posts = append(posts, post)
}
if err := cursor.Err(); err != nil {
return nil, err
}
if len(posts) == 0 {
return []*models.DBPost{}, nil
}
return posts, nil
}
Create Controllers to Perform the CRUD Operations
Once the services are ready, it’s time to perform the actual CRUD operations by calling the appropriate service to access and mutate the database.
Initialize the Controller Struct
Since we are dealing with the PostService, let’s use composition to inherit all the methods we defined above in our own struct.
controllers/post.controller.go
type PostController struct {
postService services.PostService
}
func NewPostController(postService services.PostService) PostController {
return PostController{postService}
}
Define a Controller to Create a Post
To validate the request body, Gin Gonic provides us with ShouldBindJSON() method that accepts an interface which in this case is the CreatePostRequest struct and returns a validation error to the user based on the rules specified in the binding tag.
With the validation out of the way, we can now call the CreatePost() service we defined above to add the new post to the MongoDB database.
controllers/post.controller.go
func (pc *PostController) CreatePost(ctx *gin.Context) {
var post *models.CreatePostRequest
if err := ctx.ShouldBindJSON(&post); err != nil {
ctx.JSON(http.StatusBadRequest, err.Error())
return
}
newPost, err := pc.postService.CreatePost(post)
if err != nil {
if strings.Contains(err.Error(), "title already exists") {
ctx.JSON(http.StatusConflict, gin.H{"status": "fail", "message": err.Error()})
return
}
ctx.JSON(http.StatusBadGateway, gin.H{"status": "fail", "message": err.Error()})
return
}
ctx.JSON(http.StatusCreated, gin.H{"status": "success", "data": newPost})
}
Define a Controller to Update a Post
Updating a post is nearly similar to creating a post, with the addition of the postId parameter, since we need to tell the MongoDB server what document to update.
To update a post, we first need to extract the postId from the request parameter and pass it along with the request body to the UpdatePost service.
controllers/post.controller.go
func (pc *PostController) UpdatePost(ctx *gin.Context) {
postId := ctx.Param("postId")
var post *models.UpdatePost
if err := ctx.ShouldBindJSON(&post); err != nil {
ctx.JSON(http.StatusBadGateway, gin.H{"status": "fail", "message": err.Error()})
return
}
updatedPost, err := pc.postService.UpdatePost(postId, post)
if err != nil {
if strings.Contains(err.Error(), "Id exists") {
ctx.JSON(http.StatusNotFound, gin.H{"status": "fail", "message": err.Error()})
return
}
ctx.JSON(http.StatusBadGateway, gin.H{"status": "fail", "message": err.Error()})
return
}
ctx.JSON(http.StatusOK, gin.H{"status": "success", "data": updatedPost})
}
Define a Controller to Delete a Post
Am always happy when I reach the delete operation since it’s the simplest of all the CRUD operations.
To delete a document from the collection, you need to retrieve the postId from the request parameter and pass it to the DeletePost() service.
controllers/post.controller.go
func (pc *PostController) DeletePost(ctx *gin.Context) {
postId := ctx.Param("postId")
err := pc.postService.DeletePost(postId)
if err != nil {
if strings.Contains(err.Error(), "Id exists") {
ctx.JSON(http.StatusNotFound, gin.H{"status": "fail", "message": err.Error()})
return
}
ctx.JSON(http.StatusBadGateway, gin.H{"status": "fail", "message": err.Error()})
return
}
ctx.JSON(http.StatusNoContent, nil)
}
Define a Controller to Get a Single Post
The read CRUD operation follows the same strategy to delete a post. Here, you provide the FindPostById() service with the postId to retrieve the post that matches that Id.
controllers/post.controller.go
func (pc *PostController) FindPostById(ctx *gin.Context) {
postId := ctx.Param("postId")
post, err := pc.postService.FindPostById(postId)
if err != nil {
if strings.Contains(err.Error(), "Id exists") {
ctx.JSON(http.StatusNotFound, gin.H{"status": "fail", "message": err.Error()})
return
}
ctx.JSON(http.StatusBadGateway, gin.H{"status": "fail", "message": err.Error()})
return
}
ctx.JSON(http.StatusOK, gin.H{"status": "success", "data": post})
}
Define a Controller to Get All Posts
If we want to retrieve all the posts, the JSON payload might be too large if we have a lot of posts in the database.
To fix this, I added a pagination feature to allow the user to specify the number of documents to skip and the number of documents to return.
controllers/post.controller.go
func (pc *PostController) FindPosts(ctx *gin.Context) {
var page = ctx.DefaultQuery("page", "1")
var limit = ctx.DefaultQuery("limit", "10")
intPage, err := strconv.Atoi(page)
if err != nil {
ctx.JSON(http.StatusBadGateway, gin.H{"status": "fail", "message": err.Error()})
return
}
intLimit, err := strconv.Atoi(limit)
if err != nil {
ctx.JSON(http.StatusBadGateway, gin.H{"status": "fail", "message": err.Error()})
return
}
posts, err := pc.postService.FindPosts(intPage, intLimit)
if err != nil {
ctx.JSON(http.StatusBadGateway, gin.H{"status": "fail", "message": err.Error()})
return
}
ctx.JSON(http.StatusOK, gin.H{"status": "success", "results": len(posts), "data": posts})
}
Create the Routes for the Controllers
Now let’s use the power of composition to inherit all the methods of the PostController struct we defined above.
Next, let’s create all the CRUD routes to call the controllers.
routes/post.routes.go
type PostRouteController struct {
postController controllers.PostController
}
func NewPostControllerRoute(postController controllers.PostController) PostRouteController {
return PostRouteController{postController}
}
func (r *PostRouteController) PostRoute(rg *gin.RouterGroup) {
router := rg.Group("/posts")
router.GET("/", r.postController.FindPosts)
router.GET("/:postId", r.postController.FindPostById)
router.POST("/", r.postController.CreatePost)
router.PATCH("/:postId", r.postController.UpdatePost)
router.DELETE("/:postId", r.postController.DeletePost)
}
Initialize the Constructors in the Main file
The final piece of the puzzle is to evoke all the constructors we defined above. I left some comments in the code snippets below to help you.
Focus only on the Gin server.
cmd/server/main.go
package main
import (
"context"
"fmt"
"log"
"net"
"net/http"
"github.com/gin-contrib/cors"
"github.com/gin-gonic/gin"
"github.com/go-redis/redis/v8"
"github.com/wpcodevo/golang-mongodb/config"
"github.com/wpcodevo/golang-mongodb/controllers"
"github.com/wpcodevo/golang-mongodb/gapi"
"github.com/wpcodevo/golang-mongodb/pb"
"github.com/wpcodevo/golang-mongodb/routes"
"github.com/wpcodevo/golang-mongodb/services"
"go.mongodb.org/mongo-driver/mongo"
"go.mongodb.org/mongo-driver/mongo/options"
"go.mongodb.org/mongo-driver/mongo/readpref"
"google.golang.org/grpc"
"google.golang.org/grpc/reflection"
)
var (
server *gin.Engine
ctx context.Context
mongoclient *mongo.Client
redisclient *redis.Client
userService services.UserService
UserController controllers.UserController
UserRouteController routes.UserRouteController
authCollection *mongo.Collection
authService services.AuthService
AuthController controllers.AuthController
AuthRouteController routes.AuthRouteController
// ? Add the Post Service, Controllers and Routes
postService services.PostService
PostController controllers.PostController
postCollection *mongo.Collection
PostRouteController routes.PostRouteController
)
func init() {
config, err := config.LoadConfig(".")
if err != nil {
log.Fatal("Could not load environment variables", err)
}
ctx = context.TODO()
// Connect to MongoDB
mongoconn := options.Client().ApplyURI(config.DBUri)
mongoclient, err := mongo.Connect(ctx, mongoconn)
if err != nil {
panic(err)
}
if err := mongoclient.Ping(ctx, readpref.Primary()); err != nil {
panic(err)
}
fmt.Println("MongoDB successfully connected...")
// Connect to Redis
redisclient = redis.NewClient(&redis.Options{
Addr: config.RedisUri,
})
if _, err := redisclient.Ping(ctx).Result(); err != nil {
panic(err)
}
err = redisclient.Set(ctx, "test", "Welcome to Golang with Redis and MongoDB", 0).Err()
if err != nil {
panic(err)
}
fmt.Println("Redis client connected successfully...")
// Collections
authCollection = mongoclient.Database("golang_mongodb").Collection("users")
userService = services.NewUserServiceImpl(authCollection, ctx)
authService = services.NewAuthService(authCollection, ctx)
AuthController = controllers.NewAuthController(authService, userService, ctx, authCollection)
AuthRouteController = routes.NewAuthRouteController(AuthController)
UserController = controllers.NewUserController(userService)
UserRouteController = routes.NewRouteUserController(UserController)
// ? Add the Post Service, Controllers and Routes
postCollection = mongoclient.Database("golang_mongodb").Collection("posts")
postService = services.NewPostService(postCollection, ctx)
PostController = controllers.NewPostController(postService)
PostRouteController = routes.NewPostControllerRoute(PostController)
server = gin.Default()
}
func main() {
config, err := config.LoadConfig(".")
if err != nil {
log.Fatal("Could not load config", err)
}
defer mongoclient.Disconnect(ctx)
startGinServer(config)
// startGrpcServer(config)
}
func startGrpcServer(config config.Config) {
authServer, err := gapi.NewGrpcAuthServer(config, authService, userService, authCollection)
if err != nil {
log.Fatal("cannot create grpc authServer: ", err)
}
userServer, err := gapi.NewGrpcUserServer(config, userService, authCollection)
if err != nil {
log.Fatal("cannot create grpc userServer: ", err)
}
grpcServer := grpc.NewServer()
pb.RegisterAuthServiceServer(grpcServer, authServer)
pb.RegisterUserServiceServer(grpcServer, userServer)
reflection.Register(grpcServer)
listener, err := net.Listen("tcp", config.GrpcServerAddress)
if err != nil {
log.Fatal("cannot create grpc server: ", err)
}
log.Printf("start gRPC server on %s", listener.Addr().String())
err = grpcServer.Serve(listener)
if err != nil {
log.Fatal("cannot create grpc server: ", err)
}
}
func startGinServer(config config.Config) {
value, err := redisclient.Get(ctx, "test").Result()
if err == redis.Nil {
fmt.Println("key: test does not exist")
} else if err != nil {
panic(err)
}
corsConfig := cors.DefaultConfig()
corsConfig.AllowOrigins = []string{config.Origin}
corsConfig.AllowCredentials = true
server.Use(cors.New(corsConfig))
router := server.Group("/api")
router.GET("/healthchecker", func(ctx *gin.Context) {
ctx.JSON(http.StatusOK, gin.H{"status": "success", "message": value})
})
AuthRouteController.AuthRoute(router, userService)
UserRouteController.UserRoute(router, userService)
// ? Evoke the PostRoute
PostRouteController.PostRoute(router)
log.Fatal(server.Run(":" + config.Port))
}
Conclusion
Congrats for reaching the end. In this comprehensive article, you learned how to build a CRUD API server with Golang, MongoDB-Go-driver, Gin Gonic, Docker, and Docker-compose.
Check out the source code on GitHub
brilliant tutorial..it seems users don’t need to be authorized to create a post ?
That’s correct. You can require users to be authorized before creating a post by adding the
middleware.DeserializeUser(userService)middleware to the pipeline of each route within the post router inside theroutes/post.routes.gofile.