In my previous posts, I explained how to access MongoDB from Clojure and how to manage your configurations. In this post, we’ll use this knowledge to create a small webservice which supports the basic CRUD operations (create, update, delete) on a MongoDB database.

As first step, we create a new Leiningen project. To simplify the project setup, we are using the default Luminus template.

lein new luminus simple-mongo-webservice

Next, we add the dependency to Monger:

 [com.novemberain/monger "3.5.0"]

We could theoretically remove a few dependencies, that we do not necessarily need for our project, but for the sake of simplicity we’ll keep them all for now.

We’ll now build a very simple MongoDB data access layer with the following interface:

(defn create-document [document collection])
(defn get-document-by-id id collection)
(defn get-all collection)
(defn update-document-by-id id updated-document collection)
(defn delete-document-by-id id collection)

This layer may seem a bit unnecessary as they will not do much more than just call the Monger functions that perform these operations on the MongoDB, but we’ll include some error handling and return simpler return values than the plain responses from the Mongo Server.

First we will apply what we learned in my blog post about configurations in Clojure and configure our database connection.

The Luminus template created a file called config.clj with following content:

(ns simple-mongo-webservice.config
  (:require [cprop.core :refer [load-config]]
            [cprop.source :as source]
            [mount.core :refer [args defstate]]))

(defstate env
  :start
  (load-config
    :merge
    [(args)
     (source/from-system-props)
     (source/from-env)]))

This will read the config from following sources (later ones override previous ones):

  1. from classpath resource config.edn
  2. from file specified by system property conf
  3. from mount args
  4. from system properties
  5. from environment variables

Furthermore the file project.clj has following profiles section:

  :profiles
  {:uberjar {:omit-source true
             :aot :all
             :uberjar-name "simple-mongo-webservice.jar"
             :source-paths ["env/prod/clj"]
             :resource-paths ["env/prod/resources"]}

   :dev           [:project/dev :profiles/dev]
   :test          [:project/dev :project/test :profiles/test]

   :project/dev  {:jvm-opts ["-Dconf=dev-config.edn"]
                  :dependencies [[expound "0.7.2"]
                                 [pjstadig/humane-test-output "0.9.0"]
                                 [prone "1.6.1"]
                                 [ring/ring-devel "1.7.1"]
                                 [ring/ring-mock "0.3.2"]]
                  :plugins      [[com.jakemccrary/lein-test-refresh "0.23.0"]]
                  
                  :source-paths ["env/dev/clj"]
                  :resource-paths ["env/dev/resources"]
                  :repl-options {:init-ns user}
                  :injections [(require 'pjstadig.humane-test-output)
                               (pjstadig.humane-test-output/activate!)]}
   :project/test {:jvm-opts ["-Dconf=test-config.edn"]
                  :resource-paths ["env/test/resources"]}
   :profiles/dev {}
   :profiles/test {}})

This means that for profile dev we have following possibilities to put our configurations:

  • as classpath resource env/dev/resources/config.edn (already created by Luminus)
  • into the file specified by the conf parameter: def-config.edn (already created by Luminus)
  • from args from mount
  • via system properties (passed with -Dkey=value)
  • via environment variable (e.g. MONGO_HOST=192.168.0.1)

We need to create our Mongo connection object using these possibilities. For example, my development environment should use the Mongo DB instance running on localhost:27018 and should connect user “dev-admin” with password “S3cuRE!” on authentication database “dev”. Note that these are not default settings, so if you want to have the exact same setup, you’ll have to configure your Mongo DB instance accordingly (here is the official MongoDB docu on how to enable authorization, also change the port in the mongod.cfg as this is not the default port).

Here is how you can create the user (in the Mongo Console):

use dev
db.createUser(
  {
    user: "dev-admin",
    pwd: "S3cuRE!",
    roles: [ { role: "dbOwner", db: "dev" } ]
  }
)

This user can now create new users in the database “dev” and can create objects in any database.

To use this user for our connection object, we need to use the following function call:

(let [db "dev"
      user "admin-dev"
      pw "S3cuRE!"
      host "localhost"
      port 27018
      credentials (monger.credentials/create user db pw)]
  (mg/connect-with-credentials host port credentials))

Note that you can omit the port if you use default port 27017 or both host and port if you want to connect to localhost:27017.

So now that we now how to setup our dev connection, let’s write our configuration. We will write the credentials into the file dev-config.edn:

;; WARNING
;; The dev-config.edn file is used for local environment variables, such as database credentials.
;; This file is listed in .gitignore and will be excluded from version control by Git.

{:dev true
 :port 3000
 ;; when :nrepl-port is set the application starts the nREPL server on load
 :nrepl-port 7000

 :mongo {:port 27018
         :host "localhost"
         :credentials {:user "dev-admin" :auth-db "dev" :password "S3cuRE!"}
         :db "dev"}
 }

Only the mongo part is new, the rest is from the Luminus template and is fine. As the file is not checked in, you will not expose the credentials of your database in your repo. That’s why it is a good place to put them. If you want you can split the part that is okay to check in (host, port, db) into env/dev/resources/config.edn (classpath resource) and only store the credentials into dev-config.edn. Or you can put some useful defaults in the checked in classpath file (e.g. Mongo default localhost:27017 without authentication) and provide the connection settings you use on the machine.

Here is how the file env/dev/resources/config.edn looks like, if you provide defaults without credentials:

{:mongo {:port 27017
         :host "localhost"
         :db "dev"}
 }

Now someone who would check out the project can use Mongo DB how it is configured by default and I can use my custom port and custom settings for authorization.

Note that the config from dev-config.edn is not used in the Leiningen profile test (e.g. used when calling lein test). So don’t forget to configure your Mongo connection in test-config.edn as well when you write tests that need to connect to MongoDB! (Same for env/dev/resources/config.edn).

Now we still have to create our connection object, this we’ll do by creating a new file src/clj/simple_mongo_webservice/conn.clj with following content:

(ns simple-mongo-webservice.conn
  (:require [simple-mongo-webservice.config :refer [env]]
            [monger.core :as mg]
            [monger.credentials :as mcred]
            [mount.core :refer [defstate]]))

(defstate mongo-connection
  :start (let [mongo-cfg (:mongo env)
               host (:host mongo-cfg)
               port (:port mongo-cfg)]
           (if-let [cred-cfg (:credentials mongo-cfg)]
               (let [user (:user cred-cfg)
                     auth-db (:auth-db cred-cfg)
                     password (:password cred-cfg)
                     credentials (mcred/create user auth-db password)]
                 (mg/connect-with-credentials host port credentials))
               ;fallback: connect without credentials
               (mg/connect {:host host :port port}))

Now that we have this, we’ll create a small wrapper around Monger to have the default CRUD operations return informations that we don’t need to interpret anymore when we define our REST routes.
Let’s create a new file mongo-service.clj in src/clj/simple_mongo_webservice with following ns declaration:

(ns simple-mongo-webservice.mongo-service
  (:require [simple-mongo-webservice.conn :refer [mongo-connection]]
            [simple-mongo-webservice.config :refer [env]]
            [mount.core :refer [defstate]]
            [monger.core :as mg]
            [monger.collection :as mc])
  (:import org.bson.types.ObjectId))

We need our mongo config and our configuration (to retrieve the database name from). Apart from this we need Monger classes and ObjectId to create ObjectIds for new objects.

Then we create the database object:

(defstate db 
  :start (mg/get-db mongo-connection (get-in env [:mongo :db])))

Now we add the CRUD operations. First CREATE:

(defn create-document [document collection]
  (try 
    (-> document
        (merge {:_id (ObjectId.)}) 
        (#(mc/insert-and-return db collection %))
        :_id
        .toString)
    (catch Exception e nil)))

We pass a document and a collection name. We will always generate an ID ourself, so we don’t even bother to look if there is one in and just override it with a generated one. Finally, we return the ID as string (so that outside of the function, we don’t have to bother with ObjectId class). In case something goes wrong, we return nil (and would normally log the exception which I omit here).

Next we’ll add the function to read values from a collection, either only one (by id) or all. We always return the result with the id transformed to String (as we still don’t want to expose the ObjectId class).

(defn- transform-id-to-string [document]
  (if-let [id (:_id document)]
    (assoc document :_id (.toString id)))) 

(defn get-document-by-id [^String id  collection]
  (try
    (->> id
         ObjectId.
         (mc/find-map-by-id db collection)
         transform-id-to-string)
    (catch Exception e)))

(defn get-all [collection]
  (map transform-id-to-string (mc/find-maps db collection)))

Next, we update documents. The Monger function update-by-id returns a WriteResult object of which we will return the number of updated documents: 1 in case of success, 0 if id didn’t exist and -1 in case of exception (e.g. id is not a binary string of length 24)

(defn update-document-by-id [^String id document collection]
  (try
    (-> document
        (dissoc :_id)
        (#(mc/update-by-id db collection (ObjectId. id) %))
        .getN)
    (catch Exception e -1)))

Finally, we delete documents by id. Monger also returns a WriteRsult here, so we’ll do the same as with update result and return 1, 0 or -1:

(defn delete-document-by-id [^String id collection]
  (try
    (->> id
         ObjectId.
         (mc/remove-by-id db collection)
         .getN)
    (catch Exception e -1)))

Note that if you evaluate your code in the repl, you’ll have to call the function (mount.core/start) to get all your states initialized.

So that was the Mongo data access layer. Our final step is now to expose this layer via a RESTful web service.
To do that, we need to define our routes. We create a file src/clj/simple_mongo_webservice/routes/mongo.clj with following content:

(ns simple-mongo-webservice.routes.mongo
  (:require [simple-mongo-webservice.mongo-service :refer :all]
            [compojure.core :refer [defroutes GET POST PATCH DELETE context]]
            [ring.util.http-response :as response]
            [clojure.java.io :as io]))

(defroutes mongo-routes
  (context "/:collection" [collection]
    (GET "/" []
         {:status 200 :body (get-all collection)})
    (GET "/:id" [^String id]
         (if-let [document (get-document-by-id id collection)]
           {:status 200 :body document}
           {:status 404}))
    (POST "/" [:as request]
          (if-let [id (create-document (:body-params request) collection)]
            {:state 201 :body id}
            {:state 400}))
    (PATCH "/:id" [id :as request]
           (case (update-document-by-id id (:body-params request) collection)
             1 {:status 204}
             0 {:status 404}
             -1 {:status 400}))
    (DELETE "/:id" [id]
           (case (delete-document-by-id id collection)
             1 {:status 204}
             0 {:status 404}
             -1 {:status 400}))))

Now you can see why we build our small wrapper around Monger: in our routes code we don’t bother about exceptions and such things, but can directly return the response or the appropriate http response. No need to do big interpretations of return values here. This way, we have separated the interpretation of the object returned by the MongoDB driver (or Monger) and the construction of the REST response.

Now we only need to connect our routes with our handler so that we can call it via REST. For this, we need to change the file src/simple_mongo_webservice/handler.clj so that it has following content:

(ns simple-mongo-webservice.handler
  (:require [simple-mongo-webservice.middleware :as middleware]
            [simple-mongo-webservice.layout :refer [error-page]]
            [simple-mongo-webservice.routes.mongo :refer [mongo-routes]]
            [compojure.core :refer [routes wrap-routes]]
            [ring.util.http-response :as response]
            [compojure.route :as route]
            [simple-mongo-webservice.env :refer [defaults]]
            [mount.core :as mount]))

(mount/defstate init-app
  :start ((or (:init defaults) identity))
  :stop  ((or (:stop defaults) identity)))

(mount/defstate app
  :start
  (middleware/wrap-base
    (routes
      (-> #'mongo-routes
          (wrap-routes middleware/wrap-formats))
      (route/not-found
        (:body
          (error-page {:status 404
                       :title "page not found"}))))))

The wrap-base function puts some default middleware around all routes. You can look at the details in the file src/simple_mongo_webservice/middleware.clj. As we didn’t touch it, I won’t go into details here.
The wrap-formats middleware that we wrapped around our route will cause our requests to be decoded and encoded from/to transport formats e.g. json. So depending on the “Accept” and “Content-Type” headers in the http request, you can send and receive different formats. If you want to test if your REST service works, you can use [Postman][postman].

Run your project with

lein run

Now you know how to expose your Mongo collections via REST. Of course this service is very generic and the caller can alter any collection of the database as he wants. So for a real world application, you’ll probably restrict the service to a few collections and add some data validation functions before creating or updating some data.