Mongo in Kubernetes with StatefulSets

From conventional point of view database can not be run in a container. Because containers are stateless and databases can not be stateless. However Kubernetes StatefulSet API object make that impossible happened. This post is based on this and this blog posts. I just copy pasted from those posts.

StorageClass

The storage class tells Kubernetes what kind of storage to use for the database nodes. You can set up many different types of StorageClasses in a ton of different environments. For example, if you run Kubernetes in your own datacenter, you can use GlusterFS. On GCP, your storage choices are SSDs and hard disks. There are currently drivers for AWS, Azure, Google Cloud, GlusterFS, OpenStack Cinder, vSphere, Ceph RBD, and Quobyte. The configuration for the StorageClass looks like this.

Headless Service

Now you have created the Storage Class, you need to make a Headless Service. These are just like normal Kubernetes Services, except they don’t do any load balancing for you. When combined with StatefulSets, they can give you unique DNS addresses that let you directly access the pods! This is perfect for creating MongoDB replica sets, because our app needs to connect to all of the MongoDB nodes individually. The configuration for the Headless Service looks like this. You can tell this is a Headless Service because the clusterIP is set to “None.” Other than that, it looks exactly the same as any normal Kubernetes Service.

StatefulSet

The StatefulSet actually runs MongoDB and orchestrates everything together. StatefulSets differ from Kubernetes ReplicaSets (not to be confused with MongoDB replica sets!) in certain ways that makes them more suited for stateful applications. Unlike Kubernetes ReplicaSets, pods created under a StatefulSet have a few unique attributes. The name of the pod is not random, instead each pod gets an ordinal name. Combined with the Headless Service, this allows pods to have stable identification. In addition, pods are created one at a time instead of all at once, which can help when bootstrapping a stateful system. You can read more about StatefulSets in the documentation. Just like before, this “sidecar” container will configure the MongoDB replica set automatically. A “sidecar” is a helper container which helps the main container do its work. The configuration for the StatefulSet looks like thisIt’s a little long, but fairly straightforward. The first second describes the StatefulSet object. Then, we move into the Metadata section, where you can specify labels and the number of replicas. Next comes the pod spec. The terminationGracePeriodSeconds is used to gracefully shutdown the pod when you scale down the number of replicas, which is important for databases! Then the configurations for the two containers is shown. The first one runs MongoDB with command line flags that configure the replica set name. It also mounts the persistent storage volume to /data/db, the location where MongoDB saves its data. The second container runs the sidecar. Finally, there is the volumeClaimTemplates. This is what talks to the StorageClass we created before to provision the volume. It will provision a 100 GB disk for each MongoDB replica.

Steps are simple:

1. create the Cluster
2. git clone https://github.com/thesandlord/mongo-k8s-sidecar.git
cd ./mongo-k8s-sidecar/example/StatefulSet/
3. Create the Storage Class by kubectl apply -f googlecloud_ssd.yaml
4. Deploying the Headless Service and StatefulSet bykubectl apply -f mongo-statefulset.yaml
5. Wait for the MongoDB Replica Set to be fully deployed; kubectl get statefulset
6. Initiating and Viewing the MongoDB replica set; kubectl get pods
7. connect to the first Replica Set membe by kubectl exec -ti mongo-0 mongo
8. Initiate the replica set with a default configuration by rs.initiate()
9. Print the replica set configuration by rs.conf()
10. scale up by kubectl scale --replicas=5 statefulset mongoand scale down by kubectl scale --replicas=3 statefulset mongo
11. To connect Mongo in code: Each pod in a StatefulSet backed by a Headless Service will have a stable DNS name. The template follows this format: pod-name.service-name This means the DNS names for the MongoDB replica set are: mongo-0.mongo, mongo-1.mongo, mongo-2.mongo You can use these names directly in the connection string URI of your app. In this case, the connection string URI would be:"mongodb://mongo-0.mongo,mongo-1.mongo,mongo-2.mongo:27017/dbname_?"
12. Save some data in the Mongo
rs0:PRIMARY> show dbs
admin 0.000GB
config 0.000GB
local 0.000GB
rs0:PRIMARY> use testdb
switched to db testdb
rs0:PRIMARY> db.sometable.insert({somefield:"somedata"})
WriteResult({ "nInserted" : 1 })
rs0:PRIMARY> db.sometable.find()
{ "_id" : ObjectId("5a7b71802a636a07aefe2132"), "somefield" : "somedata" }