In this post you will learn how to use Google’s Cloud Launcher to set up instances for a MongoDB replica set in the Google Compute Engine.
Replication in MongoDB
A minimal MongoDB replica set consists of two data bearing nodes and one so-called arbiter that only takes part in the election of a new primary in case of failure.
Unlike other distributed databases, MongoDB does not offer auto-discovery of cluster nodes. If all nodes are up, you have to initialize the cluster via an administrative command inside the Mongo CLI that takes list of all replica set members. This fact makes it hard (but not impossible) to script this with the cloud provisioning tool of your choice.
Cloud Launcher for MongoDB
With the help of Google’s Cloud Launcher for MongoDB , the provisioning of a replica set is done in just a few steps.
First, we define a deployment name (that will prefix the names of the instances), a zone and the name of the replica set.
Then, we set up the instances that will run in the Compute Engine. We define a minimal replica set with two server instances and one arbiter. All other parameters will use the defaults for now (the server instances will use a n1-highmem-4 machine type ).
This will finally lead to three running instances.
Compute Engine Instances
The instances will show up in the Compute Engine dashboard where they can be managed:
If you prefer your CLI, you can list the instances with the gcloud tool:
1$ gcloud compute instances list 2NAME ZONE MACHINE_TYPE PREEMPTIBLE INTERNAL_IP EXTERNAL_IP STATUS 3mongo-0-arbiters-vm-0 europe-west3-b n1-standard-1 10.156.0.2 35.198.139.198 RUNNING 4mongo-0-servers-vm-0 europe-west3-b n1-highmem-4 10.156.0.4 35.198.158.94 RUNNING 5mongo-0-servers-vm-1 europe-west3-b n1-highmem-4 10.156.0.3 35.198.189.164 RUNNING
Replica Set Status
In order to check if everything is up and running, we open a SSH window to one of the instances from the dasboard and start the mongo CLI:
After connecting to the MongoDB, we run the rs.status() to get the status of the replica set:
1rs0:ARBITER>rs.status() 2{ 3 "set" : "rs0", 4 "date" : ISODate("2018-02-12T12:52:47.562Z"), 5 "myState" : 7, 6 ... 7 }, 8 "members" : [ 9 { 10 "_id" : 0, 11 "name" : "mongo-0-servers-vm-0:27017", 12 "health" : 1, 13 "state" : 1, 14 "stateStr" : "PRIMARY", 15 "uptime" : 787, 16 ... 17 }, 18 { 19 "_id" : 1, 20 "name" : "mongo-0-servers-vm-1:27017", 21 "health" : 1, 22 "state" : 2, 23 "stateStr" : "SECONDARY", 24 "uptime" : 787, 25 ... 26 }, 27 { 28 "_id" : 2, 29 "name" : "mongo-0-arbiters-vm-0:27017", 30 "health" : 1, 31 "state" : 7, 32 "stateStr" : "ARBITER", 33 "uptime" : 801, 34 "configVersion" : 3, 35 "self" : true 36 } 37 ], 38 "ok" : 1 39}
(I ommitted some of the output, so we can focus on the important things.)
We can see the name of the replica set rs0 and our three instances
- mongo-0-servers-vm-0
- mongo-0-servers-vm-1
- mongo-0-arbiters-vm-0
Restore Data Dump from Google Storage
We’ll restore a data dump from a file located in the Google Storage . Since only the the primary node can do write operations, we connect to the instance mongo-0-servers-vm-0 that has to role PRIMARY right now.
Inside that instance, we download a BSON dump from the storage by using the gsutil tool, assuming your storage bucket is called [BUCKET_ID]:
1mongo-0-servers-vm-1:/tmp$ cd /tmp 2mongo-0-servers-vm-1:/tmp$ gsutil cp gs://[BUCKET_ID].appspot.com/pois.* ./ 3Copying gs://[BUCKET_ID].appspot.com/pois.bson... 4Copying gs://[BUCKET_ID].appspot.com/pois.metadata.json... 5/ [2 files][ 5.6 KiB/ 5.6 KiB] 6Operation completed over 2 objects/5.6 KiB.
Now the dump is on our disk and we restore its data to the MongoDB replica set:
1mongo-0-servers-vm-1:/tmp$ mongorestore pois.bson 22018-02-12T13:12:33.402+0000 checking for collection data in pois.bson 32018-02-12T13:12:33.402+0000 reading metadata for test.pois from pois.metadata.json 42018-02-12T13:12:33.418+0000 restoring test.pois from pois.bson 52018-02-12T13:12:33.481+0000 restoring indexes for collection test.pois from metadata 62018-02-12T13:12:33.590+0000 finished restoring test.pois (7 documents) 72018-02-12T13:12:33.590+0000 done
Finally, we perfom a simple query from the OS command line:
1mongo-0-servers-vm-0:/tmp$ mongo --quiet --eval "db.pois.count()" test 27
to check if our 7 imported documents are really there.
Summary
We created a MongoDB replica set and deployed it to the Compute Engine. Then we checked the replica set status and imported some data.
In one of my next posts I will show you how to build a REST-ful microservice in the AppEngine that will access our MongoDB backend in the Compute Engine.
Was this post helpful?
More articles
fromTobias Trelle
17.12.2018 Your job at codecentric?
Jobs
Agile Developer und Consultant (w/d/m)
Alle Standorte
More articles in this subject area
Discover exciting further topics and let the codecentric world inspire you.
Gemeinsam bessere Projekte umsetzen.
Wir helfen deinem Unternehmen.
Du stehst vor einer großen IT-Herausforderung? Wir sorgen für eine maßgeschneiderte Unterstützung. Informiere dich jetzt.
Hilf uns, noch besser zu werden.
Wir sind immer auf der Suche nach neuen Talenten. Auch für dich ist die passende Stelle dabei.
Blog author
Tobias Trelle
Software Architect
Do you still have questions? Just send me a message.
Do you still have questions? Just send me a message.