Docker + Mongo + Windows

My laptop runs on windows 7 and so I used the following link to install docker-toolbox.

https://docs.docker.com/toolbox/toolbox_install_windows/

My laptop is enabled with virtualization. I validated this by using the tool provided by Microsoft. Link to the tool is provided in the docker installation steps.

Windows 7 depends on VirtualBox to achieve the virtualization and it will be installed along with docker components. As I have installed the virtualbox already in my machine, I skipped it.

Docker quick start terminal is like a terminal in Linux and this is the one will be used for doing all the docker commands.

A virtual machine called default is created in Virtualbox which acts as a base for creating docker containers.

Once the installation is complete, I pulled and created containers for hello-world and ubuntu images. It worked as expected.

Now I pulled and created the container for Postgresql with 5432 port mapping with the host. Port is mapped fine and I was able to access the postgres using pgAdmin.

Later I pulled a mongo image and created a container with port mapping but my surprise I was not able to access mongodb from my host using RoboMongo.

I deleted and recreated the containers but it didn’t work out. So next option is google.

I searched and found the following link which solved the problem.

http://codehangar.io/mongodb-image-instance-with-docker-toolbox-tutorial/

Port mapping has to be done with the host after creating the container. I do not know why this didn’t happen with Postgres. I will find out and update the blog.

Update:

Create mongodb container with replica set enabled,

docker run -p 27017:27017 –name fmdb2 mongod –replSet fmreplica —noprealloc —smallfiles

Advertisement

Privacy Settings

GC Flags

Prints all available GC flags:

-XX:+PrintFlagsFinal

Don’t JIT huge methods: 

Methods that have more than 2500 lines are considered are huge method.

Impact: Long GC Pauses

-XX:-DontCompileHugeMethods

CMS Steps:

1. STW Initial Mark
2. Concurrent Mark
3. Concurrent Preclean
4. STW Remark
5. Concurrent Sweep
6. Concurrent Reset

CMS Flags:

-XX:+UseConcMarkSweepGC

Default on: -XX:+CMSParallelRemarkEnabled

Default on: -XX:+UseParNewGC

Default on: -XX:+CMSConcurrentMTEnabled

-XX:CMSInitiatingOccupancyFraction=70

-XX+UseCMSInitiatingOccupancyOnly

-XX:+CMSClassUnloadingEnabled

(< 1.8) -XX:+CMSPermGenSweepingEnabled

-XX:+ExplicitGCInvokesConcurrent or  -XX:+ExplicitGCInvokesConcurrentAndUnloadsClasses

Trigger minor collection before Remark Phase in CMS:(off by default)

Impact is Remark phase might be longer. Remember Remark phase is Stop The World.

-XX:+CMSScavengeBeforeRemark

Ref: http://blog.sokolenko.me/2014/11/javavm-options-production.html

Rename the project in Bitbucket

Let’s say the existing project name as MyTestApp

I want to change the app name to sandbox-app

Let us see how to rename a project in bitbucket and as well as in local.

Bitbucket:

• Login to bitbucket and traverse to your project and go to settings
• Change the name from MyTestApp to sandbox-app
• Save the changes

That’s it at bitbucket.

Local:

• Go to project location. cd /home/loganathan/git/MyTestApp
• Go to .git directory. cd .git
• Open config file in vim. vim config
• Change the name to sandbox-app in the url
• Save and exit the file
• Come 2 directories back, cd ../../
• Change directory name from MyTestApp to sandbox-app. mv MyTestApp sandbox-app
• Go to sandbox-app
• Open .project file in vim and change name from MyTestApp to sandbox-app. Save and exit.
• Open pom.xml in vim and change artifact and name to crawler-app. Save and exit.

Making docker to listen on network

The default installation of docker-engine is accessed only from the local machine where it is installed.

If we want to make the docker to listen over the network which helps in working with docker remotely without logging into the machine where docker is installed.

Below is the simplest way to achieve it but this is not a secured way. We have tweak it further to make this secure,

Docker configuration file with its location: /etc/default/docker

By adding -H option with IP as 0.0.0.0 in the config file, we can make docker listen over the network.

Add DOCKER_OPTS="-H tcp://0.0.0.0:2375"

Save the file and restart the docker.

You can verify in browser, hit the URL in browser: http://192.168.1.11:2375/_ping

OK is the response returned if the setup is successful.

Here 192.168.1.11 is IP of docker installed the machine.

Now you can run docker commands remotely:

Ex:

wagonsoft@Lenovo-G50-80:~$ sudo docker -H 5.44.98.132 ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
cd12fff3cf09 ubuntu “bash” 4 hours ago Exited (0) 3 hours ago adoring_jang
e2695f761862 hello-world “/hello” 4 hours ago Exited (0) 2 minutes ago agitated_allen

Otherwise, we can try: chrome addon Simple Docker UI

Now docker will listen only on TCP port, so we have to use -H option even in the docker installed machine.

In order to make docker work without -H option in local, it has to listen on default unix socket.

To enable this add following in DOCKER_OPTS in /etc/default/docker,

-H unix:///var/run/docker.sock

Now DOCKER_OPTS looks like,

DOCKER_OPTS="-H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock"

Save the file and restart the docker,

service docker restart

Now we will be able to run docker without -H option in local.

Reference: https://docs.docker.com/v1.10/engine/reference/commandline/daemon/

My docker commands

Start docker process:

service docker start

Docker process status:

service docker status

Stop docker process:

service docker stop

List all images:

docker images

Create a container from an image:

Interactive Mode:

docker run -it <image name>:<tag>

<tag> is optional, latest tag will be taken if not provided

With tag: docker run -it elasticsearch:latest
Without tag: docker run -it elasticsearch

Detached Mode:

docker run -d <image name>:<tag>

With tag: docker run -d elasticsearch:latest
Without tag: docker run -d elasticsearch

Expose port with the host:

docker run -p <host_port>:<container_port> <image name>

docker run -d -p 9200:9200 -p 9300:9300 elasticsearch

List all containers:

docker ps -a

List only running containers:

docker ps

Start a docker container:

docker start <image id/name>

docker start 42e175dd9630

Stop a docker container:

docker stop <container id/name>

docker stop 42e175dd9630

Cloning a container:

docker commit <existing_container_id/name> <new_container_name>

docker commit elasticsearch-latest elastic-logz

Execute Command:

docker exec <docker container id/name> <command>

docker exec 42e175dd9630 ls -ltrh

Execute Command interactive:

docker exec -it <docker container id/name> <command>

Get terminal access: docker exec -it 42e175dd9630 bash

Get bash as root user:

docker exec -it –user root <container id/name> bash

docker exec -it --user root 42e175dd9630 bash

 

Useful Linux commands

Change default shell of the user:

sudo chsh -s /bin/bash <username>

Create a user with home directory:

sudo useradd -m loganathan

Add a user to sudo list:

sudo usermod -aG sudo loganathan

Add a user to a group:

sudo usermod -aG docker loganathan

 

Rotate mongodb logs without restarting mongodb

MongoDB provides varies ways to rotate log files, but most of them are configurations you have to do and restart MongoDB.

One more way provided by MongoDB to rotate the log without any configuration change or restart. It is a command and works in Linux/Unix like OS.

kill -SIGUSR1 <mongo process id>

This will rotate the current log by appending the UTC time and also creates new mongodb.log.

You will be having 2 files in /var/log/mongo, one mongo.log which is current log and another one mongo.log.2016-04-07T11-17-18

 

 

Extracting keys and values from json string using Gson

package com.logztechstuff.json.gson;

import java.util.Iterator;
import java.util.Map;

import com.google.gson.JsonArray;
import com.google.gson.JsonElement;
import com.google.gson.JsonObject;
import com.google.gson.JsonParser;

public class GsonParser {
public static void main(String[] args) {

        String json = "{ name : \"foo\", company : \"bar\", ph : [9912345667, 7744554287]}";
        JsonParser parser = new JsonParser();
        JsonObject jsonObject = (JsonObject) parser.parse(json);
        //Check the parsed string is json or not
        if(jsonObject.isJsonObject()) {
            System.out.println("Yes it is json object");
            //Extracting keys and values from json
            for (Map.Entry<String, JsonElement> entry : jsonObject.entrySet()) {
                System.out.println("Key :: " + entry.getKey());
                System.out.println("Value :: " + entry.getValue());
            }

            //Checking extracted json is a JsonArray or not
            JsonArray jsonArray = jsonObject.get("ph").isJsonArray() ? jsonObject.get("ph").getAsJsonArray() : null;

            //Checking jsonarray is not null, printing all values from array if it is not null
            if (jsonArray != null) {
                Iterator<JsonElement> itr = jsonArray.iterator();
                while (itr.hasNext()) {
                    System.out.println("Number :: " + itr.next());
                }
            }
        }
}
}

Ping Success or Failure

Shell script to check whether ping is success for given IP.

IP=x.x.x.x
PING_COUNT=3
ping_result=$(ping $IP -c$PING_COUNT | grep 'received' | awk -F',' '{print $2}' | awk '{print $1}')
if [ $ping_result -eq 0 ] ; then
    ping_msg="Ping is failed to $IP"
else
    ping_msg="Ping is successful to $IP"
fi

Useful Links

ELK Installation:

Ubuntu: https://www.digitalocean.com/community/tutorials/how-to-install-elasticsearch-logstash-and-kibana-4-on-ubuntu-14-04

CentOS: https://www.digitalocean.com/community/tutorials/how-to-install-elasticsearch-logstash-and-kibana-4-on-centos-7