Monitor Docker resource metrics with Grafana, InfluxDB, and Telegraf

Datetime:2016-08-24 12:28:14         Topic: Docker          Share        Original >>
Here to See The Original Article!!!

I needed a way to monitor Docker resource usage and metrics (CPU, Memory, Network, Disk). I also wanted historical data, and ideally, pretty graphs that I could navigate and drill into.

Whatever the solution was going to be, it had to be very open and customizable, easy to setup and scale for a production-like environment (stability, size), and ideally cheap/free. But most of all — it had to make sense and really be straight forward.

3 Containers and 10 minutes is all you need

To get this:

There are 3 components that are started via containers:

Grafana (dashboard/visual metrics and analytics)

InfluxDB (time-series DB)

Telegraf (time-series collector) – 1 per Docker host

The idea is that you first launch Grafana, and then launch InfluxDB. You configure Grafana (via the web) to point to InfluxDB’s IP, and then you setup a Telegraf container on each Docker host that you want to monitor. Telegraf collects all the metrics and feeds them into a central InfluxDB, and Grafana displays them.

Setup Tutorial/Examples

In our example, we have a bunch of servers running Docker:

vm01, vm02, vm03 [VM IPs do not matter in this case]

A “data” folder (/data) on each vm for the data mounts.

And we create an empty folder for each container: “/data/grafana”, “/data/influxdb”, “/data/telegraf”

A common network overlay or macvlan network (docker 1.12+) — we use the name “someNetworkName”

Static IPs (either via “–ip” directly, or something like pipework)

We will launch Grafana and InfluxDB on vm01, and a Telegraf container on vm01, vm02, and vm03.

Grafana (10.0.0.10)

InfluxDB (10.0.0.20)

Telegraf (10.0.0.101 on vm01, 10.0.0.102 on vm02, and 10.0.0.103 on vm03)

Grafana

Beautiful metric and analytics dashboard. Ideal for visualization and querying of time series data.

This is the first container you will launch on vm01:

(note: you are mounting the dirs so that you can access/change the configs, and data that is generated. Dirs can be empty)

 
dockerrun --restart=always -d --net=someNetworkName --ip=10.0.0.10 \
 --namegrafana \
 --hostnamegrafana \
 -v /data/grafana/var/lib/grafana:/var/lib/grafana \
 -v /data/grafana/etc/grafana:/etc/grafana \
 -v /data/grafana/var/log/grafana:/var/log/grafana \
 -p 3000:3000 \
 -e "GF_SERVER_ROOT_URL=http://10.0.0.10"  \
 -e "GF_SECURITY_ADMIN_PASSWORD=somepasswordhere"  \
 grafana/grafana
 

InfluxDB

Time-series data storage. A database designed for time-series data. Very easy to use, and feeds data into Grafana. It can accept data from many tools, specifically collectors like Telegraf.

First generate a config file:

 
dockerrun --rminfluxdbinfluxdconfig > /data/influxdb/influxdb.conf
 

Then run the container as your second container on vm01:

 
dockerrun --restart=always -d --net=someNetworkName --ip=10.0.0.20 \
 --name=influxdb \
 --hostname=influxdb \
 -p 8083:8083 -p 8086:8086 \
 -v /data/influxdb:/var/lib/influxdb \
 -v /data/influxdb/influxdb.conf:/etc/influxdb/influxdb.conf:ro \
 influxdb -config /etc/influxdb/influxdb.conf
 

Telegraf

Time-series data collection. This uses a config to know what data to collect and where to feed it.

First generate a config file:

 
dockerrun --rmtelegraf -sample-config > /data/telegraf/telegraf.conf
 

Edit the config and just enable the Docker portion (uncomment the “[[inputs.docker]]” section until the timeout)

Then run the container as your third container on vm01 (and later on vm02, and vm03, with changes bellow code):

 
dockerrun -d --restart=always --net=admin01 --ip=10.0.0.101 \
 --add-host="influxdb:10.0.0.20"\
 --name=stats-s01 \
 --hostname=vm01 \
 -e "HOST_PROC=/rootfs/proc" \
 -e "HOST_SYS=/rootfs/sys" \
 -e "HOST_ETC=/rootfs/etc" \
 -v /data/telegraf/telegraf.conf:/etc/telegraf/telegraf.conf:ro \
 -v /var/run/docker.sock:/var/run/docker.sock:ro \
 -v /sys:/rootfs/sys:ro -v /proc:/rootfs/proc:ro 







New

Put your ads here, just $200 per month.