Graphitecollectdstatsd


Graphite Docker install

docker run -d \
 --name graphite \
 --restart=always \
 -p 80:80 \
 -p 2003-2004:2003-2004 \
 -p 2023-2024:2023-2024 \
 -p 8125:8125/udp \
 -p 8126:8126 \
 graphiteapp/graphite-statsd
                                                                                                       

Using Mounted Volumes

docker run -d\
 --name graphite\
 --restart=always\
 -v /path/to/graphite/configs:/opt/graphite/conf\
 -v /path/to/graphite/data:/opt/graphite/storage\
 -v /path/to/statsd_config:/opt/statsd/config\
 graphiteapp/graphite-statsd

Note: The container will initialize properly if you mount empty volumes at /opt/graphite/conf, /opt/graphite/storage, or /opt/statsd/config.

Install StatsD

Install nodejs on the target

Download statsd using wget https://github.com/statsd/statsd/archive/master.zip

Create a config file from exampleConfig.js and put it somewhere

Start the Daemon: node stats.js /path/to/config

Check if statds is running

sudo lsof | grep '2300'

Install CollectD

sudo yum install collectd

A minor configuration change will be needed to direct Collectd to report metrics to your Graphite server. Edit the configuration file (typically found at /etc/collectd/collectd.conf) and then restart the collectd service.

LoadPlugin write_graphite
<Plugin write_graphite>
  <Node "example">
    Host "localhost"
    Port "2003"
    Prefix "collectd"
  </Node>
</Plugin>

Check if Graphite and Collectd are running

netstat -nltp | grep '2700'

sudo systemctl status collectd

Grafana Collectd Graphite tips

sudo yum install collectd

collectd.conf has a section called write graphite missed it, you need to enable it

LoadPlugin write_graphite

also uncomment Interval 10

localhost:80 graphite docker running port add in grafana data source

Reset grafana admin password

$ sudo sqlite3 /var/lib/grafana/grafana.db

sqlite> update user set password = ‘59acf18b94d7eb0694c61e60ce44c110c7a683ac6a8f09580d626f90f4a242000746579358d77dd9e570e83fa24faa88a8a6’, salt = ‘F3FAxVm33R’ where login = ‘admin’;

sqlite> .exit

Grafana Queries

Queries are how Grafana panels communicate with data sources to get data for the visualization.

A query is a question written in the query language used by the data source.

Grafana asks, “Hey data source, would you send me this data, organized this way?”,If the query is properly formed, then the data source responds.

How often the query is sent to the data source and how many data points are collected can be adjusted in the panel data source options.

Some Data Sources are Graphite,InfluxDB,Prometheus,Loki

Query editors

Query editors are forms that help you write queries.

PromQL Examples

PromQL is Prometheus Query Language

Simple time series selection

Return all time series with the metric http_requests_total:

http_requests_total

Return all time series with the metric http_requests_total and the given job and handler labels:

http_requests_total{job=“apiserver”, handler=“/api/comments”}

Return a whole range of time (in this case 5 minutes) for the same vector, making it a range vector:

http_requests_total{job=“apiserver”, handler=“/api/comments”}[5m]

Note that an expression resulting in a range vector cannot be graphed directly, but viewed in the tabular (“Console”) view of the expression browser.

Using regular expressions, you could select time series only for jobs whose name match a certain pattern, in this case, all jobs that end with server:

http_requests_total{job=~".*server"}

All regular expressions in Prometheus use RE2 syntax.

To select all HTTP status codes except 4xx ones, you could run:

http_requests_total{status!~“4..”}

Subquery

Return the 5-minute rate of the http_requests_total metric for the past 30 minutes, with a resolution of 1 minute.

rate(http_requests_total[5m])[30m:1m]

This is an example of a nested subquery. The subquery for the deriv function uses the default resolution. Note that using subqueries unnecessarily is unwise.

max_over_time(deriv(rate(distance_covered_total[5s])[30s:5s])[10m:])

Using functions, operators, etc.

Return the per-second rate for all time series with the http_requests_total metric name, as measured over the last 5 minutes:

rate(http_requests_total[5m])


These writings represent my own personal views alone.
Licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.