Integrate Grafana Dashboard with Hubot and Slack

Blog: 18th June 2019 | DevOps | Featured Technology: Hubot, Grafana Dashboards , Prometheus, Slack

 

Grafana is well known dashboard which is visualized time series data. Most DevOps engineers currently use Grafana to configure various kind of data sources to generate reports. Below I am using two AWS instance to demonstrate the Grafana Dashboard with Hubot and Slack ; one instance to install the Grafana and Prometheus (Prometheus as a data source ) and another one to install Hubot (chatbot).

Design Architecture

Let’s Start Configuring!

Here I am not going to explain on how to launch an AWS E2 instance. However if you are new to AWS please check this link.  Also to please refer below links to install and configure Prometheus with node exporter.  Install Prometheus Link | Install Node Exporter Link.

I know I have sent you in different directions with links to various sources to prepare the initial configurations however I hope you can appreciate that I didn’t add these configurations to this blog due to the length of this blog being very long and also my intention of focusing on providing info on the integrations of Grafana Dashboard with Hubot and Slack.

Time To Have FUN..!

First Install Grafana.

Here I am going to install Grafana with yum repository. It is really easy and if you need more installations methods please check this link.

cd /etc/yum.repo.d
vim grafana.repo

Add below lines to grafana.repo file

[grafana]=grafana
baseurl=https://packages.grafana.com/oss/rpm
repo_gpgcheck=1
enabled=1
gpgcheck=1
gpgkey=https://packages.grafana.com/gpg.key
sslverify=1
sslcacert=/etc/pki/tls/certs/ca-bundle.crt

Trigger Grafana installation

yum install grafana -y

Start and enable Grafana service

systemctl start garafana-server
systemctl enable garafana-server

Connect Grafana With Prometheus!

Let’s connect to the Grafana Dashboard via http://<ip address>:3000

Then we add our data source- to do this, select the Configuration section

Once you select Data Source you will see a page like this:

Click on the ‘Add data source’ button to continue the configurations. Then select ‘Prometheus’ as a data source.

Configure Data Source parameters. Under name, you can define any name to your data source and under URLput your Prometheus URL (You can find it under configure Prometheus section).

Once you finish, make sure to save the configurations.

Now we can start to configure the dashboards.

Select the  “+” icon and select Dashboard. There you can select ‘Choose Visualizations ‘.

Once you click on the Choose Visualizations  button you can start creating your own graf, But for this demo, I’m going to select “Graph”.

Once you select “Graph” it will allow you to add related parameters to develop the graph depending on your requirement.

First we need to select ‘Queries’ in here you can have to add ‘Prometheus’ query from Prometheus

E.g.:

node_procs_running{instance=”<prometheus-ip>:9100″,job=”node_exporter_metrics”}

Likewise, this will allow you to configure visualization, general & alert setting according to your choice.

Once you configure this correctly you can see this type of graph on your dashboard

Now Let’s Install Hubot!

Login to your 2nd EC2 instance and hit below commands

yum update -y

Install Nodejs and Npm

before installing nodejs and npm we have to add the relevant repository to obtain the packages.

yum install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
yum install -y epel-release
yum repolist
yum install nodejs npm

Once you have finished the initial installations, we can then move forward to configure the Hubot. For more information please check this link.

npm install -g yo generator-hubot
mkdir myhubot
cd myhubot
yo hubot

Once you hit ‘yo hubot’ this will run the hubot setup and prompt you for some question related to its configurations. Make sure to grant permissions to the required folder, Otherwise, this yo hubot installation will present some errors. If you get some process break errors due to permission issues, don’t worry, you can always runt the yo hubot again. In my installations, I have grant chmod 777 <directory>

Eg:

cd /root/.config/
chmod -Rf 777 insight-nodejs/
cd /opt/
chmod -Rf 777 /myhubot/
cd /root/
7 /.npm/

Once the hubot installation had ended, we wil l then have to integrate Slack with Hubot.

Integrate Slack with Hubot!

We have to install the hubot-grafana module under the Hubot installed directory:

npm install hubot-grafana --save

Once the installation process finished add “hubot-grafan” line in to external-scripts.json file.

[
  "hubot-grafana"
]

Once done,  for all configurations, we have to create a Slack app by integrating the Hubot API.

Create The Slack App!

Please visit this link https://api.slack.com/apps to create the Slack workspace and add the Hubot app.

Once you have clicked on ‘Add Configuration’ it will open another window for you to to add some required details to finalize the installation.

Note : Make sure to save the Slack Api-key.

Once the Slak app creation process is over,  we then ae have to login to Hubot installed instance again and add environment variables to before running Hubot.

Note : Generate Grafana API-KEY

Goto Configuration -> api keys and click on ‘Add API Key’ Then enter name and select the Role as Admin and click on ‘Add’ to have the Grafana Api Key.

vim ~/.bashrc

Then add these lines:

export HUBOT_SLACK_TOKEN="Slack Token"
export HUBOT_GRAFANA_HOST="http://<Grafana Host IP>:3000"
export HUBOT_GRAFANA_API_KEY="Grafana API Key"

Finally, we can star Hubot. Make sure to move inside the Hubot directory before hitting this command.

cd /myhubot
./bin/hubot --adapter slack

If there any issue You can troubleshoot with by enabling the debug on hubot

HUBOT_LOG_LEVEL=debug ./bin/hubot --adapter slack

Now you can start chatting with Hubot!

Hubot Commands:

hubot graf list
Lists the available dashboards
hubot graf db graphite-carbon-metrics
Graph all panels in the dashboard
hubot graf db graphite-carbon-metrics:3
Graph only panel with id 3 of a particular dashboard
hubot graf db graphite-carbon-metrics:cpu
Graph only the panels containing “cpu” (case insensitive) in the title
hubot graf db graphite-carbon-metrics now-12hr
Get a dashboard with a window of 12 hours ago to now
hubot graf db graphite-carbon-metrics now-24hr now-12hr
Get a dashboard with a window of 24 hours ago to 12 hours ago
hubot graf db graphite-carbon-metrics:3 now-8d now-1d
Get only the third panel of a particular dashboard with a window of 8 days ago to yesterday
hubot graf db graphite-carbon-metrics host=carbon-a
Get a templated dashboard with the $host parameter set to carbon-a

Enjoy Chatting With Hubot..!

Author:
Go to the profile of Prasad Jayasinghe

Prasad Jayasinghe – 

Associate Tech Lead @ Global Wavenet
Connect with Prasad on LinkedIn here

More from our blog

See all posts