How to add Slack notifications to an existing Monit setup

So you've got a fine Monit setup that has a lot of checks for this and that and you're getting emails whenever something fails. Life is good and you're happy. And then your company wants to try out this hip new chat thing called Slack and you think that it would be nice to get those Monit notifications into a channel on Slack. After all, it is useful to know when services fail (and come back online), so more people should be able to know about it, right?

But wait, as we're still using Ubuntu 14.04 LTS which has Monit version 5.6 bundled we can't use the new and fancy Slack integration without adding in more complexity with a PPA or compiling Monit from source and deploying it to all our servers. Then it came to me, how about just sending the Monit email notifications to Slack, that must surely work, right? Ah, bummer, seems like the email-to-slack integration is a paid service. We're not really ready to jump on the paid Slack bandwagon quite yet... Is there another way to do this?

Searching the web points me to several articles talking about the exec feature in Monit to run a script to do your notifications, but it requires a lot of extra setup for each and every Monit check you have. Ah, that's a lot of work since we have hundreds of checks on many servers. I don't want to do that. Isn't there a way to just send that email to Slack without having to pay for the email integration? Yes, there is! As we already have Postfix set up on the servers to send outbound mail (for cron jobs and the like) we can just add an alias that sends the email to a script that takes care of converting that email to something useful and posting it to our Slack channel. Time to whip out that trusty sysadmin tool called Perl again! Who needs all those shiny new languages, right?

The code itself

You'll need a URL to post your notification to your Slack channel. You generate it by adding an integration called "Incoming WebHooks" to your Slack channel.

I'm going to assume you have everything talked about set up already, so the only thing you'll need is the additional dependencies to run our Perl script. Get your dependencies installed like this:

$ apt-get install perl curl libemail-simple-perl libjson-perl

Add an email alias to execute our script for each email sent to it that posts your Monit notification to Slack:

$ echo 'slack-monit: |/etc/monit/slack_notification' >> /etc/aliases

Create a text file with this content called /etc/monit/slack_notification and make it executable (chmod +x):

#!/usr/bin/env perl

use strict;
use warnings;

use JSON;
use Email::Simple;

my $url = 'https://hooks.slack.com/services/XXXXXX/YYYYY/ZZZZZ';
my $server = qx(hostname -f);
chomp $server;

my $stdin = do { local $/; <>; };
my $email = Email::Simple->new($stdin);

my $payload = { 'username' => "Monit ($server)" };

if ( $email->body ) {
    $payload->{'text'} = $email->body;
}

my @cmd = ('curl', '-s', '-X', 'POST',
    '--data-urlencode', 'payload=' . encode_json($payload),
    $url,
);
system(@cmd);

Last words

Obviously this script is ripe for improvements, but this is a basic setup that you can iterate on to get your perfect devopsy message over to your Slack channel. If you want to see future improvements to it, head over to the monit-slack-notification GitHub project page and have a look.