How to monitor free disk space at Amazon EC2 with Cloud Watch

As you know Amazon AWS is a great platform to develop a Cloud Infrastructure. It supposedly covers every aspect that is needed for a whole IT infrastructure. With amazon AWS, you can implement Virtual Servers, CDN, Monitoring, Performance Boosting, Scaling on demand, Storage at low cost, Big Data and so on.

The focus of today is the interaction between virtual server instances (Amazon EC2) and the server monitoring service Cloud Watch.

Cloudwatch is a great tool that offers a lot of ready to use parameters, but one missing function is that there is no parameter wich will let you monitor the amount of free space left on a partition of your instance.

Today I will show you how to implement this check in an easy way, and setup an Alarm when free space goes down below a predefined level.

I will use a Ubuntu Based EC2 instance here.

Installation and configuration of Cloud Watch

First, login into the instance that you want to start monitoring, and install possible pending updates as follows:

apt-get update
apt-get upgrade

Then we install some libraries needed by the script we later use:

apt-get install libwww-perl libdatetime-perl

Now we go through the process of the Cloudwatch script installation:

cd /opt/
wget http://aws-cloudwatch.s3.amazonaws.com/downloads/CloudWatchMonitoringScripts-1.2.1.zip
unzip CloudWatchMonitoringScripts-1.2.1.zip
rm CloudWatchMonitoringScripts-1.2.1.zip
cd aws-scripts-mon

Next we have to configure the configuration file to enable authentication to AWS infrastructure:

cp awscreds.template awscreds.conf

First of all you need to get your Access key and Secret Key of Amazon AWS Console. If you don't know where to retrieve these parameters, follow these steps: after logging into the Amazon AWS console go to Services -> All AWS Services -> IAM or simply access this link https://console.aws.amazon.com/iam/home.

Now go to:

- Users - Select your desired user - Click on User Actions - Manage Access Keys - Create Acces Key

ATTENTION!!: Your secret key will no longer be available through the AWS Management Console, you will have the only copy!!

Now we have to edit the file awscreds.conf file to add our credentials.

nano awscreds.conf

In the file, we add these parameters:

AWSAccessKeyId=XXXXXXXXXXXXXXX
AWSSecretKey=YYYYYYYYYYYYYYYYYY

Now save and exit (ctrl+o).

Setup the Disk space monitor

In the next step we setup the cronjob to update the filesystem usage statistics. In my example, I've setup a check every 5 minutes, you can change this parameter as you like it.

nano /etc/cron.d/cloudwatch-monitor

And add the following code

*/5 * * * * root /opt/aws-scripts-mon/mon-put-instance-data.pl --disk-space-avail --disk-path=/ --disk-path=/var --from-cron
chmod +x /etc/cron.d/cloudwatch-monitor
/etc/init.d/cron restart

In my case, I'll monitor the root (/) filesystem and a secondo disk for /var

Now everything is setup in our instance, the last thing we need to do, is to add the monitor in AWS.

Login to the AWS Console and go to Services -> All AWS Services -> CloudWatch -> Metrics -> Linux System Metrics, if the cronjob is working right you should see two new metrics with the Metric Name "DiskSpaceAvailabe", one for / and one for /var.

Now you can create an Alarm, simply selecting the new Metric.