Menu
Sign In to the Console
Amazon Elastic Compute Cloud
User Guide for Linux Instances

Monitoring Memory and Disk Metrics for Amazon EC2 Linux Instances

The Amazon CloudWatch Monitoring Scripts for Amazon Elastic Compute Cloud (Amazon EC2) Linux-based instances demonstrate how to produce and consume Amazon CloudWatch custom metrics. These sample Perl scripts comprise a fully functional example that reports memory, swap, and disk space utilization metrics for a Linux instance. You can download the Amazon CloudWatch Monitoring Scripts for Linux from the AWS sample code library.

Important

These scripts are examples only. They are provided "as is" and are not supported.

These monitoring scripts are intended for use with Amazon EC2 instances running Linux operating systems. The scripts have been tested on the following Amazon Machine Images (AMIs) for both 32-bit and 64-bit versions:

  • Amazon Linux 2014.09.2

  • Red Hat Enterprise Linux 6.6

  • SUSE Linux Enterprise Server 12

  • Ubuntu Server 14.04

Note

Standard Amazon CloudWatch free tier quantities and usage charges for custom metrics apply to your use of these scripts. For more information, see the Amazon CloudWatch pricing page.

You can use EC2Config on Amazon EC2 instances running Microsoft Windows to monitor memory and disk metrics by sending this data to CloudWatch Logs. To get started with CloudWatch Logs on an Amazon EC2 instance running Microsoft Windows, see Sending Performance Counters to CloudWatch and Logs to CloudWatch Logs in the Amazon EC2 User Guide for Microsoft Windows Instances.

Contents

Prerequisites

You must perform additional steps on some versions of Linux.

Note

The CloudWatchClient.pm module included in the script package locally caches instance metadata. If you create an AMI from an instance where you have run the scripts, any instances launched from this AMI within the cache TTL (default: six hours, 24 hours for Auto Scaling groups) will emit metrics using the original instance's ID. After the cache TTL time period passes, the script will retrieve fresh data and the scripts will use the current instance's ID. To immediately correct this, remove the cached data using: $ rm /var/tmp/aws-mon/instance-id.

Amazon Linux AMI

If you are running Amazon Linux AMI, version 2014.03 or later, you’ll need to add some additional Perl modules in order for the monitoring scripts to work. Use the following procedures to configure your server.

To install the scripts for the first time

  1. Log on to your Amazon Linux AMI instance.

  2. At a command prompt, install the following package:

    $ sudo yum install perl-Switch perl-DateTime perl-Sys-Syslog perl-LWP-Protocol-https

Red Hat Enterprise Linux

If you are running Red Hat Enterprise Linux, you’ll need to add some additional Perl modules in order for the monitoring scripts to work. Use the following procedures to configure your server.

To install the scripts for the first time

  1. Log on to your Red Hat Enterprise Linux instance.

  2. At a command prompt, install the following packages:

    $ sudo yum install perl-Switch perl-DateTime perl-Sys-Syslog perl-LWP-Protocol-https perl-Digest-SHA -y
    $ sudo yum install zip unzip

SUSE Linux Enterprise Server

If you are running SUSE Linux Enterprise Server, you’ll need to add some additional Perl modules in order for the monitoring scripts to work. Use the following procedures to configure your server.

To install the scripts for the first time

  1. Log on to your SUSE Linux Enterprise Server instance.

  2. At a command prompt, install the following packages:

    $ sudo zypper install perl-Switch perl-DateTime
    $ sudo zypper install –y "perl(LWP::Protocol::https)"

Ubuntu Server

If you are running Ubuntu Server, use the following procedures to configure your server.

To install the scripts for the first time

  1. Log on to your Ubuntu Server instance.

  2. At a command prompt, install the following packages:

    $ sudo apt-get update
    $ sudo apt-get install unzip
    $ sudo apt-get install libwww-perl libdatetime-perl

For information about connecting to Amazon EC2 Linux instances, see Connect to Your Linux Instance.

Getting Started

The following steps show you how to download, uncompress, and configure the CloudWatch Monitoring Scripts on an EC2 Linux instance.

To download, install, and configure the script

  1. Open a command prompt, move to a folder where you want to store the scripts and then type the following:

    curl http://aws-cloudwatch.s3.amazonaws.com/downloads/CloudWatchMonitoringScripts-1.2.1.zip -O
unzip CloudWatchMonitoringScripts-1.2.1.zip
rm CloudWatchMonitoringScripts-1.2.1.zip
cd aws-scripts-mon

    The CloudWatchMonitoringScripts-1.2.1.zip package contains these files:

    • CloudWatchClient.pm—Shared Perl module that simplifies calling Amazon CloudWatch from other scripts.

    • mon-put-instance-data.pl—Collects system metrics on an Amazon EC2 instance (memory, swap, disk space utilization) and sends them to Amazon CloudWatch.

    • mon-get-instance-stats.pl—Queries Amazon CloudWatch and displays the most recent utilization statistics for the EC2 instance on which this script is executed.

    • awscreds.template—File template for AWS credentials that stores your access key ID and secret access key.

    • LICENSE.txt—Text file containing the Apache 2.0 license.

    • NOTICE.txt—copyright notice.

  2. If you already have an AWS Identity and Access Management (IAM) role associated with your instance, make sure that it has permissions to perform the following operations:

    • cloudwatch:PutMetricData

    • cloudwatch:GetMetricStatistics

    • cloudwatch:ListMetrics

    • ec2:DescribeTags

    Otherwise, you can create a new IAM role with permissions to perform CloudWatch operations and associate that role when you launch a new instance. For more information, see Controlling User Access to Your AWS Account.

  3. Optional: If you aren't using an IAM role, update the awscreds.template file that you downloaded earlier. The content of this file should use the following format:

    AWSAccessKeyId=YourAccessKeyID

    AWSSecretKey=YourSecretAccessKey

    Note

    This step is optional if you have already created a file for credentials. You can use an existing file by specifying its location on the command line when you call the scripts. Alternatively, you can set the environment variable AWS_CREDENTIAL_FILE to point to the file with your AWS credentials.

    For instructions on how to access your credentials, see Creating, Modifying, and Viewing User Security Credentials in the IAM User Guide.

mon-put-instance-data.pl

This script collects memory, swap, and disk space utilization data on the current system. It then makes a remote call to Amazon CloudWatch to report the collected data as custom metrics.

Options

NameDescription

--mem-util

Collects and sends the MemoryUtilization metrics in percentages. This option reports only memory allocated by applications and the operating system, and excludes memory in cache and buffers.

--mem-used

Collects and sends the MemoryUsed metrics, reported in megabytes. This option reports only memory allocated by applications and the operating system, and excludes memory in cache and buffers.

--mem-avail

Collects and sends the MemoryAvailable metrics, reported in megabytes. This option reports memory available for use by applications and the operating system.

--swap-util

Collects and sends SwapUtilization metrics, reported in percentages.

--swap-used

Collects and sends SwapUsed metrics, reported in megabytes.

--disk-path=PATH

Selects the disk on which to report.

PATH can specify a mount point or any file located on a mount point for the filesystem that needs to be reported. For selecting multiple disks, specify a --disk-path=PATH for each one of them.

To select a disk for the filesystems mounted on / and /home, use the following parameters:

--disk-path=/ --disk-path=/home

--disk-space-util

Collects and sends the DiskSpaceUtilization metric for the selected disks. The metric is reported in percentages.

--disk-space-used

Collects and sends the DiskSpaceUsed metric for the selected disks. The metric is reported by default in gigabytes.

Due to reserved disk space in Linux operating systems, disk space used and disk space available might not accurately add up to the amount of total disk space.

--disk-space-avail

Collects and sends the DiskSpaceAvailable metric for the selected disks. The metric is reported in gigabytes.

Due to reserved disk space in the Linux operating systems, disk space used and disk space available might not accurately add up to the amount of total disk space.

--memory-units=UNITS

Specifies units in which to report memory usage. If not specified, memory is reported in megabytes. UNITS may be one of the following: bytes, kilobytes, megabytes, gigabytes.

--disk-space-units=UNITS

Specifies units in which to report disk space usage. If not specified, disk space is reported in gigabytes. UNITS may be one of the following: bytes, kilobytes, megabytes, gigabytes.

--aws-credential- file=PATH

Provides the location of the file containing AWS credentials.

This parameter cannot be used with the --aws-access-key-id and --aws-secret-key parameters.

--aws-access-key-id=VALUE

Specifies the AWS access key ID to use to identify the caller. Must be used together with the --aws-secret-key option. Do not use this option with the --aws-credential-file parameter.

--aws-secret-key=VALUE

Specifies the AWS secret access key to use to sign the request to CloudWatch. Must be used together with the --aws-access-key-id option. Do not use this option with --aws-credential-file parameter.

--aws-iam-role=VALUE

Specifies the IAM role used to provide AWS credentials. The value =VALUE is required. If no credentials are specified, the default IAM role associated with the EC2 instance is applied. Only one IAM role can be used. If no IAM roles are found, or if more than one IAM role is found, the script will return an error.

Do not use this option with the --aws-credential-file, --aws-access-key-id, or --aws-secret-key parameters.

--aggregated[=only]

Adds aggregated metrics for instance type, AMI ID, and overall for the region. The value =only is optional; if specified, the script reports only aggregated metrics.

--auto-scaling[=only]

Adds aggregated metrics for the Auto Scaling group. The value =only is optional; if specified, the script reports only Auto Scaling metrics. The IAM policy associated with the IAM account or role using the scripts need to have permissions to call the EC2 action DescribeTags.

--verify

Performs a test run of the script that collects the metrics, prepares a complete HTTP request, but does not actually call CloudWatch to report the data. This option also checks that credentials are provided. When run in verbose mode, this option outputs the metrics that will be sent to CloudWatch.

--from-cron

Use this option when calling the script from cron. When this option is used, all diagnostic output is suppressed, but error messages are sent to the local system log of the user account.

--verbose

Displays detailed information about what the script is doing.

--help

Displays usage information.

--version

Displays the version number of the script.


Examples

The following examples assume that you have already updated the awscreds.conf file with valid AWS credentials. If you are not using the awscreds.conf file, provide credentials using the --aws-access-key-id and --aws-secret-key arguments.

To perform a simple test run without posting data to CloudWatch

  • Run the following command:

    ./mon-put-instance-data.pl --mem-util --verify --verbose

To collect all available memory metrics and send them to CloudWatch

  • Run the following command:

    ./mon-put-instance-data.pl --mem-util --mem-used --mem-avail

To set a cron schedule for metrics reported to CloudWatch

  1. Start editing the crontab using the following command: 

    crontab -e

  2. Add the following command to report memory and disk space utilization to CloudWatch every five minutes:

    */5 * * * * ~/aws-scripts-mon/mon-put-instance-data.pl --mem-util --disk-space-util --disk-path=/ --from-cron

    If the script encounters an error, the script will write the error message in the system log.

To collect aggregated metrics for an Auto Scaling group and send them to Amazon CloudWatch without reporting individual instance metrics

  • Run the following command:

    ./mon-put-instance-data.pl --mem-util --mem-used --mem-avail --auto-scaling=only

To collect aggregated metrics for instance type, AMI ID and region, and send them to Amazon CloudWatch without reporting individual instance metrics

  • Run the following command:

    ./mon-put-instance-data.pl --mem-util --mem-used --mem-avail --aggregated=only

mon-get-instance-stats.pl

This script queries CloudWatch for statistics on memory, swap, and disk space metrics within the time interval provided using the number of most recent hours. This data is provided for the Amazon EC2 instance on which this script is executed.

Options

NameDescription

--recent-hours=N

Specifies the number of recent hours to report on, as represented by N where N is an integer.

--aws-credential-file=PATH

Provides the location of the file containing AWS credentials.

--aws-access-key-id=VALUE

Specifies the AWS access key ID to use to identify the caller. Must be used together with the --aws-secret-key option. Do not use this option with the --aws-credential-file option.

--aws-secret-key=VALUE

Specifies the AWS secret access key to use to sign the request to CloudWatch. Must be used together with the --aws-access-key-id option. Do not use this option with --aws-credential-file option.

--aws-iam-role=VALUE

Specifies the IAM role used to provide AWS credentials. The value =VALUE is required. If no credentials are specified, the default IAM role associated with the EC2 instance is applied. Only one IAM role can be used. If no IAM roles are found, or if more than one IAM role is found, the script will return an error.

Do not use this option with the --aws-credential-file, --aws-access-key-id, or --aws-secret-key parameters.

--verify

Performs a test run of the script that collects the metrics, prepares a complete HTTP request, but does not actually call CloudWatch to report the data. This option also checks that credentials are provided. When run in verbose mode, this option outputs the metrics that will be sent to CloudWatch.

--verbose

Displays detailed information about what the script is doing.

--help

Displays usage information.

--version

Displays the version number of the script.


Examples

To get utilization statistics for the last 12 hours

  • Run the following command:

    ./mon-get-instance-stats.pl --recent-hours=12

    The returned response will be similar to the following example output:

    Instance metric statistics for the last 12 hours.

CPU Utilization
    Average: 1.06%, Minimum: 0.00%, Maximum: 15.22%

Memory Utilization
    Average: 6.84%, Minimum: 6.82%, Maximum: 6.89%

Swap Utilization
    Average: N/A, Minimum: N/A, Maximum: N/A

Disk Space Utilization on /dev/xvda1 mounted as /
    Average: 9.69%, Minimum: 9.69%, Maximum: 9.69%

Viewing Your Custom Metrics in the Console

If you successfully call the mon-put-instance-data.pl script, you can use the AWS Management Console to view your posted custom metrics in the Amazon CloudWatch console.

To view custom metrics

  1. Execute mon-put-instance-data.pl, as described earlier.

  2. Sign in to the AWS Management Console and open the CloudWatch console at https://console.aws.amazon.com/cloudwatch/.

  3. Click View Metrics.

  4. In the Viewing list, your custom metrics posted by the script are displayed with the prefix System/Linux.