Setting up the Datadog integration with Amazon Web Services requires configuring role delegation using AWS IAM. To get a better
understanding of role delegation, refer to the AWS IAM Best Practices guide.
Note: The GovCloud and China regions do not currently support IAM role delegation. If you are deploying in these regions please skip to the configuration section below.
aws.logs.incoming_bytes
(gauge) |
The volume of log events in uncompressed bytes uploaded to Cloudwatch Logs.
shown as byte
|
aws.logs.incoming_log_events
(count) |
The number of log events uploaded to Cloudwatch Logs.
shown as event
|
aws.logs.forwarded_bytes
(gauge) |
The volume of log events in compressed bytes forwarded to the subscription destination.
shown as byte
|
aws.logs.forwarded_log_events
(count) |
The number of log events forwarded to the subscription destination.
shown as event
|
aws.logs.delivery_errors
(count) |
The number of log events for which CloudWatch Logs received an error when forwarding data to the subscription destination.
shown as event
|
aws.logs.delivery_throttling
(count) |
The number of log events for which CloudWatch Logs was throttled when forwarding data to the subscription destination.
shown as event
|
aws.ec2spot.available_instance_pools_count
(count) |
The Spot Instance pools specified in the Spot Fleet request.
shown as instance
|
aws.ec2spot.bids_submitted_for_capacity
(count) |
The capacity for which Amazon EC2 has submitted bids.
shown as instance
|
aws.ec2spot.eligible_instance_pool_count
(count) |
The Spot Instance pools specified in the Spot Fleet request where Amazon EC2 can fulfill bids.
shown as instance
|
aws.ec2spot.fulfilled_capacity
(count) |
The capacity that Amazon EC2 has fulfilled.
shown as instance
|
aws.ec2spot.max_percent_capacity_allocation
(gauge) |
The maximum value of PercentCapacityAllocation across all Spot Instance pools specified in the Spot Fleet request.
shown as percent
|
aws.ec2spot.pending_capacity
(count) |
The difference between TargetCapacity and FulfilledCapacity.
shown as instance
|
aws.ec2spot.percent_capacity_allocation
(gauge) |
The capacity allocated for the Spot Instance pool for the specified dimensions.
shown as percent
|
aws.ec2spot.target_capacity
(count) |
The target capacity of the Spot Fleet request.
shown as instance
|
aws.ec2spot.terminating_capacity
(count) |
The capacity that is being terminated due to Spot Instance interruptions.
shown as instance
|
aws.dms.cpuutilization
(gauge) |
Average percentage of allocated EC2 compute units that are currently in use on the instance.
|
aws.dms.free_storage_space
(gauge) |
The amount of available storage space
shown as byte
|
aws.dms.freeable_memory
(gauge) |
The amount of available random access memory.
shown as byte
|
aws.dms.write_iops
(gauge) |
The average number of disk I/O operations per second
shown as operation/second
|
aws.dms.read_iops
(gauge) |
The average number of disk I/O operations per second.
shown as operation/second
|
aws.dms.write_throughput
(gauge) |
The average number of bytes written to disk per second.
shown as byte/second
|
aws.dms.read_throughput
(gauge) |
The average number of bytes read from disk per second.
shown as byte/second
|
aws.dms.write_latency
(gauge) |
The average amount of time taken per write disk I/O operation
shown as second
|
aws.dms.read_latency
(gauge) |
The average amount of time taken per read disk I/O operation
shown as second
|
aws.dms.swap_usage
(gauge) |
The amount of swap space used on the DB Instance
shown as byte
|
aws.dms.network_transmit_throughput
(gauge) |
The outgoing (Transmit) network traffic on the DB instance including both customer database traffic and Amazon RDS traffic used for monitoring and replication
shown as byte/second
|
aws.dms.network_receive_throughput
(gauge) |
The incoming (Receive) network traffic on the DB instance including both customer database traffic and Amazon RDS traffic used for monitoring and replication.
shown as byte/second
|
aws.dms.full_load_throughput_bandwidth_source
(gauge) |
Incoming network bandwidth from a full load from the source
shown as kibibyte/second
|
aws.dms.full_load_throughput_bandwidth_target
(gauge) |
Outgoing network bandwidth from a full load for the target
shown as kibibyte/second
|
aws.dms.full_load_throughput_rows_source
(gauge) |
Incoming changes from a full load from the source in rows per second
shown as row/second
|
aws.dms.full_load_throughput_rows_target
(gauge) |
Outgoing changes from a full load for the target
shown as row
|
aws.dms.cdcincoming_changes
(gauge) |
Total row count of changes for the task
shown as row
|
aws.dms.cdcchanges_memory_source
(gauge) |
Amount of rows accumulating in a memory and waiting to be committed from the source
shown as row
|
aws.dms.cdcchanges_memory_target
(gauge) |
Amount of rows accumulating in a memory and waiting to be committed to the target
shown as row
|
aws.dms.cdcchanges_disk_source
(gauge) |
Amount of rows accumulating on disk and waiting to be committed from the source
shown as row
|
aws.dms.cdcchanges_disk_target
(gauge) |
Amount of rows accumulating on disk and waiting to be committed to the target
shown as row
|
aws.dms.cdcthroughput_bandwidth_source
(gauge) |
Incoming task network bandwidth from the source
shown as kibibyte/second
|
aws.dms.cdcthroughput_bandwidth_target
(gauge) |
Outgoing task network bandwidth for the target
shown as kibibyte/second
|
aws.dms.cdcthroughput_rows_source
(gauge) |
Incoming task changes from the source
shown as row/second
|
aws.dms.cdcthroughput_rows_target
(gauge) |
Outgoing task changes for the target
shown as row/second
|
aws.dms.cdclatency_source
(gauge) |
Latency reading from source
shown as second
|
aws.dms.cdclatency_target
(gauge) |
Latency writing to the target
shown as second
|
aws.events.invocations
(count) |
Measures the number of times a target is invoked for a rule in response to an event. This includes successful and failed invocations but does not include throttled or retried attempts until they fail permanently.
|
aws.events.failed_invocations
(count) |
Measures the number of invocations that failed permanently. This does not include invocations that are retried or that succeeded after a retry attempt
|
aws.events.triggered_rules
(count) |
Measures the number of triggered rules that matched with any event.
|
aws.events.matched_events
(count) |
Measures the number of events that matched with any rule.
|
aws.events.throttled_rules
(count) |
Measures the number of triggered rules that are being throttled.
|
aws.states.execution_time
(gauge every 60 seconds) |
The average time interval, in milliseconds, between the time the execution started and the time it closed.
shown as millisecond
|
aws.states.execution_time.maximum
(gauge every 60 seconds) |
The maximum time interval, in milliseconds, between the time the execution started and the time it closed.
shown as millisecond
|
aws.states.execution_time.minimum
(gauge every 60 seconds) |
The minimum time interval, in milliseconds, between the time the execution started and the time it closed.
shown as millisecond
|
aws.states.execution_time.p95
(gauge every 60 seconds) |
The 95th percentile time interval, in milliseconds, between the time the execution started and the time it closed.
shown as millisecond
|
aws.states.execution_time.p99
(gauge every 60 seconds) |
The 99th percentile time interval, in milliseconds, between the time the execution started and the time it closed.il
shown as millisecond
|
aws.states.executions_aborted
(count every 60 seconds) |
The number of executions that were aborted/terminated.
|
aws.states.executions_failed
(count every 60 seconds) |
The number of executions that failed.
|
aws.states.executions_started
(count every 60 seconds) |
The number of executions started.
|
aws.states.executions_succeeded
(count every 60 seconds) |
The number of executions that completed successfully.
|
aws.states.executions_timed_out
(count every 60 seconds) |
The number of executions that timed out for any reason.
|
aws.states.lambda_function_run_time
(gauge every 60 seconds) |
The average time interval, in milliseconds, between the time the lambda function was started and when it was closed.
shown as millisecond
|
aws.states.lambda_function_run_time.maximum
(gauge every 60 seconds) |
The maximum time interval, in milliseconds, between the time the lambda function was started and when it was closed.
shown as millisecond
|
aws.states.lambda_function_run_time.minimum
(gauge every 60 seconds) |
The minimum time interval, in milliseconds, between the time the lambda function was started and when it was closed.
shown as millisecond
|
aws.states.lambda_function_run_time.p95
(gauge every 60 seconds) |
The 95th percentile time interval, in milliseconds, between the time the lambda function was started and when it was closed.
shown as millisecond
|
aws.states.lambda_function_run_time.p99
(gauge every 60 seconds) |
The 99th percentile time interval, in milliseconds, between the time the lambda function was started and when it was closed.
shown as millisecond
|
aws.states.lambda_function_schedule_time
(gauge every 60 seconds) |
The avg time interval, in milliseconds, that the activity stayed in the schedule state.
shown as millisecond
|
aws.states.lambda_function_schedule_time.maximum
(gauge every 60 seconds) |
The maximum time interval, in milliseconds, that the activity stayed in the schedule state.
shown as millisecond
|
aws.states.lambda_function_schedule_time.minimum
(gauge every 60 seconds) |
The minimum time interval, in milliseconds, that the activity stayed in the schedule state.
shown as millisecond
|
aws.states.lambda_function_schedule_time.p95
(gauge every 60 seconds) |
The 95th percentile time interval, in milliseconds, that the activity stayed in the schedule state.
shown as millisecond
|
aws.states.lambda_function_schedule_time.p99
(gauge every 60 seconds) |
The 99th percentile time interval, in milliseconds, that the activity stayed in the schedule state.
shown as millisecond
|
aws.states.lambda_function_time
(gauge every 60 seconds) |
The average time interval, in milliseconds, between the time the lambda function was scheduled and when it was closed.
shown as millisecond
|
aws.states.lambda_function_time.maximum
(gauge every 60 seconds) |
The maximum time interval, in milliseconds, between the time the lambda function was scheduled and when it was closed.
shown as millisecond
|
aws.states.lambda_function_time.minimum
(gauge every 60 seconds) |
The minimum time interval, in milliseconds, between the time the lambda function was scheduled and when it was closed.
shown as millisecond
|
aws.states.lambda_function_time.p95
(gauge every 60 seconds) |
The 95th percentile time interval, in milliseconds, between the time the lambda function was scheduled and when it was closed.
shown as millisecond
|
aws.states.lambda_function_time.p99
(gauge every 60 seconds) |
The 99th percentile time interval, in milliseconds, between the time the lambda function was scheduled and when it was closed.
shown as millisecond
|
aws.states.lambda_functions_failed
(count every 60 seconds) |
The number of lambda functions that failed.
|
aws.states.lambda_functions_heartbeat_timed_out
(count every 60 seconds) |
The number of lambda functions that were timed out due to a heartbeat timeout.
|
aws.states.lambda_functions_scheduled
(count every 60 seconds) |
The number of lambda functions that were scheduled.
|
aws.states.lambda_functions_started
(count every 60 seconds) |
The number of lambda functions that were started.
|
aws.states.lambda_functions_succeeded
(count every 60 seconds) |
The number of lambda functions that completed successfully.
|
aws.states.lambda_functions_timed_out
(count every 60 seconds) |
The number of lambda functions that were timed out on close.
|
aws.states.activity_run_time
(gauge every 60 seconds) |
The average time interval, in milliseconds, between the time the activity was started and when it was closed.
shown as millisecond
|
aws.states.activity_run_time.maximum
(gauge every 60 seconds) |
The maximum time interval, in milliseconds, between the time the activity was started and when it was closed.
shown as millisecond
|
aws.states.activity_run_time.minimum
(gauge every 60 seconds) |
The minimum time interval, in milliseconds, between the time the activity was started and when it was closed.
shown as millisecond
|
aws.states.activity_run_time.p95
(gauge every 60 seconds) |
The 95th percentile time interval, in milliseconds, between the time the activity was started and when it was closed.
shown as millisecond
|
aws.states.activity_run_time.p99
(gauge every 60 seconds) |
The 99th percentile time interval, in milliseconds, between the time the activity was started and when it was closed.
shown as millisecond
|
aws.states.activity_schedule_time
(gauge every 60 seconds) |
The avg time interval, in milliseconds, that the activity stayed in the schedule state.
shown as millisecond
|
aws.states.activity_schedule_time.maximum
(gauge every 60 seconds) |
The maximum time interval, in milliseconds, that the activity stayed in the schedule state.
shown as millisecond
|
aws.states.activity_schedule_time.minimum
(gauge every 60 seconds) |
The minimum time interval, in milliseconds, that the activity stayed in the schedule state.
shown as millisecond
|
aws.states.activity_schedule_time.p95
(gauge every 60 seconds) |
The 95th percentile time interval, in milliseconds, that the activity stayed in the schedule state.
shown as millisecond
|
aws.states.activity_schedule_time.p99
(gauge every 60 seconds) |
The 99th percentile time interval, in milliseconds, that the activity stayed in the schedule state.
shown as millisecond
|
aws.states.activity_time
(gauge every 60 seconds) |
The average time interval, in milliseconds, between the time the activity was scheduled and when it was closed.
shown as millisecond
|
aws.states.activity_time.maximum
(gauge every 60 seconds) |
The maximum time interval, in milliseconds, between the time the activity was scheduled and when it was closed.
shown as millisecond
|
aws.states.activity_time.minimum
(gauge every 60 seconds) |
The minimum time interval, in milliseconds, between the time the activity was scheduled and when it was closed.
shown as millisecond
|
aws.states.activity_time.p95
(gauge every 60 seconds) |
The 95th percentile time interval, in milliseconds, between the time the activity was scheduled and when it was closed.
shown as millisecond
|
aws.states.activity_time.p99
(gauge every 60 seconds) |
The 99th percentile time interval, in milliseconds, between the time the activity was scheduled and when it was closed.
shown as millisecond
|
aws.states.activitys_failed
(count every 60 seconds) |
The number of activities that failed.
|
aws.states.activitys_heartbeat_timed_out
(count every 60 seconds) |
The number of activities that were timed out due to a heartbeat timeout.
|
aws.states.activitys_scheduled
(count every 60 seconds) |
The number of activities that were scheduled.
|
aws.states.activitys_started
(count every 60 seconds) |
The number of activities that were started.
|
aws.states.activitys_succeeded
(count every 60 seconds) |
The number of activities that completed successfully.
|
aws.states.activitys_timed_out
(count every 60 seconds) |
The number of activities that were timed out on close.
|
The core Datadog-AWS integration pulls data from AWS CloudWatch. At a minimum, your Policy Document will need to allow the following actions:
Note that these actions and the ones listed below are included in the Policy Document using wild cards such as List* and Get*. If you require strict policies, please use the complete action names as listed and reference the Amazon API documentation for the services you require.
By allowing Datadog to read the following additional endpoints, the AWS integration will be able to add tags to CloudWatch metrics and generate additional metrics.
CloudTrail also requires some s3 permissions to access the trails. These are required on the CloudTrail bucket only
The main use of the Resource Group Tagging API is to reduce the number of API calls we need to collect custom tags.
For more information on Tag policies, review the documentation on the AWS website.
When using the AWS integration, we’re pulling in metrics via the CloudWatch API. You may see a slight delay in metrics from AWS due to some constraints that exist for their API.
To begin, the CloudWatch API only offers a metric-by-metric crawl to pull data. The CloudWatch APIs have a rate limit that varies based on the combination of authentication credentials, region, and service. Metrics are made available by AWS dependent on the account level. For example, if you are paying for “detailed metrics” within AWS, they are available more quickly. This level of service for detailed metrics also applies to granularity, with some metrics being available per minute and others per five minutes.
On the Datadog side, we do have the ability to prioritize certain metrics within an account to pull them in faster, depending on the circumstances. Please contact [email protected] for more info on this.
To obtain metrics with virtually zero delay, we recommend installing the Datadog Agent on those hosts. We’ve
written a bit about this here, especially in relation to CloudWatch.
CloudWatch’s api returns only metrics with datapoints, so if for instance an ELB has no attached instances, it is expected not to see metrics related to this ELB in Datadog.
When the Cross-Zone Load Balancing option is enabled on an ELB, all the instances attached to this ELB are considered part of all A-Zs (on CloudWatch’s side), so if you have 2 instances in 1a and 3 in ab, the metric will display 5 instances per A-Z.
As this can be counter-intuitive, we’ve added a new metric, aws.elb.host_count, that displays the count of healthy instances per AZ, regardless of if this Cross-Zone Load Balancing option is enabled or not.
This metric should have value you would expect.
When installing the agent on an aws host, you might see duplicated hosts on the infra page for a few hours if you manually set the hostname in the agent’s configuration. This second host will disapear a few hours later, and won’t affect your billing.
