Introduction to Amazon CloudWatch

Amazon CloudWatch is a log monitoring and management service from Amazon Web Services (AWS). It’s designed to provide DevOps engineers, site reliability engineers (SREs), IT administrators, and software engineers access to metrics, data, and statistics to help understand their operations. 

CloudWatch Features 

Amazon CloudWatch features fall into four primary categories, which we’ll discuss below.

Logs and Metrics Collection and Analytics

Amazon CloudWatch allows users to gather metrics and logs. The types of logs supported by CloudWatch fit into three primary buckets. Vendor logs are the first bucket supported and published by AWS on behalf of the customer. Currently, AWS supports Amazon VPC Flow Logs and Amazon Route 53 logs. The second type of logs supported by CloudWatch are logs published by AWS Services, including AWS Lambda, API Gateway, CloudTrail, and others. The third bucket is custom logs.

CloudWatch also allows users to collect and aggregate pre-ordained and custom metrics. The built-in default metrics pull from AWS services, including AWS Lambda, API Gateway, EC2, S3, ECS, and DynamoDB. Users are not limited to default metrics, they can also collect performance metrics and event tracking metrics.

Monitoring and Views

Amazon CloudWatch offers basic dashboards, alarms, and other monitoring insights that allow users to see, organize, and act upon their logs and metrics.

Dashboards help users to visualize cloud resources and cloud applications. Users can graph data to help diagnose the root causes of problems or contextualize system-wide issues for systems health monitoring and troubleshooting.

While dashboards provide actionable intelligence, Amazon CloudWatch also can be set with alarms to flag specific default or custom-dictated occurrences and anomaly detection to help notify the necessary parties in the event of an issue. Alarms group to reduce noise in the event of a more significant problem and can vary for specific thresholds of metrics based on an environment’s specific resources. 

Insights and dashboards for container monitoring insights and Lambda monitoring insights offer real-time and logged tracing of performance. Tools such as CloudWatch ServiceLens, which visualizes the performance of applications, and CloudWatch Synthetics, which monitors application endpoints, provide visual insights into traffic and infrastructure.

Action and Automation

Amazon CloudWatch allows for the automation of capacity planning through Auto Scaling, where custom-set thresholds can cue responses for scaling that can optimize resource use or minimize costs. CloudWatch Events can automate corrective action through rule-based event matching and automation attached to alarms can automate actions.

Compliance

Depending on the audit periods, data generated through CloudWatch can be retained for set timeframes, assuring that when logs need to be accessed or audited for compliance, the information is there.

CloudWatch Pricing

Amazon CloudWatch offers both a free-tier and a premium paid tier that is calculated based on usage.

Free Tier

Unlimited basic monitoring metrics come at a 5-minute frequency within the free tier, including all non-custom events. Premium and tailored features change at specific thresholds.

Premium/Paid Tier

With the paid tier, users pay for utilization every month. The first 10,000 metrics are billed at $.30/mo, after which the next 240,000 metrics are $.10/mo, the next 750,000 at $.05/mo, and anything upwards of 1,000,000 metrics is billed at $.02/mo. Up to 5 statistics for the same metric in single GetMetricData API requests get included. Afterwards, the bill starts at $.01 per 1,000 metric requests, except GetMetricWidgetImage metrics, which are $.02 per 1,000 requests. Metric streams are available at $.003 per 1,000 metric updates. 

Dashboards cost $3 per dashboard per month. Alarms range in price from $.10 to $.90 per alarm, depending on the resolution and nature of the alarm. 

There is no cost for log data transfer, but data transfer out is priced depending on where and how much data goes. Collection is billed at $.50/GB, Storage at $.03/GB, and Analysis at $.005/GB. Vendor logs are billed at tiers up to 10TB..

Events bill users at a rate of $1 per million events and $1 per million cross-account events.

CloudWatch Contributor Insights for Cloud Watch bill at $.50/rule per month and $.02 per one million log events that match the rule. CloudWatch Contributor Insights for DynamoDB also bills at $.50/rul per month, but events bill at $.03 per one million logs that match the event per month.

Canaries run at $.0012 per canary run, although they may incur additional charges for other AWS services that are also utilizing them.

CloudWatch Limitations

CloudWatch is optimized for AWS logs. Although it supports log ingestion from sources outside of AWS, you can only ingest them using their agent. It doesn’t support ingestion via Syslog, APIs, or code libraries. Depending on your needs, there is a lot of customization work required to set it up, which may require an expert on your team to get started. Once you’ve managed to get your logs into the service, searching them in the CloudWatch user interface (UI) and command line interface (CLI) is tedious and complex. Other expected features like data visualizations and alerts are limited in CloudWatch. And, it is missing integrations with commonly used tools like Slack and PagerDuty and doesn't support Webhooks, making it difficult for engineers to receive notifications of issues when they arise.