6 Essential Ingredients For AWS Observability

August 9, 2021

1. Data collection from any service or workload

First and foremost, complete AWS observability requires the ability to collect data from any AWS service, as well as any application or other workload running on top of it. No matter how the data is formatted or where AWS stores it by default, it should be easy to ingest it into a centralized observability tool.

2. Data correlation

Collecting data is only half the observability battle (if that). The real value comes from correlating multiple data sources together to understand the total state of your AWS environment. Effective AWS observability entails the ability to map relationships between all of the services and workloads running in your cloud and understand how they impact overall performance.

3. Integration with external data

Even if all of your primary workloads run in AWS, you very likely have some resources or tools that run externally. You may have a CI/CD pipeline that deploys to AWS but is hosted elsewhere. Or, you may have a ticketing system that helps you support applications hosted in AWS, but that does not itself run in AWS.

Complete observability requires the correlation of these outside data sources with data from AWS. You need to know how ticketing requests relate to Lambda performance trends, or how application deployment frequency impacts EC2 resource utilization.

4. Track historical state

Sometimes you need to know what happened within your AWS environment in the past, and how it relates to performance in the past. Your AWS observability tools should provide the option of reconstructing the historical state of your environment and tracking performance trends over time.

5. Analyze, don’t just monitor

AWS observability is not about just monitoring what happens, but also about using analytics to understand why it happened, and what its impact was. This is why you need to be able to shape and link together data from every resource and analyze it collectively — while at the same time retaining the ability to drill down into the behavior of individual services or applications when necessary.

6. Simple data ingestion

Finally, ingesting data into AWS observability tools shouldn’t be a nightmare. You should be able to set up straightforward data collectors that efficiently move data in real-time into your analytics tools. In other words, you should be able to focus on understanding your observability data, not on collecting and managing it.

This is an excerpt for a longer white paper “An AWS Observability Strategy That Actually Works.” you can download the full piece here.