The complete guide

Do you think your cloud costs are too high? I think most businesses can find savings if they look in the right places. In this article I will provide a complete guide to reducing your AWS costs covering five different, but equally important perspectives:

This article will be somewhat long, but rather than giving you “10 tips to reduce your AWS costs”, I’ll instead provide you with a comprehensive, end-to-end guide covering every perspective that should be addressed to truly optimise your AWS cloud costs.

Perspective 1: Architecture

In this perspective, we’ll look at how to architect for improved cost efficiency.

Select the right architecture

There’s a lot of architectural styles to choose from. Carefully select the most appropriate style and don’t let emotion influence your choice. You may love the idea of microservices on Kubernetes, but there may be another architecture that’s more suitable.

Select the right services

As with architecture styles, there are many AWS services to choose from, and selecting the wrong one can really increase your costs. Maybe you can use Lambda instead of an EC2 instance or swap an RDS database for a DynamoDB table. Carefully compare and choose the right service for the job.

Scale applications based on demand

Use Auto Scaling to scale your application based on demand. Consider how you can optimise your scaling types and policies. Auto Scaling is one of the most powerful capabilities for cost control on AWS so use it whenever possible.

Use the well-architected framework

The AWS Well-Architected Framework (WAF) provides best practice guidance on how to properly architect AWS cloud solutions. It’s comprised of five pillars, one of which is Cost Optimisation. The guidance provided in the framework is easy to understand and very sensible. Try to implement as much of the framework as possible and consider getting a Well-Architected Review (WAR) from an AWS partner to identify any gaps.

Avoid excessive security or resilience

I know what you’re going to say, security and resilience are important. Absolutely, but it’s important to take a balanced view of security and resilience requirements against the cost of a solution. Do you really need 5 layers of firewall appliances in front of that meme-generating web app?

Make cost-efficiency an architecture principle

Is cost treated with the same priority as performance, resilience, or security? You should address cost efficiency in every architecture. At a minimum, consider including a cost forecast, elements that will influence cost and a description of how cost will be monitored and managed for every solution.

Implement an effective tag strategy

To be blunt, you have zero chance of effectively managing your cloud costs without a good resource tagging strategy. Resource tags enable you to accurately analyse and allocate costs across business, units, environments, apps, components, etc. Spend the time to define an effective tagging strategy that makes sense then implement and enforce it.

Perspective 2: Operations

In this perspective, we’ll explore some important cost-related operational processes.

Cost optimisation roles and responsibilities

Like any other operational process, cost optimisation needs ownership and clearly defined roles and responsibilities. Without this, it’s less likely that cost optimisation activities will be prioritised or even performed at all.

Chargeback and showback

Nothing makes people care about costs more than making them pay for them. Chargeback is a well-understood but rarely implemented IT financial process that allocates costs directly to a consuming business unit’s cost centre. It’s a very effective way of changing business unit’s behaviour towards cloud costs but can be tough to implement for a variety of reasons. At the very least you should have a showback system in place to report on each business unit’s cloud costs.

Cost analysis and reporting

You can’t optimise your cloud costs if you don’t understand them. You should analyse and report on your cloud costs on a regular basis. You can use native or 3rd party services (described in Perspective 4 — Cost Management) for this purpose. Reports should be made easily accessible and shared widely.

Cost and utilisation monitoring

You should be monitoring cost the same way that you monitor performance and capacity. Consider adding cost metrics to your dashboards and monitoring solution. Your operations team should watch any unexpected cost increase or decreases. Ideally, they should also be monitoring utilisation and asking questions when they see resources with low utilisation.

Application lifecycle management

An Application Lifecycle Management (ALM) framework puts structure around the activities that occur over the life of an app. For cost optimisation, we’re interested in what happens at the end of this lifecycle — decommissioning. It’s common to find resources running for a retired app. If a full-blown ALM framework seems like overkill for you, make sure you at least have a process in place to remove resources that are no longer needed.

Hunt for zombies and orphans

You should regularly hunt for zombies, idle and orphaned resources and terminate them. Several native and 3rd party services (described in Perspective 4 — Cost Management) can help identity these resources. Operations teams executing this process should be empowered to act upon their findings.

Continuous improvement

Taking an iterative, continuous improvement approach to cost optimisation can be extremely effective. Teams should be encouraged to identify and implement changes that improve cost efficiency. Improvements should be shared across teams, implemented more broadly and fed back into architecture patterns and standards.

Perspective 3: Pricing Models

In this perspective, we’ll look at a few AWS pricing models and discounts that can reduce your costs.

Savings Plans

The new kid on the AWS “get a discount for committed spend” block, Savings Plans provide a solid discount across a range of services when you commit to a minimum period of use. Savings Plans come in two flavours: Compute Savings Plans offer lower discounts (up to 66%) but are more flexible and EC2 Instance Savings Plans offer higher discounts (up to 77%) but are less flexible. Savings Plans are simpler than their predecessor, Reserved Instances, but still require some careful thought when purchasing.

Savings Plans will apply discounts in a way that maximises savings, first within the account that owns the plan and then within other accounts in the same AWS Organisation. For this reason, you should purchase plans in your master account or an account that doesn’t contain any resources to maximise your savings.

You should purchase Savings Plans to match your minimum consistent monthly spend, giving some consideration to any forecasted increase or decrease. Buy your plans in small increments and try not to overdo it. If you’re not sure whether you will use a plan you don’t buy it. You can see savings plan utilisation, coverage and purchasing recommendations in the Cost Explorer Savings Plans console.

Reserved Instances

Reserved Instances are more complex to manage but are still required for some services like RDS, ElastiCache and RedShift that aren’t covered by Savings Plans yet. There are a few different types of reserved instances, each with varying discount-levels, restrictions and characteristics.

You should only purchase new Reserved Instances for resources that aren’t supported by Savings Plans yet. As with Savings Plans, purchase in increments based on your minimum consistent spend, forecast carefully and don’t overdo it.

Spot Instances

Spot Instances offer a discount if you volunteer to keep AWS’s spare compute warm and can deal with having your instance terminated. Spot Instances offer tight integration with EC2 Auto Scaling, ECS, EMR and Batch plus a slew of flexible configuration options.

Spot Instances provide a discount up to 90% off On-Demand pricing, though usually, pricing falls somewhere between On-Demand and Savings Plans pricing. You should give Spot Instances some consideration for any variable capacity or transient workloads.

Enterprise Discount Program

AWS Enterprise Discount Program (EDP) is a private, negotiated pricing agreement between AWS and a customer. It provides a discount based on an organisation’s committed spend over a set term, for example, an 18% discount for spending $10M over 3 years. Just make sure you understand that you’re committing to that spend, because use it or not, you’ll pay that amount.

Funding and Credits

AWS offers a range of funding programs and credits either directly to customers or through an AWS partner. You should work closely with your AWS partner and/or AWS account team to find out if you might be eligible for any funding programs or credits.

Perspective 4: Cost Management

In this perspective, we’ll explore some cost management services that can help manage your costs.

AWS Cost Explorer

Cost Explorer is an easy to use, free service that can help analyse and manage your AWS costs. It lets you dig into your cloud costs by account, service, tags, etc. You should use it to understand your cost analysis activities and to investigate any unusual or unexpected costs.

AWS Cost and Usage Reports

You can get extremely detailed cost data using Cost and Usage Reports (CUR). Reports can be output to S3 or obtained using the CUR API. Reports are broken down by products, resources, usage types, operations and tags. This data can be used to perform cost analysis and reporting.

AWS Budgets

Budgets allows you to setup cost or usage budgets that, when breached or forecasted to breach, will notify you. Budgets can be configured for cost, usage, Reserved Instances and Savings Plans. They can be centralised in your AWS Organizations master account and created as code. You should be creating budgets based on your expected cost and usage profile.

AWS Trusted Advisor

Trusted Advisor includes cost optimisation checks. It can identify underutilised, unassociated and idle resources and provides Savings Plans and Reserved Instance recommendations. You should regularly review its findings to identify waste. Multi-account environments can use Systems Manager Explorer to get a consolidated view of findings across accounts.

AWS Pricing Calculator

Use the Pricing Calculator when (re-)architecting your apps. Many services have complicated pricing that you may not fully appreciate until you’re stepping through the pricing calculator fields. Consider making pricing calculator estimates a key part of your architecture and design processes.

AWS Compute Optimizer

Compute Optimizer is a standalone service that can help identify underutilised instances and offer a right-sizing recommendation. It works by analysing instance specifications and performance metrics over time. It’s a free service, so you should opt-in and start reviewing your recommendations.

3rd Party Cost Management Services

Several 3rd party services may make your cost management activities a lot easier. These services can consolidate the capabilities of AWS-native tools into a single interface. They also provide a range of additional capabilities like multi-cloud support, cost-allocation and chargeback mechanisms, unique recommendation engines, better reporting and tag governance. You should investigate whether the benefits of a cost management service might significantly outweigh the cost.

Perspective 5: Service Optimisation

In this perspective, we’ll look at how to optimise the cost of some common AWS services.

Right sizing

Applies to: Most AWS services.

As most services can be easily scaled up or out, there’s no reason to oversize your workloads. Base your sizing on estimates and testing and then monitor and resize if you need to. Continually review resource utilisation and right size anything that’s underutilised.

Auto scaling

Applies to: EC2, ECS, DynamoDB, Aurora, RDS storage and a range of other services.

Use auto scaling whenever possible. Configuring your resources to scale to meet demand will result in significantly lower costs. This is a fundamental cloud capability that you really should be using as often as possible to balance performance and cost.

Automatic stop/start

Applies to: EC2, RDS, RedShift.

Consider automatically stopping instances and databases when they’re not needed. Running an on-demand instance 8 hours a day costs roughly 50% less than the same instance covered by a Savings Plan running 24 hours a day. Carefully consider whether you have applications, environments or resources that could be run on a scheduled basis.

Software licensing

Applies to: EC2.

Investigate whether you have unused Windows (or other) OS licenses that may be eligible for cloud use. You can then Bring Your Own License to AWS to avoid paying for AWS-provided licensing. Also, be aware of AWS Dedicated Hosts if you happen to have any legacy, cloud-unfriendly licensing that might be causing you licensing headaches.

Lambda tuning

Applies to: Lambda.

Lambda is a great service, truly a game changer for many applications. However, it’s not the easiest service to tune for cost efficiency. Start by making sure your app logic is sound, minimise invocations where you can and avoid having functions waiting around for things. Monitor your run durations and memory usage and tune accordingly. Lastly, if your use case doesn’t make sense for Lambda, don’t force it, EC2 isn’t that uncool and you can always impress your friends by using containers.

DynamoDB tuning

Applies to: DynamoDB.

DynamoDB is another great service that can also be a bit more challenging to tune for cost. Start by making sure you carefully plan your primary keys and any secondary indexes to avoid excessive and difficult queries. Use provisioned capacity mode unless your use case really needs on-demand. Tune your RCU / RWU values carefully and use auto scaling to adjust these based on demand.

S3 object classes (tiers)

Applies to: S3.

Use S3 object classes (tiers) to reduce your storage cost for infrequently accessed or archive data. If you haven’t got a good handle on your data lifecycle then consider using intelligent tiering to figure it out for you.

S3 storage lifecycle

Applies to: S3.

If you have data with a predictable lifecycle then you should configure S3 storage lifecycle rules to automatically manage object storage classes and potentially delete objects when they’re no longer required.

EBS volume types

Applies to: EC2/EBS.

Plan your EBS volume type and size carefully. You may be able to reduce your EBS storage costs by using larger General Purpose (GP2) volumes instead of smaller Provisioned IOPS (IO1) volumes. If you are feeling adventurous, you can also create RAID volumes across several GP2 volumes to get greater performance at a lower price than IO1 volumes.

EBS snapshot lifecycle

Applies to: EC2/EBS.

You should actively manage the lifecycle of EBS snapshots to ensure you’re not paying for snapshots that are no longer required. You can use Data Lifecycle Manger (DLM) to simplify and automate the management of snapshots. You can also use AWS Backup for more complex retention requirements that are difficult to achieve using snapshots directly.

RDS database engine

Applies to: RDS.

Consider using Aurora, MySQL and Postgres in place of Oracle or Microsoft SQL RDS instances. There is a dramatic price difference between these different engines which may justify the cost of a DB re-platform project.

Aurora serverless

Applies to: Aurora.

If you have a database that only handles infrequent queries or has unpredictable load you should look at Aurora Serverless. It may provide better performance and cost for these types of database workload.

Parameter Store vs. Secrets Manager

Applies to: Secrets Manager, Systems Manager.

Secrets Manager is awesome, but if you don’t need secret generation, automatic-rotation or cross-account access then you should consider storing your secrets in Parameter Store which is free up to 10,000 parameters.

Direct Connect and VPN

Applies to: Direct Connect, Managed VPN.

Keep your Direct Connect and VPN connections to a minimum. Look for and terminate connections that are no longer required. Make sure that you size your Direct Connect connections appropriately.

Transit Gateway and VPC Peering

Applies to: Transit Gateway, VPC Peering.

If you have many VPCs and/or Direct Connect or VPN connections, you should consider using Transit Gateway to consolidate your network connectivity and NAT gateways. This will not only reduce your costs but will also simplify your network architecture. If you only have a handful of VPCs, consider using VPC Peering instead of Transit Gateway as it will cost you less.

NAT Gateways and NAT Instances

Applies to: NAT Gateway, NAT Instances.

NAT Gateways are nice and simple but can start costing a lot if you have many of them. Depending on your architecture, you might be able to reduce the number of NAT Gateways you’re running or switch to cheaper NAT Instances or even use public subnets.

CloudWatch Optimisation

Applies to: CloudWatch.

For CloudWatch Logs, only ingest logs that you need and configure retention settings so that you’re not indefinitely storing large volumes of logs. Disable detailed monitoring on resources when it’s not required. Clean up any unnecessary custom metrics, alarms and events.

VPC Endpoints

Applies to: VPC.

Gateway Endpoints provide access to S3 and DynamoDB and are free. Interface Endpoints provide access to a range of other services but are most certainly not free. A few interface endpoints aren’t a big deal, but the cost of many interface endpoints adds up quickly. You should consider centralising your Interface Endpoints using Transit Gateway or skip them altogether and use NAT Gateways or public subnets instead.

Internet egress, cross-region and cross-AZ traffic

Applies to: VPC.

Network traffic costs on AWS are infamously complex. At a high-level, keep in mind that you will pay for traffic that leaves AWS (Internet egress) or goes between regions or availability zones. Plan the placement of workloads to avoid racking up large inter-region / AZ charges and do whatever you can to optimise your egress charges. You should consider using CloudFront to serve content, use Direct Connect or the Snow family for bulk data transfer, use HTTP compression and architect apps to limit unnecessary network traffic.

Conclusion

I hope that you’ve seen there’s a lot you can do to reduce your AWS cloud costs. Organisations are generally quite good at optimising AWS services that they’re familiar with and using flexible pricing models. However, they sometimes forget about some of the bigger picture things like architecture, operations, and cost management services. Try not to neglect any of these perspectives as you’ll be leaving money on the table.

Cost optimisation isn’t easy, and it’s a job that’s never done. You’ll become more successful at cost optimisation the more you do it. You don’t need to go it alone, reach out for help if you need to, as AWS and AWS partners both want you to get maximum value out of the AWS cloud.