AWS RDS is a cloud relational database. It’s suitable for both early-stage startups and Web-Scale companies. One catch is that it’s possible to irreversibly loose all your data if you’re using it. In this blog post, I’ll describe why the default backup system offered by RDS is insecure and how to improve it.
One simple trick to kill your startup…
It’s about as “clickbaity” as it can get but bear with me.
RDS offers a robust backup system. Every database has a continuous backup allowing you to revert it to a specific point in the past. Additionally, point-in-time snapshots are automatically created every day and whenever you perform an action like upgrading the database, etc.
So, where’s the catch here?
Your account. All the backups are stored within a single account. Unless you’re a company with mature security practices, I guess that all your developers use AdministratorAccess or PowerUserAccess IAM policies. It’s not straightforward to fine-tune IAM policies for every single developer, especially when working on more obscure parts of the AWS stack. It’s understandable that unless you’re hiring a full-time AWS security specialist applying admin access for every developer is a sane default.
Why is that an issue?
Everyone with admin access to AWS CLI or console can easily remove all the backups. For most companies losing all data would probably mean that everyone can pack their bags and go home.
Let that sink in. EVERYONE in your company with AWS admin access can IRREVERSIBLY remove your database and all its backups. Every developer, support agent, an external consultant with admin access is a single point of failure that can destroy your startup. Imagine the following scenarios:
- CLI credentials are leaked to a public GitHub repo
- dev mistakes staging for production environment during the cleanup
- the fired developer decides to take the company down with him
- admin’s cat walks on a keyboard and triggers a destructive shell script
- …
The list goes on and on. Better safe than sorry. Let’s learn how you can significantly improve the security of RDS backups.
How to add redundant backups to RDS database [free eBook]
The good news is that you can protect yourself from these tragic scenarios. Here’s how:
- Periodically upload the newest database dump to an S3 bucket in your primary account.
- Replicate data to a bucket in a separate AWS account that only necessary users can access.
- Profit.
With the following mechanism in place, even if your primary account is compromised and your data eradicated, you can still restore your project.
You can grab a free eBook describing step-by-step on how to set it up. You’ll receive a link to the eBook just after subscribing to my newsletter.
This eBook covers backups for PostgreSQL, but with few changes to the backup script, it is applicable to all the database engines.
Summary
From my experience, unfortunately, teams only become more security-aware after the actual incident. Secondary “panic” backups is a simple best practice that can save your company from disaster.
