Pattern: Object storage lifecycle rules
Define lifecycle rules for object storage to move objects to cheaper storage or drop them entirely.
Context
By default, objects stored in cloud object storage are retained, and therefore billed, indefinitely. Objects also have a storage class or access tier, which can be used to balance access performance and cost depending on the use case.
Solution
By configuring lifecycle rules or policies, objects can be transitioned to cheaper storage classes or deleted after a certain amount of time.
Example
Transition objects under the "log/" prefix to the Glacier storage class after 60 days, and expire after 90 days:
resource "aws_s3_bucket_lifecycle_configuration" "example" {
bucket = aws_s3_bucket.bucket.id
rule {
id = "log"
expiration {
days = 90
}
filter {
prefix = "log/"
}
status = "Enabled"
transition {
days = 60
storage_class = "GLACIER"
}
}
}
References
- Resource: aws_s3_bucket_lifecycle_configuration
- Resource: google_storage_bucket (Terraform) - Life cycle settings
- Resource: azurerm_storage_management_policy (Terraform)
Occurrences
Terraform
- https://github.com/alphagov/govuk-aws/commit/f844cd8e254b161bebef04101f8ce177bcd0840c
- https://github.com/alphagov/govuk-terraform-provisioning/commit/ac105ab0a5ae38fbf69167e072f8970a4a61c3e8
- https://github.com/ExpediaGroup/apiary-data-lake/commit/47e62f2fc73a96611606cd619c084d1ded9d844d
- https://github.com/SamTowne/BasketballDrillBot/commit/4ec6d54e4d36ab02b0a7daf042e727717371eaec
- https://github.com/utilitywarehouse/tf_telecom/commit/17007456991c1a8faa26b1f4ac993883f577d124
- https://github.com/trajano/terraform-s3-backend/commit/cb9f00a2f6f23b44f7db08863ef5fb0b9ea0bc0c
AWS CloudFormation
- https://github.com/Sage-Bionetworks/service-catalog-library/commit/7d5224b90cf2c47eee807f851dbf0ff14d051191
- https://github.com/jamesoff/cloud/commit/9ac70dc914bef24c62e2b603cc03a0eaefaed391
- https://github.com/gbourniq/django-on-aws/commit/32a058dd02229c8bf8e272f549f60c02bf2009ce
- https://github.com/Jessinra/TriveryID-Skadi/commit/d91bc4e786a86f7c088413ac1f7ba701cb7e499f
- https://github.com/wolfeidau/wolfeidau_site/commit/52de425e9eedfae9b51a29366a4b6aefcca51658
- https://github.com/kriation/cloudformation/commit/daaea80eaa33ca0041a76ef06baee031bd4ffaf4
- https://github.com/gbourniq/django-on-aws/commit/4400bcccee4356214d332b9549efcae4e74458be
- https://github.com/chef-boneyard/aws_native_chef_server/commit/f63839453506420064afb7350971c65be47d39a6
- https://github.com/m-chandler/factorio-spot-pricing/commit/7290615791393f15f636eb616e76ae7e7d2384c5
- https://github.com/adamjreed/dogchecker/commit/6fe5fba49531813b2c473d3805a9a17075e41fff
- https://github.com/h1f1x/aws-cfn-examples/commit/b157e0f038a8acb4d917f9e18d0adddd7cd4192c
- https://github.com/sjakthol/cfn-execute-change-set/commit/7aa1557a573a4d44acec840698e34f41efc29df0
- https://github.com/widdix/aws-cf-templates/commit/03a8713f53a4a514898bd481eb6628b6b56cfff1